This content not available without scripts

YellowDog Dec-2021 Release Notes

Home >YellowDog Dec-2021 Release Notes

Component Versions

Platform: 3.4
Portal: 3.4

YellowDog Platform Dec-2021

New Features and Improvements for the YellowDog Platform

Auto Scaling Worker Pools

If the platform has more waiting Tasks than there are Nodes available to run them, the Scheduler may now automatically request that the Compute Service provision more instances. Conversely, if there are no tasks waiting it may shut down idle Nodes to prevent unnecessary usage. This autoscaling behaviour can be controlled by a small number of settings on Worker Pools and Task Groups.

For each Task Group, the following settings in the Run Specification control the Task Group’s demand:

  • minWorkers This many Workers must be claimed before the Task Group can run.
  • maxWorkers The Task Group will not attempt to claim more than this number of Workers.
  • tasksPerWorker This can be used to reduce demand from this Task Group by scaling the number of Workers to claim.

For each Worker Pool provisioned by YellowDog, the following properties control the Pool’s provisioning and deprovisioning of Nodes:

  • minNodes Setting a minimum number of nodes reduces latency when demand increases from a very low level.
  • maxNodes This prevents upscaling beyond the constraints of budget or available provider capacity.
  • nodeIdleGracePeriod This controls how long a Node may wait to be claimed after registration, before it enters the Idle Time Limit.
  • nodeIdleTimeLimit This controls how long a Node may be idle, following either the Grace Period or a period of activity, before it is automatically terminated.

It is possible to use these parameters to set a number of different autoscaling strategies according to your organisation’s requirements. For more information, see the Scaling the Worker Pool topic within the Workflow section of the Implementation Guide (PDF).

Custom Instance Tags

Instance tags, also sometimes called instance labels, are offered by cloud providers to enable you to categorise and manage your instances. Each instance tag consists of a key:value pair. YellowDog now enables you to specify up to 15 of these tags, which can be set at the level of Source Template, Compute Requirement Template and/or Compute Requirement as required.

YellowDog will only allow you to enter tags in a format that will be accepted by all cloud providers. This means, for example, that tags may not be longer than 63 characters or contain spaces. For full details, see the Custom Instance Tags topic within the Setting Up the Portal section of the Portal User Guide (PDF).

Extended Support for Cloud Provider Formats

YellowDog increasingly offers a ‘one stop shop’ for provisioning the right compute across multiple cloud provider interfaces. Version 3.4 introduces support for the following formats:

  • Google Cloud Bulk API
  • OCI preemptible instances

Fixes for the YellowDog Platform

  • The YellowDog SDK has been upgraded to Java 11. This means that clients using the Java SDK now require Java 11 or greater.

For more information, contact YellowDog.


YellowDog Portal Dec-2021

New Features and Improvements for the YellowDog Portal

System Dashboard

YellowDog now provides the ability to see an overview of your account activity and status at a glance.

On logging in, you will now see the new Dashboard, which summarises the status of your last 24 hours of Compute Requirements, Work Requirements and Worker Pools, as well as your recent Usage.

Example Dashboard

Clicking on any section of one of the charts opens the relevant summary page, filtered to show the items of interest. This makes it easier than ever to manage your work within YellowDog and detect and investigate any problems as they occur.

Creating Worker Pools

Following on from the introduction of Worker Pool visualisation in version 3.3, it is now possible to provision Worker Pools quickly and simply from within the YellowDog Portal.

Worker Pool provisioning pane

Provisioning through the Portal provides all the properties available for normal Worker Pools. More expert options such as the new autoscaling functions are hidden as standard and set to sensible defaults, but can still be accessed by advanced users.

It is not currently possible to configure Advanced Worker Pools through the Portal.

Creating Allowances

The Allowances menu introduced in version 3.3 now also enables users to create the most commonly used Allowances. The Portal provides the same funcationality as the SDK, including the ability to set reset intervals, grace periods and specific instance statuses that count towards the limit.

It is not currently possible to edit or boost an Allowance through the Portal.

ActiveDirectory Login

It is now possible to enable your users to log into the YellowDog Portal using your organisation’s ActiveDirectory account. You can use existing ActiveDirectory options to enable access for only specific users or groups.

In order to set up external authentication, you simply:

  • Create an app registration for YellowDog in the Microsoft Azure Portal.
  • Assign users and groups to the app.
  • Add an Authentication for ActiveDirectory in the YellowDog Portal.

Note that it is currently only possible to integrate one Microsoft Tenant ID with your YellowDog account. Additionally, each of your external users will need to create an ‘encryption password’ to use within YellowDog when managing Compute Requirements or Object Storage.

For full details and instructions, see the Setting up External Authentication topic within the Before You Begin section of the Implementation Guide (PDF)

Copying Existing Templates

The process of system setup has been streamlined by enabling you to use existing Source Templates and Compute Requirement Templates as a basis for creating new ones.

From within the details screen of any Source Template or Compute Requirement Template, You can now click the Copy option at top right to open a new Create pane with all the current values populated. Simply change the Name and any other desired settings to create your new template.

Displaying Text Files in the Object Store

When viewing item information in the Details pane of the Object Store, selecting a text file now automatically displays its content within the Details pane.

Fixes for the YellowDog Portal

  • Various changes have been made to improve the visibility of tables at smaller screen sizes.
  • When the same error is generated multiple times, the Portal no longer requires the user to view and clear it every time.
  • On the Compute Source Template screen, user data is no longer wrapped. This makes it easier to distinguish line breaks when debugging.

For more information, contact YellowDog.

YellowDog Please upgrade your browser

You are seeing this because you are using a browser that is not supported. The YellowDog website is built using modern technology and standards. We recommend upgrading your browser with one of the following to properly view our website:

Windows Mac

Please note that this is not an exhaustive list of browsers. We also do not intend to recommend a particular manufacturer's browser over another's; only to suggest upgrading to a browser version that is compliant with current standards to give you the best and most secure browsing experience.