Update a Library variable from a pipeline in Azure DevOps

Update a Library variable from a pipeline in Azure DevOps

If you’re using Azure DevOps for your automation, you may use Variable Groups to manage your configuration in a centralized and secure way. Reading a variable from a pipeline is easy but updating its value can quickly be messy and complicated…

Hopefully, there are easy ways to update a variable value and they will be described in this post!

Example of a Variable Group in Azure DevOps (Pipelines > Library section)

Context

First of all, let’s be clear on one thing: you cannot update the value of a variable with a simple call from a task in Azure pipeline, it is not possible.

The only solution is to use the Azure DevOps API, and it’s ok to do so! After all, Azure DevOps is basically a front-end on top of Azure DevOps API.

The keystone here is given by one of the predefined variable from an Azure pipeline: the almighty System.AccessToken.

Setup: Permission first

As explained in the documentation, you’ll have to do some admin actions in Azure DevOps for the API call to work.

  • In your “Project Settings > Settings”, make sure “Limit job authorization scope to current project” is not checked. (If you can’t edit it, you have to uncheck it in your “Organization Settings > Settings”)
Project > Settings > Pipelines > Settings
  • In your Variable Group > Security, add “Administrator” Role to “Project Collection Build Service (xxxx)” (xxxx being your project name)
Project > Pipelines > Library

Solution 1: Use an extension

There is an Azure DevOps free extension that relates to this action: Shared variable updater. Source code is available on GitHub: lanalua/azure-pipeline-variables-updater.

It works with hard coded values but not if the new variable value to be set is inside a pipeline variable unfortunately.

Failed

Solution 2: DIY with PowerShell

One way to update a variable is to use some PowerShell code that will make the call to Azure DevOps API.

In your Azure pipeline (yaml or classic or release), add a PowerShell task (Type: Inline). This will work on all types of platforms: Linux, Windows, macOS.

Then copy/paste the following code by updating the key and value for the variable.

$VariableGroupId = $(variable.group.id)
$NewValue = $(variablenewvalue)
$VariableName = $(variablename)

Write-Host "NewValue : $NewValue"

$url = "$($env:SYSTEM_TEAMFOUNDATIONCOLLECTIONURI)$env:SYSTEM_TEAMPROJECTID/_apis/distributedtask/variablegroups/$($VariableGroupId)?api-version=5.1-preview.1"

Write-Host "URL: $url"

$authHeader = @{Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"}

$definition = Invoke-RestMethod -Uri $url -Headers $authHeader

Write-Host "Pipeline = $($definition | ConvertTo-Json -Depth 100)"

$definition.variables.$VariableName.Value = "$($NewValue)"

$definitionJson = $definition | ConvertTo-Json -Depth 100 -Compress

Invoke-RestMethod -Method Put -Uri $url -Headers $authHeader -ContentType "application/json" -Body ([System.Text.Encoding]::UTF8.GetBytes($definitionJson)) | Out-Null

Ok but…

Solution 3: Use an existing tool

The code used in Solution 2 works but it’s not simple. There is another option, which comes from an open source tool: almops.

You only need to have the .NET Core SDK (which comes by default in Azure DevOps self-hosted agents).

Here is the YAML definition:

steps:
- task: UseDotNet@2
  displayName: 'Get latest .NET SDK'
  inputs:
    version: '3.1.300'
- task: DotNetCoreCLI@2
  displayName: 'Install almops .NET global tool'
  inputs:
    command: custom
    custom: tool
    arguments: 'install --global almops'
- bash: |
   almops config --org myorgname --token $(System.AccessToken)
   almops update variables -p $(System.TeamProject) --id $(variable.group.id) --type group --var mynewvariable
  displayName: 'Update XXX value'

The same tasks can work also on Release pipelines (classic editor):

In this case (classic editor), it’s very important to check the Agent “Allow scripts to access the OAuth token” option:

Et voilà!

Azure DevOps is really an incredible tool and if you’re limited by the UI, have a look at its API 😉

References:
Use the OAuth token to access the REST API
Build job authorization scope

bertrand

7 thoughts on “Update a Library variable from a pipeline in Azure DevOps

  1. I’m adding this comment and link because, even though this post is a year old, it came up high in my search results for a way to update group variables from a pipeline. Since this was posted, there is now another pipeline extension that is specifically for creating, updating, and/or deleting variable groups. It’s ManageVariableGroupTask by Moreno Bruschi, available in the Marketplace here: https://marketplace.visualstudio.com/items?itemName=MorenoBruschi.manage-variable-group-task

  2. I’m getting some value property not found on $definition but the json does have it. Logs is shown below.

    NewValue : b
    2023-08-13T22:33:03.5139841Z URL: https://dev.azure.com/*********/**************/_apis/distributedtask/variablegroups/27?api-version=5.1-preview.1
    2023-08-13T22:33:03.5142312Z URL: ***
    2023-08-13T22:33:03.5142718Z Pipeline = {
    2023-08-13T22:33:03.5142883Z “id”: 27,
    2023-08-13T22:33:03.5143025Z “name”: “dev-updatevariable”,
    2023-08-13T22:33:03.5143218Z “isShared”: false,
    2023-08-13T22:33:03.5143391Z “variableGroupProjectReferences”: null
    2023-08-13T22:33:03.5143553Z }
    2023-08-13T22:33:03.6168636Z The property ‘Value’ cannot be found on this object. Verify that the property exists and can be set.
    2023-08-13T22:33:03.6169109Z At D:\a\_temp\4f341332-bcd5-404a-b652-e2912e4f648a.ps1:21 char:1
    2023-08-13T22:33:03.6169332Z + $definition.variables.$VariableName.Value = “$NewValue”
    2023-08-13T22:33:03.6169656Z + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    2023-08-13T22:33:03.6169876Z + CategoryInfo : InvalidOperation: (:) [], ParentContainsErrorRecordException
    2023-08-13T22:33:03.6170151Z + FullyQualifiedErrorId : PropertyNotFound
    2023-08-13T22:33:03.6170831Z

  3. It turned out to be permission issue. I switched to use PAT in place of system access token and it works. Thoughts?

    I hate using PAT so please advise.

    1. Hi Manohar, Personal Access Tokens is a secure way in many systems to validate actions. I also use it with GitHub, Gitlab for example.
      Giving the right name, permissions and expiration date ensure the right administration of PAT.
      Don’t know other way to do it with Azure DevOps.

      1. Well PATs are always having the limited validity and need to be updated manually so not a suit for automation 🙁

        1. Depending on the tool, sometimes it’s possible to have a PAT without an expiration date.
          I use them for automation and I get reminder email before the expiration (or I set one myself). That’s the only tradeoff I can see.
          Usually we can see the last time a PAT has been used, it’s quite convenient.
          Storing them in Library / Variables in Azure DevOps is easy, stored only once and their secret value protected.

Leave a Reply

Your email address will not be published. Required fields are marked *