Update a Library variable from a pipeline in Azure DevOps
If you’re using Azure DevOps for your automation, you may use Variable Groups to manage your configuration in a centralized and secure way. Reading a variable from a pipeline is easy but updating its value can quickly be messy and complicated…
Hopefully, there are easy ways to update a variable value and they will be described in this post!
Context
First of all, let’s be clear on one thing: you cannot update the value of a variable with a simple call from a task in Azure pipeline, it is not possible.
The only solution is to use the Azure DevOps API, and it’s ok to do so! After all, Azure DevOps is basically a front-end on top of Azure DevOps API.
The keystone here is given by one of the predefined variable from an Azure pipeline: the almighty System.AccessToken.
Setup: Permission first
As explained in the documentation, you’ll have to do some admin actions in Azure DevOps for the API call to work.
- In your “Project Settings > Settings”, make sure “Limit job authorization scope to current project” is not checked. (If you can’t edit it, you have to uncheck it in your “Organization Settings > Settings”)
- In your Variable Group > Security, add “Administrator” Role to “Project Collection Build Service (xxxx)” (xxxx being your project name)
Solution 1: Use an extension
There is an Azure DevOps free extension that relates to this action: Shared variable updater. Source code is available on GitHub: lanalua/azure-pipeline-variables-updater.
It works with hard coded values but not if the new variable value to be set is inside a pipeline variable unfortunately.
Failed
Solution 2: DIY with PowerShell
One way to update a variable is to use some PowerShell code that will make the call to Azure DevOps API.
In your Azure pipeline (yaml or classic or release), add a PowerShell task (Type: Inline). This will work on all types of platforms: Linux, Windows, macOS.
Then copy/paste the following code by updating the key and value for the variable.
$VariableGroupId = $(variable.group.id) $NewValue = $(variablenewvalue) $VariableName = $(variablename) Write-Host "NewValue : $NewValue" $url = "$($env:SYSTEM_TEAMFOUNDATIONCOLLECTIONURI)$env:SYSTEM_TEAMPROJECTID/_apis/distributedtask/variablegroups/$($VariableGroupId)?api-version=5.1-preview.1" Write-Host "URL: $url" $authHeader = @{Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"} $definition = Invoke-RestMethod -Uri $url -Headers $authHeader Write-Host "Pipeline = $($definition | ConvertTo-Json -Depth 100)" $definition.variables.$VariableName.Value = "$($NewValue)" $definitionJson = $definition | ConvertTo-Json -Depth 100 -Compress Invoke-RestMethod -Method Put -Uri $url -Headers $authHeader -ContentType "application/json" -Body ([System.Text.Encoding]::UTF8.GetBytes($definitionJson)) | Out-Null
Ok but…
Solution 3: Use an existing tool
The code used in Solution 2 works but it’s not simple. There is another option, which comes from an open source tool: almops.
You only need to have the .NET Core SDK (which comes by default in Azure DevOps self-hosted agents).
Here is the YAML definition:
steps: - task: UseDotNet@2 displayName: 'Get latest .NET SDK' inputs: version: '3.1.300' - task: DotNetCoreCLI@2 displayName: 'Install almops .NET global tool' inputs: command: custom custom: tool arguments: 'install --global almops' - bash: | almops config --org myorgname --token $(System.AccessToken) almops update variables -p $(System.TeamProject) --id $(variable.group.id) --type group --var mynewvariable displayName: 'Update XXX value'
The same tasks can work also on Release pipelines (classic editor):
In this case (classic editor), it’s very important to check the Agent “Allow scripts to access the OAuth token” option:
Et voilà!
Azure DevOps is really an incredible tool and if you’re limited by the UI, have a look at its API 😉
References:
– Use the OAuth token to access the REST API
– Build job authorization scope
- Install Rancher on Azure Kubernetes Service (AKS) - September 25, 2023
- Automate NeuVector installation and management with Fleet – The GitOps way - August 25, 2023
- AKS startup error: Token refresh failed with invalid client secret error - August 22, 2023
7 thoughts on “Update a Library variable from a pipeline in Azure DevOps”
I’m adding this comment and link because, even though this post is a year old, it came up high in my search results for a way to update group variables from a pipeline. Since this was posted, there is now another pipeline extension that is specifically for creating, updating, and/or deleting variable groups. It’s ManageVariableGroupTask by Moreno Bruschi, available in the Marketplace here: https://marketplace.visualstudio.com/items?itemName=MorenoBruschi.manage-variable-group-task
Thank you Ray!
I’m getting some value property not found on $definition but the json does have it. Logs is shown below.
NewValue : b
2023-08-13T22:33:03.5139841Z URL: https://dev.azure.com/*********/**************/_apis/distributedtask/variablegroups/27?api-version=5.1-preview.1
2023-08-13T22:33:03.5142312Z URL: ***
2023-08-13T22:33:03.5142718Z Pipeline = {
2023-08-13T22:33:03.5142883Z “id”: 27,
2023-08-13T22:33:03.5143025Z “name”: “dev-updatevariable”,
2023-08-13T22:33:03.5143218Z “isShared”: false,
2023-08-13T22:33:03.5143391Z “variableGroupProjectReferences”: null
2023-08-13T22:33:03.5143553Z }
2023-08-13T22:33:03.6168636Z The property ‘Value’ cannot be found on this object. Verify that the property exists and can be set.
2023-08-13T22:33:03.6169109Z At D:\a\_temp\4f341332-bcd5-404a-b652-e2912e4f648a.ps1:21 char:1
2023-08-13T22:33:03.6169332Z + $definition.variables.$VariableName.Value = “$NewValue”
2023-08-13T22:33:03.6169656Z + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2023-08-13T22:33:03.6169876Z + CategoryInfo : InvalidOperation: (:) [], ParentContainsErrorRecordException
2023-08-13T22:33:03.6170151Z + FullyQualifiedErrorId : PropertyNotFound
2023-08-13T22:33:03.6170831Z
It turned out to be permission issue. I switched to use PAT in place of system access token and it works. Thoughts?
I hate using PAT so please advise.
Hi Manohar, Personal Access Tokens is a secure way in many systems to validate actions. I also use it with GitHub, Gitlab for example.
Giving the right name, permissions and expiration date ensure the right administration of PAT.
Don’t know other way to do it with Azure DevOps.
Well PATs are always having the limited validity and need to be updated manually so not a suit for automation 🙁
Depending on the tool, sometimes it’s possible to have a PAT without an expiration date.
I use them for automation and I get reminder email before the expiration (or I set one myself). That’s the only tradeoff I can see.
Usually we can see the last time a PAT has been used, it’s quite convenient.
Storing them in Library / Variables in Azure DevOps is easy, stored only once and their secret value protected.