Edit

Use Azure Key Vault secrets in Azure Pipelines

Azure DevOps Services | Azure DevOps Server | Azure DevOps Server 2022

Azure Key Vault is a cloud service that helps developers securely store and manage sensitive information such as API keys, credentials, and certificates. Key Vault supports two types of containers: vaults and managed hardware security module (HSM) pools. Vaults can store both software and HSM-backed keys, secrets, and certificates. Managed HSM pools exclusively support HSM-backed keys.

In this article, you learn how to create a key vault, add a secret, configure access policies, and then use that secret in Azure Pipelines. This tutorial uses a key vault with public network access. If you need to access a private key vault from your pipeline, see Access a private key vault from your pipeline. To link Key Vault secrets to variable groups, see Link a variable group to secrets in Azure Key Vault.

Prerequisites

Category Requirements
Azure DevOps - An Azure DevOps organization.
- An Azure DevOps project.
Azure An Azure subscription.

Get the code

If you don't have your own project, import the following sample repository into your Azure repo:

  1. Sign in to your Azure DevOps organization, and then go to your project.

  2. Select Repos, and then select Import. Enter the following repository URL, and then select Import.

    https://github.com/MicrosoftDocs/pipelines-dotnet-core
    

Create a key vault

To create a new key vault in Azure by using the Azure CLI, follow these steps:

  1. Go to the Azure portal, and then select Azure Cloud Shell in the upper-right corner.

  2. If your account is associated with multiple Azure subscriptions, set your default subscription.

    az account set --subscription <YOUR_SUBSCRIPTION_NAME_OR_ID>
    
  3. Set a default Azure region. To see a list of available regions, run az account list-locations.

    az config set defaults.location=<YOUR_REGION>
    
  4. Create a new resource group.

    az group create --name <YOUR_RESOURCE_GROUP_NAME>
    
  5. Create a new key vault.

    az keyvault create \
      --name <YOUR_KEY_VAULT_NAME> \
      --resource-group <YOUR_RESOURCE_GROUP_NAME>
    
  6. Add a secret to your key vault.

    az keyvault secret set \
      --name <YOUR_SECRET_NAME> \
      --value <YOUR_ACTUAL_SECRET> \
      --vault-name <YOUR_KEY_VAULT_NAME>
    

Set up authentication

After you create your key vault, the next step is to set up authentication. Select Managed Identity or Service Principal, and follow the instructions to configure authentication.

Create a user-assigned managed identity

  1. Go to the Azure portal, and then search for Managed Identities in the search bar.

  2. Select Create, and provide the following information:

    • Subscription: Select your Azure subscription from the dropdown menu.
    • Resource group: Select an existing resource group or create a new one.
    • Region: Select the region where the managed identity is created.
    • Name: Enter a name for the user-assigned managed identity.
  3. Select Review + create, and then select Create to start the deployment.

  4. After the deployment finishes, select Go to resource, and copy the Subscription ID and Client ID values. You need these values in later steps.

  5. Under Settings, select Properties, and copy your managed identity's Tenant ID value for later use.

Set up key vault access policies

  1. Go to the Azure portal, and use the search bar to locate the key vault you created earlier.

  2. Select Access policies, and then select Create to add a new policy.

  3. Under Secret permissions, select the Get and List checkboxes.

  4. Select Next. Paste the Client ID of the managed identity that you created earlier into the search bar, and then select the managed identity.

  5. Select Next, and then select Next again.

  6. Review the access policy details, and then select Create to apply the policy.

Create a service connection

  1. Sign in to Azure DevOps, and then go to your project.

  2. Select Project settings > Service connections > New service connection.

  3. Select Azure Resource Manager, and then select Next.

  4. For Identity Type, select Managed identity.

  5. Under Step 1: Managed identity details, provide the following information:

    • Subscription for managed identity: Select the subscription that contains your managed identity.
    • Resource group for managed identity: Select the resource group that hosts your managed identity.
    • Managed identity: Select your managed identity from the dropdown list.
  6. For Step 2: Azure Scope, provide the following information:

    • Scope level for service connection: Select Subscription.
    • Subscription for service connection: Select the subscription that the managed identity accesses.
    • Resource group for service connection: (Optional) Specify a resource group to limit the managed identity's access to one resource group.
  7. For Step 3: Service connection details, provide the following information:

    • Service connection name: Enter a name for the service connection.
    • Service management reference: (Optional) Include context information from an ITSM database.
    • Description: (Optional) Enter a description.
  8. Under Security, the Grant access permission to all pipelines option allows all pipelines to use this service connection. We don't recommend this option. Instead, authorize each pipeline individually to use the service connection.

  9. Select Save to validate and create the service connection.

    Screenshot that shows how to create a managed identity Azure Resource Manager service connection.

Access key vault secrets from your pipeline

Warning

This tutorial is for educational purposes only. For security best practices and guidance on safely working with secrets, see Manage secrets in your server apps with Azure Key Vault.

  1. Sign in to Azure DevOps, and then go to your project.

  2. Select Pipelines > New Pipeline.

  3. Select Azure Repos Git (YAML), and then select your repository.

  4. Select the Starter pipeline template.

  5. The default pipeline includes sample echo commands. You don't need these commands, so you can remove them.

  6. Add the Azure Key Vault task to your pipeline. Replace the placeholders with the name of the service connection that you created earlier and your key vault name. Your YAML file should look similar to the following example:

    trigger:
    - main
    
    pool:
      vmImage: ubuntu-latest
    
    steps:
    - task: AzureKeyVault@2
      displayName: Azure Key Vault
      inputs:
        azureSubscription: 'SERVICE_CONNECTION_NAME'
        KeyVaultName: 'KEY_VAULT_NAME'
        SecretsFilter: '*'
        RunAsPreJob: false
    
  7. Add the following tasks to copy and publish the secret. This example is for demonstration purposes only. Don't use it in a production environment.

    trigger:
    - main
    
    pool:
      vmImage: ubuntu-latest
    
    steps:
    - task: AzureKeyVault@2
      displayName: Azure Key Vault
      inputs:
        azureSubscription: 'SERVICE_CONNECTION_NAME'
        KeyVaultName: 'KEY_VAULT_NAME'
        SecretsFilter: '*'
        RunAsPreJob: false
    
    - task: CmdLine@2
      displayName: Create file
      inputs:
        script: 'echo $(SECRET_NAME) > secret.txt'
    
    - task: CopyFiles@2
      displayName: Copy file
      inputs:
        Contents: secret.txt
        targetFolder: '$(Build.ArtifactStagingDirectory)'
    
    - task: PublishBuildArtifacts@1
      displayName: Publish Artifact
      inputs:
        PathtoPublish: '$(Build.ArtifactStagingDirectory)'
        ArtifactName: 'drop'
        publishLocation: 'Container'
    
  8. Select Save and run, and then select it once more to commit your changes and trigger the pipeline. If prompted, select Allow to grant the pipeline access to Azure resources.

  9. After the pipeline starts, select the CmdLine task to view the logs.

    Screenshot that shows the command-line task logs.

  10. When the pipeline run finishes, return to the pipeline summary and select the published artifact.

    Screenshot that shows the published artifact in the summary tab.

  11. Select drop > secret.txt to download the file.

    Screenshot that shows how to download the published artifact.

  12. Open the downloaded text file. It should contain the secret retrieved from your key vault.

Clean up resources

To delete the resources that you created, follow these steps:

  1. If you created a new organization to host your project, see how to delete your organization. Otherwise, delete your project.

  2. All Azure resources created during this tutorial are hosted in a single resource group. Run the following command to delete the resource group and all of its resources.

    az group delete --name <YOUR_RESOURCE_GROUP_NAME>
    

Troubleshooting

Error: "The user or group does not have secrets list permission"

This error occurs when the service principal or managed identity used by your pipeline doesn't have permission to list secrets in the key vault. To resolve this issue, make sure the identity has the Get and List permissions for secrets. Run the following commands to grant the required permissions to your service principal:

az login

az account set --subscription <YOUR_SUBSCRIPTION_ID>

$spnObjectId = az ad sp show --id <YOUR_SERVICE_PRINCIPAL_ID>

az keyvault set-policy --name <YOUR_KEY_VAULT_NAME> --object-id $spnObjectId --secret-permissions get list