Categories
Azure DevOps

How to fix Azure DevOps library group permission errors

Are you trying to edit a variable group in an Azure DevOps Library, and getting the error “you do not have permission to create a variable group within library“? Continue on to find out how to rectify this issue.

The problem

DevOps project settings – these don’t apply to variable groups

Variable groups within Azure DevOps can have different permissions to your project settings. This can be useful to limit the number of people who can view and edit your config values, but can be confusing.

As such, while you may have appropriate permissions to edit and your project you may find yourself unable to create variable groups within your Azure DevOps instance.

If this has happened to you, you’ll be shown an error something along the lines of “Error: you do not have permission to create a variable group within library.”

The fix

Luckily there’s a quick fix to this issue, although it will require you to find someone with the correct administrator privileges first.

Once you’ve found the person who has the correct privileges, navigate to your project in the DevOps portal, and create a new group. Alternatively, if this is an existing group click on the title of the group you wish to edit.

If you did create a new group, name it and make sure at least one variable exists. Note that this can be a dummy value – but without a variable DevOps won’t let you save the new group.

Then, within the edit group screen, select “Security“. The title of the modal that appears should be something like “Assign security roles for Library/<your group name>”. If it’s not, make sure you selected the group first and that you haven’t clicked “Security” from the main Library screen.

Security roles apply to variable groups

DevOps libraries have 3 tiers of roles/permissions:

  1. Reader: Can only view items within the library
  2. User: Can use items within the library, but can’t edit them
  3. Administrator: Can use AND manage items within the library

Search for the user(s) you want to be able to add and edit variables (and to avoid the above error) and choose the “administrator” role.

Click “Add”, close the modal and then click “Save”. The above error should no longer occur for users trying to edit or create variables in the DevOps library.

Categories
Azure DevOps

Azure DevOps system variables now read-only

Azure DevOps provides a number of predefined system variables out of the box on hosted build agents.

While these have officially always been read-only, informally you’ve previously been able to change the value of a predefined system variable by overwriting it in a pipeline task.

However, as of the January 28 2020 DevOps release this is no longer the case. The DevOps team have improved the security of system variables, and as such now actively prohibit system variables from being written to.

The variables include items such as the DefaultWorkingDirectory, JobName and TeamProject. These are all variables that you can otherwise use and refer to in your DevOps pipeline tasks.

This means you’ll need to now create your own task or pipeline variables where possible. If you can’t modify the task or pipeline variables, speak to someone with admin rights who can.

Categories
Azure DevOps

How to retry a failed stage in Azure DevOps

One of the biggest limitations of Azure DevOps has until now been the fact that you can’t retry an individual stage or step in a pipeline if it fails. Instead, you’d have to retry the entire pipeline again and hope that the failed stage passes.

As of September 2019, in most cases you can now retry a failed stage without the need to retry the whole pipeline – but there’s a catch. You’ll need to enable the Multi-stage pipelines preview feature, if you haven’t already. Note that this still doesn’t allow you to retry failed steps within a stage – rather it relies on the new multi-stage support to allow you to retry a collection of steps within a stage.

You can check if this is enabled by clicking your avatar in the top-right corner of the Azure DevOps portal, and then “Preview features”. Select “for this organisation [<your organisation name]” from the drop-down menu at the top of the screen, and scroll down until you see “Multi-stage pipelines”. If this is toggled off, you’ll need to turn this on before you can retry failed stages – you will need administrator rights to change it though.

Once you’ve enabled this, navigate back to your release pipelines and you should see the ability to re-rerun individual stages that fail throughout your pipeline.

Note that as this is still in beta, there are some limitations. Perhaps one of the biggest is that you can’t retry stages that are explicitly cancelled – only those that fail as a result of the task being run having an error can be retried today.

Categories
Azure DevOps

What is Azure DevOps used for?

Azure DevOps is a Microsoft product that can be used as part of the software development lifecycle, to manage the delivery, development and release of software products.

DevOps is a paid product (although there is a free tier available) that many software development companies use – particularly those within the Microsoft ecosystem. It used to be called Visual Studio Team Studio (VSTS).

DevOps is made up of 5 core areas of functionality:

  • Boards: Boards are used to track your teams work, and work best for teams using Agile methodologies to deliver work. Cards (or issues) can be created and tracked on boards, and linked to work as it gets delivered and released. Think of this as an equivalent to the likes of Atlassian Jira.
  • Repos: Employ version control for your code by creating a Git repository within DevOps. DevOps supports Git repositories in a similar manner to the likes of Github and Bitbucket.
  • Pipelines: Create continuous delivery pipelines that allow you to build your app and then release it automatically. Run your tests, deploy your code and store your build artefacts using Pipelines – DevOps has a whole host of tasks supported out of the box, or you can easily create your own.
  • Test Plans: Depending on your account level (paid or free) you can use test plans to track the progress of your test activities including both manual and automated tests, and easily analyse reports including passed/failed tests and defects raised.
  • Artifacts: Finally, DevOps will store your artifacts that are generated as part of your pipelines. This allows you to easily rollback to a previous version of your app if required, or the ability to re-deploy to new servers if required. You can also create package feeds with public sharing now in preview for public projects.
Categories
Azure DevOps

How to generate an Azure DevOps pipeline status badge for README files

Microsoft has made it very easy to generate badge icons that you can use wherever you’d like, showing the status of your build and release pipelines in Azure DevOps.

The icons – which are made available as images – can be generated for both build pipelines and release pipelines on the Azure DevOps website. They change based on the outcome of the latest run, allowing you to place them in places such as your project’s Readme files.

They aren’t particularly flexible, but you can choose the branch and scope of the success/fail status.

How to generate a badge icon

Login to the Azure DevOps portal. Navigate to Pipelines, and then choose the release or build pipeline that you would like to generate an icon for.

In the top right-hand corner, select the three dots and then choose “Status badge”.

This brings up a modal window that allows you to customise the badge, and choose a branch, scope for the status (pipeline or per job), and then a URL is generated for the image.

Microsoft also provides the markdown text which allows you to immediately copy and paste the badge into a markdown file such as your project’s Readme file.

Categories
Azure DevOps

How to delete an Azure DevOps project

In this post we’ll look at how to delete an Azure DevOps project.

First, you’ll need to login with your account to the DevOps portal. Choose the organisation that contains the project you want to delete from the left-hand sidebar, and then select the project you want to delete.

Then, once you’re on the homepage for the project you wish to delete, click “Project Settings” in the bottom left-hand corner. If you don’t see this option you may not have the required permission to manage the project.

Navigate to the General > Overview screen in Project Settings and scroll to the very bottom. A big red “Delete” button should appear at the bottom.

Click “Delete” and you’ll be asked to enter type your project name to confirm the deletion. Type the name of the project in the textfield, click “Delete” and your project removed.

Note that it won’t actually be deleted immediately – it will be placed in a temporary recycling bin for 28 days before being permanently removed. This gives you time to change your mind about the deletion, for example if you removed the wrong project or need access to the information again. After the 28 days the information will be permanently erased.

Categories
Azure DevOps

Using key vault values from variable groups in Azure DevOps pipeline tasks

Earlier this week we had a post about how you can easily access secrets stored within Azure Key Vault in an Azure DevOps pipeline task, using the Key Vault Task. One other way you can achieve this same functionality is by using a variable group, and in this post we’re going to show you how.

Why would you use a variable group instead of the key vault task? If you know you require access to the secrets from across multiple stages within a pipeline using a group allows you to easily manage access without having to include the task in every single stage by scoping the group to the release or specific stages.

However, if only one stage requires access to the secrets it might be easier to just include the task in that particular stage and follow our previous post.

Getting started

First you’ll need to setup your key vault so that your service principal or managed identity has GET access to your vault. Then, follow our previous post on creating a variable group with a key vault to setup DevOps for this integration.

Once you’ve done the above, it’s time to get started. Navigate to your release pipeline in Azure DevOps.

Connecting the variable group

From your release pipeline, click “Edit”. Go to the “Variables” tab at the top of the screen, and choose “Variable groups”.

Select the variable group you created in the above section, scope it to either the entire release or a stage of your pipeline (depending on where you need to have access to the secrets, keeping in mind the more restricted you can make it the better from a security perspective) and click “Save”.

Now from within your tasks, if you need to reference the secrets you can use the $(your-variable-name) syntax. For instance, in the Azure Function App Deploy task, if we wanted to specify an app setting under “Application and Configuration Settings” we could use the following syntax:

-NOTIFICATIONHUB_CONNECTIONSTRING $(your-connection-string-secret-identifier)

The benefit of this is that the syntax is identical to that used with the key vault task (they both set the secrets as task variables) so if you need to, you can swap between using a variable group or key vault task with ease.

Summary

In summary, this is an alternative approach to using the key vault task in a pipeline. Depending on your needs, this may be a better approach than the other lesson, but mileage may vary.

Categories
Azure DevOps

Accessing Key Vault secrets in an Azure DevOps pipeline task

In the post, we’ll take a look at one option for accessing Azure Key Vault secrets from within an Azure DevOps release pipeline.

Want to secure your Azure DevOps application secrets in Key Vault? Find out how in  our short e-book guide on Amazon

Setting up your Azure Key Vault

Before you can add the secret to your pipeline, you first need to make sure that there’s a key vault setup in Azure, and that you have given either your pipeline managed service identity or account GET access to the secrets within the vault. Note that you need to ensure this is set under your key vault’s “Settings” → “Access Policies” section.

If you haven’t already, add your secrets into the “Secrets” section, and take note of the names used for the secrets – you’ll need these a bit later on.

Adding the Azure Key Vault pipeline task

Now that you’ve got your secrets stored and accessible from key vault, it’s time to configure the Dev Ops pipeline. Open the visual pipeline editor for your pipeline by clicking “Edit”, and choose the stage in which you need access to the secrets.

Then, click the “+” button next to “Run on agent” (or whatever the first step of your pipeline may be) and search for “Azure Key Vault”. Note that you’ll want to add the “Download Key Vault Secrets” task that appears first – this is the official task from Microsoft.

Then, click on the new “Azure Key Vault” task you just added to your pipeline, and set a display name. Choose the Azure subscription in which you created your key vault, and then select from the “Key vault” dropdown list the name of the key vault you stored your secrets in.

If you can’t see it listed, it’s possible you managed service identity doesn’t have the correct permission, so be sure to check it’s been added to your key vault with GET permission.

Now drag the key vault task up your pipeline task list (if applicable) so that it runs before any other task that requires a secret stored within your key vault.

And that’s it!

Accessing a Key Vault secret from other tasks

Now other tasks can access the secrets, by using a task variable. The key vault task will make all your secrets available using the $(<your secret name here>) syntax, such as $(api-secret).

Summary

This is just one of the various ways that you can access key vault secrets from a Dev Ops pipeline task. Stay tuned for more posts where we explore other ways of accessing the secrets in tasks.

If you want a more in-depth guide and comparison between alternative approaches to storing secrets in DevOps, you can also get our book on Amazon today.

Categories
Azure DevOps

How to use variable groups in Azure DevOps

In a previous post looking at how to use Azure Key Vault to store secrets for a DevOps pipeline, we touched on variable groups and how they can be used. In this post, we’re going to dive a bit deeper into what a variable group is, how you can create one and how you can link variable groups into your build pipeline.

Want to secure your Azure DevOps application secrets in Key Vault? Find out how in  our short e-book guide on Amazon

What is a variable group?

A variable group is a logical collection of environment variables (or properties) used throughout your build and/or deployment pipelines. They are essentially key value pairs, that can include things like API keys, database connection strings or configuration items such as downstream API endpoint URLs.

Variable groups can store both plain text variables and secrets, which should never be committed into your source code repository. Note that if you want to use DevOps’ Azure Key Vault integration, you’ll need to create a separate variable group as you can’t mix and match Key Vault and DevOps variables in the same group.

How do I create one?

Creating a variable group is simple. Login to Azure DevOps and navigate to “Pipelines” > “Library”. You’ll see in the top navigation bar the option to “+ Variable Group”. Clicking that will take you to a “New variable group” screen, that asks for a number of properties:

  • Variable group name: A friendly name used to refer to your new variable group. Use something that has meaning to you and the types of properties that will be stored within the group (ie. My App – Production)
  • Description: Provide a short description that describes a bit more about the types of variables that should be placed within this group (eg. This group contains build settings and environment variables for production builds only)
  • Allow access to all pipelines: Enable this toggle to ensure that you can access all the variables from all of your pipelines. If you don’t enable it, you’ll need to authorise pipelines defined in YAML manually in order to let them access your properties.
  • Link secrets from an Azure key vault as variables: Enable this toggle if you want to use Azure Key Vault to store your secrets instead of DevOps. Note that by enabling this, you’ll need to then provide key vault connection details, and enter your secrets via the Azure Portal instead.

If you left the key vault integration disabled, you’ll now be able to click “Add” below the “Variables” section to begin creating new properties. Each property can have a name and a value, and can be marked as clear text or secret by clicking the padlock icon at the end of each row.

If you need to customise security permissions, click “Security” at the top of the screen. This will bring up a modal window where you can add or remove user groups from being able to access this variable group.

Once you’re done, hit “Save” and your variable group will be persisted.

How do I use it?

If you selected “Allow access to all pipelines” when creating the variable group, linking it with your build pipeline through the DevOps website is simple. Navigate to the pipeline you want to link the group with, click “Edit” in the top-left corner and then click “Variables” underneath your pipeline name.

You should see on the left-hand side “Pipeline variables” and “Variable groups”. Navigate to the latter, and click “Link variable group”. This brings up a modal that lists all the variable groups that your role has access to – if you can’t see a group that you know exists, check to make sure you have set the right permissions in the “Security” window for that group.

Choose the group you want to link with the pipeline, and then the scope that the group applies to. If the variables are used throughout the pipeline, then you can choose to make it visible to the entire release, or if you know only a few scopes require access you can also choose specific scopes.

Click “Link” and you’ve now made the variable group be accessible from your pipeline!

Alternatively, if you’re using a YAML file to describe your pipelines and builds, you can also add a variable group by adding the following section in your YAML file:

variables:- group: your-new-variable-group-name

One thing worth noting is that when you run a pipeline, DevOps will create an immutable snapshot with the values of the variables within your group so that your release remains in the same state. This ensures that it isn’t influenced by future changes or modifications that you might make to the values, and means you can redeploy a release later if needed.

Summary

Variable groups can be powerful tools for logically grouping properties and secrets that you need for build pipelines, and are simple to configure and use.

Categories
Azure DevOps

How to store Azure DevOps secrets in Azure Key Vault

Often when creating an Azure DevOps continuous integration/deployment pipeline there’s a need to store and use app secrets, such as client keys. While you can store secrets within Azure DevOps variable groups, an alternative approach is to use Azure Key Vault instead.

Want to secure your Azure DevOps application secrets in Key Vault? Find out how in  our short e-book guide on Amazon

By using Azure Key Vault you get the same enhanced data protection that your other cloud apps can enjoy including activation and expiration dates, and the DevOps integration allows for the centralised management of keys used across apps or pipelines. Keep in mind if you decide to use key vault, you will be charged according to the Azure Key Vault pricing for storing your secrets.

Setting up Key Vault access in Azure DevOps

Getting started is easy. Open Azure DevOps, and navigate to the project you wish to integrate with. Open the Pipelines section, and then go to Library.

If you already have secrets and values stored in an existing library, the easiest way to integrate with key vault is to create a separate variable group. If you don’t, you’ll get a message warning you that when you enable key vault in your existing group, it’ll blow away all your existing variables saved within the group. This is because you can’t use key vault variables side by side with Dev Ops variables within one group.

Open the new variable group, and you should see a toggle to link secrets from an Azure key vault as variables. Turn that on, and you’ll see the option to set the Azure subscription to be used, and a field to specify a key vault name.

You’ll need to ensure that you’ve previously setup a connection to your Azure subscription within Azure DevOps, and added an Azure Resource Manager service connection using an Azure Service Principal to the resource group where your key vault is located. If you haven’t, the management links next to each field will help you to setup these connections.

Once connected, pointing DevOps to your key vault is as easy as choosing the correct subscription from the drop down list and then selecting your key vault by name in the second drop down. If your service principal doesn’t have get and list secret management permissions, you’ll be prompted to automatically authorise it or manually do so in the Azure Portal.

If successfully connected, you’ll be able to then see a list of secrets from your key vault by clicking the add button. Choose the secrets you want to make available to your pipeline and click OK.

Add your new variable group to your pipeline, and that’s all there is to adding key vault secrets to an Azure DevOps pipeline.