In this blog series, we will be creating a build and release pipeline in Azure DevOps that will deploy an App Engine Flexible Environment in GCP. The application we are deploying will contain 2 services where both of these services will be created from Dockerfiles.

This blog will be covered in 2 parts:

  • Part 1 for the build pipeline where we will build and push the Docker images to Google Container Registry (GCR).
  • Part 2 for the release pipeline where we will run a Dockerised gcloud program to deploy our application to GCP.

Before We Begin

Before we start on the pipelines, we have some setting up to do. We will need:

  • a project in GCP (you can sign up for a free trial here)
  • a project in Azure DevOps to store your source code and the pipelines
  • two .NET sample applications

We will only be showing how to deploy to single deployment environment (e.g., development) but the steps will be set up in a way that you can extend it to multiple environments (e.g., staging, production). For App Engine, there can only be one App Engine application in each GCP project. While you can create different versions or services in App Engine to represent the different environments, the steps will assume that you will be creating different projects. This approach ensures complete isolation between different environments and allows you to easily configure permissions (e.g., developers only have rights to update the development environment).

Creating a GCP Project

  1. Create a new Cloud project (e.g., AzureGCP-Dev).

    GCP Project Modal

    • You can create a new project by selecting the drop down (top left next to "Google Cloud Platform") then selecting "NEW PROJECT" top right of the popped-up modal.

  2. Create a new application in App Engine:

    • Navigate to "App Engine" > "Create Application".

      App Engine welcome screen

    • Select a region (Pick carefully as this cannot be changed later).

    • Select "Create app".

    • Click "Cancel" to cancel out of the Language and Environment step as we will be deploying our code from Azure Pipelines. When you are back on the main App Engine page there will be a message of "Your App Engine application has been created". If the message is not there, try refreshing the page.

  3. Ensure that App Engine Admin API, Google App Engine Flexible Environment and Service Management API is enabled. You can check it via the API & Services page in Cloud Console. We will need these APIs to deploy our application from Azure Pipelines.

    App Engine Admin API

  4. Create a service account with the following permissions and create a JSON key for it. The service account will act as the "user" deploying our application from Azure Pipelines.

    GCP Service Accounts

Create GCR Service Connection in Azure

  1. Create a new Azure DevOps Project (e.g., AzureGCP).
  2. Create a Service Connection to Google Container Registry in Azure DevOps.
    • Navigate to "Project Settings" > "Pipelines" > "Service connections" > "New service connection".
    • Select "Docker Registry".

GCR Service Connection

  • "Docker Registry": https://gcr.io/<PROJECT_ID>
  • "Docker ID": _json_key
  • "Docker Password": <paste the entire contents of the service account key in the json file>

Create Sample ASP.NET Application

We don't usually need to write a Dockerfile to deploy ASP.NET Core applications to App Engine Flex but we will in this guide for demonstration purposes. To deploy ASP.NET without a Dockerfile, please refer to this GitHub page.

  1. Create a new ASP.NET project (e.g., aspnetapp).

    • in Program.cs add the following extension method:

  2. Create a Dockerfile.

    • replace aspnetapp.dll with <name_of_application>.dll.

  3. Create app.yaml to store the App Engine configuration for our application. This will create a default service.

Multiple Dockerfiles

Since you can only have one App Engine instance per project, if you want to have multiple applications running within App Engine, you can do so by creating different services. This feature of App Engine also allows users to develop their applications using microservices architecture.

In our case, we will be adding another service, which will be another Dockerfile, to our App Engine instance.

  1. Create another project of your choice. In our example, we are creating another ASP.NET project (e.g., AspNetAppTwo) and it resides in the same parent folder as our first ASP.NET project.

    Project Folder Structure

    • On the left aspnetapp is the parent folder. On the right, aspnetapp is the first project we created and AspNetAppTwo is the second one we are creating right now.

  2. Create an app.yaml file within the project.

    • You can see here that we have defined a service element. We use this to give our service a named to be referred to. You don't need it if you are creating a default service (recall our previous app.yaml does not have a service element because it is the default service).

  3. Create a Dockerfile for this project. You can duplicate the Dockerfile from the above section and replace the <name_of_application>.dll.

Build Pipeline

Now that we have set up everything we need, it's time to create our build pipeline.

Creating Azure Build Pipeline

  1. In Azure DevOps, upload our source code into "Repos" in the newly created Azure project using the instructions in "Push an existing repository from command line". Make sure your git repository includes both ASP.NET projects.
  2. Create a new Build Pipeline using the steps below:
    • Navigate to "Pipelines" > "New Pipeline".
    • "Connect": "Azure Repos Git"
    • "Select": Select the repository (e.g., AzureGCP)
    • "Configure" → "Docker (Build a Docker image)"
      • Select the Azure subscription you wish to use then select Continue.
      • Select Validate and configure.

Build and Push Docker Image

We will then need to configure our build pipeline to build and push the Docker images to GCR and our release pipeline will deploy to App Engine using the --image-url flag.

  1. In the variables section of the build pipeline, ensure there is a variable named tag with the value of $(Build.BuildId).

  2. For the "Docker (Build a Docker image)" task in our build pipeline, we will need to ensure the following configurations are set:

    • "containerRegistry": (e.g., GoogleContainerRegistryDev)
    • "repository": . Always start the path with your google project id (e.g., azuregcp-dev/aspnetapp). If the path you specified does not exist, it will create one for you.
    • "command": 'buildAndPush'
    • "Dockerfile": (e.g., aspnetapp/aspnetapp/Dockerfile)

    Azure Pipeline YAML

    • You can delete this part and just create a new task or select "Settings" above the task in the YAML editor.

  3. Select "Save and Run" to test it out. If you navigate to Google Container Registry of your development project, you can see the aspnetapp folder:

    GCR Folders

    • The different repository you have created for our 2 images.

    GCR Custom Images

    • Each represents a built image, and they are tagged with the value of our tag variable which is the Build.BuildId.

  4. Repeat step 2 for our AspNetAppTwo's Dockerfile and verify it in GCR.

Shared Variable

To deploy our application using the --image-url flag, we need to specify a valid Container Registry hostname. For our case, it will be in the following format:

gcr.io/<Project_ID>/<Repository_Name>:<Tag>

This means that the release pipeline needs to know what the value of the tag variable. To do this, we can share the variable using a variable group.

  1. In Azure DevOps, navigate to "Library" > "+ Variable group".

    • give it a "Variable group name".

    • ensure "Allow access to all pipelines" is enabled.

    • "Add" a variable to the group to store the value of tag by giving it a name (e.g., latestTag) and an initial value.

    • Click "Security" and add " Build Service ()" with "Administrator" role and click the "Save" icon.

    • Then click "Save" to save the variable group.

      Azure DevOps Variable Group

  2. In the build pipeline, add an Azure CLI task to update the latestTag variable in our azure-gcp-group variable group with the value of tag in the pipeline.

    If you need help creating an Azure Resource Manager connection, refer to this site.

    Azure Resource Manager Connection

  3. In the variables section, add the variable group so that we can access it in the Azure CLI task.

  4. Run the pipeline and verify that the value of latestTag has been updated. You can see the value of the variable by navigating to "Library".

To be continued...

In this blog post, we have successfully created a build pipeline that builds and pushes two Docker images to Google Cloud Registry. Stay tuned for the next part of the series where we will be creating a release pipeline that deploys our Docker images to App Engine!