In this blog series, we will be creating a build and release pipeline in Azure DevOps that will deploy an App Engine Flexible Environment in GCP. The application we are deploying will contain 2 services where both of these services will be created from Dockerfiles.
This blog will be covered in 2 parts:
- Part 1 for the build pipeline where we will build and push the Docker images to Google Container Registry (GCR).
- Part 2 for the release pipeline where we will run a Dockerised gcloud program to deploy our application to GCP.
Before We Begin
Before we start on the pipelines, we have some setting up to do. We will need:
- a project in GCP (you can sign up for a free trial here)
- a project in Azure DevOps to store your source code and the pipelines
- two .NET sample applications
We will only be showing how to deploy to single deployment environment (e.g., development) but the steps will be set up in a way that you can extend it to multiple environments (e.g., staging, production). For App Engine, there can only be one App Engine application in each GCP project. While you can create different versions or services in App Engine to represent the different environments, the steps will assume that you will be creating different projects. This approach ensures complete isolation between different environments and allows you to easily configure permissions (e.g., developers only have rights to update the development environment).
Creating a GCP Project
Create a new Cloud project (e.g., AzureGCP-Dev).
- You can create a new project by selecting the drop down (top left next to "Google Cloud Platform") then selecting "NEW PROJECT" top right of the popped-up modal.
Create a new application in App Engine:
Navigate to "App Engine" > "Create Application".
Select a region (Pick carefully as this cannot be changed later).
Select "Create app".
Click "Cancel" to cancel out of the Language and Environment step as we will be deploying our code from Azure Pipelines. When you are back on the main App Engine page there will be a message of "Your App Engine application has been created". If the message is not there, try refreshing the page.
App Engine Admin API,
Google App Engine Flexible Environmentand
Service Management APIis enabled. You can check it via the API & Services page in Cloud Console. We will need these APIs to deploy our application from Azure Pipelines.
Create a service account with the following permissions and create a JSON key for it. The service account will act as the "user" deploying our application from Azure Pipelines.
App Engine Deployer- To deploy new code to App Engine.
Cloud Build Editor- Needed to be able to use gcloud tool to deploy.
Storage Admin- Needed to be able to use gcloud tool to deploy.
Service Account User- To run operations as the service account.
App Engine Service Admin- To upgrade our new version to the default version.
Create GCR Service Connection in Azure
- Create a new Azure DevOps Project (e.g.,
- Create a Service Connection to Google Container Registry in Azure DevOps.
- Navigate to "Project Settings" > "Pipelines" > "Service connections" > "New service connection".
- Select "Docker Registry".
- "Docker Registry":
- "Docker ID": _json_key
- "Docker Password":
<paste the entire contents of the service account key in the json file>
Create Sample ASP.NET Application
We don't usually need to write a Dockerfile to deploy ASP.NET Core applications to App Engine Flex but we will in this guide for demonstration purposes. To deploy ASP.NET without a Dockerfile, please refer to this GitHub page.
Create a new ASP.NET project (e.g., aspnetapp).
- in Program.cs add the following extension method:
Create a Dockerfile.
app.yamlto store the App Engine configuration for our application. This will create a
Since you can only have one App Engine instance per project, if you want to have multiple applications running within App Engine, you can do so by creating different services. This feature of App Engine also allows users to develop their applications using microservices architecture.
In our case, we will be adding another service, which will be another Dockerfile, to our App Engine instance.
- On the left
aspnetappis the parent folder. On the right,
aspnetappis the first project we created and
AspNetAppTwois the second one we are creating right now.
- On the left
Create an app.yaml file within the project.
- You can see here that we have defined a
serviceelement. We use this to give our service a named to be referred to. You don't need it if you are creating a
defaultservice (recall our previous
app.yamldoes not have a service
elementbecause it is the
- You can see here that we have defined a
Create a Dockerfile for this project. You can duplicate the Dockerfile from the above section and replace the
Now that we have set up everything we need, it's time to create our build pipeline.
Creating Azure Build Pipeline
- In Azure DevOps, upload our source code into "Repos" in the newly created Azure project using the instructions in "Push an existing repository from command line". Make sure your git repository includes both ASP.NET projects.
- Create a new Build Pipeline using the steps below:
- Navigate to "Pipelines" > "New Pipeline".
- "Connect": "Azure Repos Git"
- "Select": Select the repository (e.g.,
- "Configure" → "Docker (Build a Docker image)"
- Select the Azure subscription you wish to use then select Continue.
- Select Validate and configure.
Build and Push Docker Image
We will then need to configure our build pipeline to build and push the Docker images to GCR and our release pipeline will deploy to App Engine using the
In the variables section of the build pipeline, ensure there is a variable named tag with the value of
For the "Docker (Build a Docker image)" task in our build pipeline, we will need to ensure the following configurations are set:
. Always start the path with your google project id (e.g.,
azuregcp-dev/aspnetapp). If the path you specified does not exist, it will create one for you.
- "command": '
- You can delete this part and just create a new task or select "Settings" above the task in the YAML editor.
Select "Save and Run" to test it out. If you navigate to Google Container Registry of your development project, you can see the
- The different repository you have created for our 2 images.
- Each represents a built image, and they are tagged with the value of our tag variable which is the Build.BuildId.
Repeat step 2 for our
AspNetAppTwo'sDockerfile and verify it in GCR.
To deploy our application using the
--image-url flag, we need to specify a valid Container Registry hostname. For our case, it will be in the following format:
This means that the release pipeline needs to know what the value of the tag variable. To do this, we can share the variable using a variable group.
In Azure DevOps, navigate to "Library" > "+ Variable group".
give it a "Variable group name".
ensure "Allow access to all pipelines" is enabled.
"Add" a variable to the group to store the value of
tagby giving it a name (e.g.,
latestTag) and an initial value.
Click "Security" and add "
Build Service (" with "Administrator" role and click the "Save" icon. )
Then click "Save" to save the variable group.
In the build pipeline, add an Azure CLI task to update the
latestTagvariable in our
azure-gcp-groupvariable group with the value of tag in the pipeline.
If you need help creating an Azure Resource Manager connection, refer to this site.
In the variables section, add the variable group so that we can access it in the Azure CLI task.
Run the pipeline and verify that the value of
latestTaghas been updated. You can see the value of the variable by navigating to "Library".
To be continued...
In this blog post, we have successfully created a build pipeline that builds and pushes two Docker images to Google Cloud Registry. Stay tuned for the next part of the series where we will be creating a release pipeline that deploys our Docker images to App Engine!