Overview

This series of blog posts describes an approach I've taken recently to implement continuous delivery for cross-platform native mobile apps built using Xamarin Forms and C#, targeting both Android and iOS. I've included the Main Goals and Prerequisites sections from my first post here for reference as well.

  • Part 1: focuses on putting together a basic Xamarin Forms application with specific patterns to support continuous delivery
  • Part 2 (this post): focuses on creating a build pipeline that will provide continuous integration for the application source, by building, testing and packaging the code whenever it changes
  • Part 3: focuses on creating a release pipeline that will allow staged deployments of app builds to App Center for testing and production release

 

Main Goals

  • The application code should be built and tested whenever commits are made to any branch in the source repository.
  • Whenever a build successfully completes on the master branch, the app should be packaged and deployed automatically to the DEV environment. This would normally occur due to a new Pull Request (PR) being merged to the master branch from a feature branch, for example.
  • The configuration of the app packages for each platform should be updated as a release is promoted from one stage to the next (eg from DEV -> UAT, or UAT -> PROD), using settings appropriate for the environment associated with each stage.
  • Manual testers and operators should be notified by email when a new release is available in App Center.

 

Prerequisites

  • I've used Visual Studio 2019 (version 16.7.1) for developing the sample solution, with the Mobile Development for .NET workload installed, which covers Xamarin requirements, but you could also use Visual Studio for Mac.
  • You will need access to a MacBook or similar macOS device for building and signing the iOS project, and a paid Apple Developer account that can be used for generating certificates and provisioning profiles - get started at https://developer.apple.com.
  • You will need an Azure DevOps account for creating the build and release pipelines used in this series. Source code can be hosted in a git repository in Azure Repos, or you could use another source control provider such as GitHub or BitBucket if you wish - go to https://dev.azure.com to get started.
  • You will need a Visual Studio App Center account in order to create the platform-specific projects we will need for deploying and testing our app packages - go to https://appcenter.ms/ to get started.

 

Step 2: Creating a continuous integration pipeline for a Xamarin Forms mobile app

In the previous post, we created a sample app to use as a foundation for developing build and release pipelines in Azure DevOps. In this post, we will create a continuous integration (CI) pipeline that will cover the following responsibilities:

  • Run automatically for any commit made to the source code repository
  • Build the code for both Android and iOS
  • Run unit tests
  • For main branch builds only, package the apps for deferred deployment as well
  • The build should fail quickly if any steps fail

Unlike standard .NET Core apps, building a Xamarin Forms app for continuous integration in Azure DevOps is much more involved, due to the combination of different platforms and frameworks that we need to use. The pipeline I'm going to demonstrate here is a good starting point for building on both Android and iOS platforms, and is based on the newer YAML format used in Azure Pipelines. YAML-based pipelines are part of the source for the application, so that they can follow the same development lifecycle as any other code would.

To start with, we will create the full pipeline file at once, so that there are no copy/paste errors in the YAML from building up a file slowly, as YAML is whitespace-sensitive and even a single tab or space out of place can cause the entire file to fail parsing, which we don't want. After that, I'll walk through each section in more detail to explain what the pipeline is doing. Finally, we will create and configure a pipeline definition in Azure DevOps that uses this YAML file.

If you would just like to get the completed solution in its entirety, you can clone the full source code at https://github.com/sam-piper/xamarin-devops at any point.

Please note that in order to set up the build job for iOS, you will also need a paid Apple Developer account and a macOS device (MacBook Pro or similar) with Xcode 11+ installed, in order to create the necessary signing certificate and provisioning profile files that can be uploaded into Azure DevOps.

Open the MobileDevOps.SampleApp solution we created in the last post, right-click the solution node in Solution Explorer, select Add > New Item..., then create a new text file called azure-pipelines.yml. Copy and paste the full YAML from below:

Don't be put off by the complexity of the file, I'll break it down for you as much as I can. If you aren't already familiar with the YAML syntax of Azure Pipelines, I recommend starting with the Microsoft reference documentation.

At a high level, this build pipeline consists of a single stage, with two independent jobs that run sequentially on different build agents. We don't define a trigger element as we want the default behaviour, which is to trigger when any commit is made on any branch in the repository that contains the build definition. As there is only one stage, we can omit the stage element entirely and just define our jobs at the top level instead. You can think of a job as an independent sequence of tasks (called steps), which runs on a specific build agent virtual machine based on Windows, Linux, or macOS.

Let's start from the top of the file and go into more detail:

Here, we are defining our first job, named Android, but you can name it however you want. At a high-level, this job will restore packages and build the code for all projects in the solution except the Xamarin.iOS project, which will be ignored on Windows. This job will also run the unit tests as well, and publish the results back to our Azure DevOps project. If the code cannot be built successfully or one or more unit tests fail, then this job will fail the entire build.

If the build is taking place on the main branch and build / test steps succeed, then the job will also package the Android app into an unsigned *.APK file which will be stored in the published output for the build. These build artifacts can then be used in other pipelines, such as the multi-stage release pipeline I will be demonstrating in my following post.

The pool property is used to specify the build agent image that we want to use to run the job, in this case I'm setting the vmImage attribute to windows-2019, which is a Windows image that allows us to use Visual Studio 2019 for restoring and building the full solution code, for both .NET Core and Xamarin.Android SDKs.

The variables section is used to pull in a variable group from Pipelines > Library called CI-Build. Variable groups are sets of parameters that you can define directly in Azure DevOps and pull into your pipeline definitions, so that their values can be controlled outside of source control. Also, we define a new variable called IsMainBranch as well, which is set to a boolean value of True when the current branch we are building is the master branch, or False for any other branch. This variable will be used later on to decide whether to conditionally execute specific steps or not.

To create the variable group CI-Build, open the Azure DevOps project you are using in a browser, and open the Pipelines > Library link from the left-hand navigation bar then click + Variable Group:

 

Then enter the details as per the screenshot below. For the AppleCertificatePassword field, just leave that blank for now, we will revisit it later on when we discuss the iOS side of things. You can use a different name for the group if you wish, just remember to update the YAML to reference the new name you use:


Let's look at the next block of YAML which describes how we test the code:


The first task is straightforward and just installs the latest version of NuGet onto the build agent, using the default settings for the task. The second task uses Visual Studio 2019 to restore all the packages required for building the projects in the solution.

Note the use of the /t:Restore target passed as the value of the msbuildArgs parameter, which instructs Visual Studio to just execute the Restore target against the solution. IVe personally found this to be the easiest way to restore both .NET Core and Mono Android package references in one operation.

The final task in this block runs the unit tests in the project using the test command of the dotnet CLI, which will build any code required for the tests to run as well. In our sample app, this includes the shared .NET Standard project. Note the use of the --no-restore flag as an argument, as we have already restored the solution and don't need to do it again here, so we can save some time by skipping that part.

The reason I've structured the tasks like this is so that the tests build and run as early as possible in the build lifecycle so that we can fail as fast as possible, before we start building and packaging the Android project, which is much more time-consuming.

Let's look at the next set of tasks which builds the Android project, and publishes the resulting *.APK file if required:


The first task in this block, android-manifest-version@1, is used to update the version number of the Android package by updating the Android Manifest XML file that will be used for building the Android project. This task comes from an extension which is developed and maintained by James Montemagno, a well-known Xamarin rep for Microsoft - to use it, you will need to install this extension in your Azure DevOps organisation using the Visual Studio Marketplace:

1) In Azure DevOps, open the Browse Marketplace link which is in the top-right navigation bar:

2) Search for Mobile App Tasks for iOS and Android, the extension should look like the below:

 

3) Click the Get it free link, then select the appropriate organisation in Azure DevOps where you want to install the extension. An administrator will likely need to approve your request, unless you already have the appropriate permissions yourself.


If you can't install the extension or don't want to use it, you can just remove this task from the YAML file, as it's not required to continue with this example, however, I strongly recommend that you use it if possible as it provides a unique version number for each package produced by this build, and becomes important when we release the packages to App Center later on from a release pipeline.

The second task in this block builds the Android project and produces an unsigned APK file, using Release configuration only, as we don't want to produce Debug packages for deployment. The output of the build is placed in the default binaries output directory, which makes it easier to locate the output *.APK file later on for publishing, if needed.

The third task is used to copy the APK package output from the binaries directory to the artifact staging directory, but this only occurs when the build is currently successful and we are building on the main branch, which is defined using an expression on the condition property of the task. If we are running on a different branch, such as a feature branch, then this step will not run.

The final task for this job publishes the contents of the artifact staging directory to the artifact output for the build, under the Android artifact folder. It uses the same condition as the previous task and will only run for main branch builds. This is done so that we can use this output later on in the release pipeline that we are going to build in the last post of this series.


This finishes our Android job. The next job is responsible for building and publishing an *.IPA package for the iOS version of the app. There is a lot more setup involved here, which I'll walk through in detail. Let's continue breaking down the YAML file by looking at the iOS build job in detail:


This YAML defines a job called iOS, which also depends on the previous Android job having completed successfully, if that job fails then the iOS job will not run. You can change this behaviour to always run the job by just removing the dependsOn attribute of the job element, however, in this example it doesn't make much sense to build an iOS package if we haven't successfully run the Android job, so we should still fail the whole build as quickly as possible.

We are using the macOS-latest pool image for our build agent, as in order to build for iOS, we must use macOS with all the relevant tooling installed for performing Xamarin builds. We also define exactly the same variables as we did for the Android job, by pulling in the CI-Build variable group and defining an IsMainBranch variable.

Let's look at the first few setup steps for this job:


The first task installs the .NET Core SDK for the version specified in our build variables, which is currently 3.1.x, meaning, install the latest version of .NET Core 3.1. This SDK is needed for building Xamarin projects on macOS. Note that the target version may already be installed on the agent image, we just want to ensure we are using the latest version of the SDK even if it is not currently installed.

The second task is used to explicitly set the version of the Mono SDK used for building the Xamarin project. This shell script is pre-installed on this image, we just need to invoke it with the version identifier from our build variable MonoVersion, which is currently 6_8_0 in our CI-Build variable group, however I recommend that this be updated to 6_10_0 as well.

Let's look at the next two tasks in our job:


In order to build an iOS project, we have to provide a certificate in *.p12 format that will act as a signing identity. This certificate must be available for download to the build agent when it runs the job, so we will use the Secure Files feature of the Library in Azure DevOps to store this certificate to meet this requirement.

First, we need to create a certificate that we can upload and use for signing and distributing iOS builds produced and deployed in our pipelines.

You will need to have a paid Apple Developer account in order to continue, as well as access to a macOS device that has Xcode 11+ installed. I recommend creating a single Apple Distribution certificate which can be used for Ad Hoc distribution and/or App Store deployment as well. If you already have an Apple Distribution certificate for your team or organisation that you can use, then you can skip the following steps.

Here are the steps to follow to create this certificate and upload to Azure DevOps. First, you will need to generate a Certificate Signing Request (CSR) from your macOS device:

1) Open the Keychain Access utility - I find it easiest to use CMD + SPACE to open Spotlight then type Keychain and hit enter on Keychain Access:

 

2) In the Keychain Access menu, select Certificate Assistant > Request a Certificate from a Certificate Authority..., this will open a new dialog:




3) Enter an email address and common name (eg 'Mobile DevOps AdHoc') to identify your certificate. Select the Saved to disk option, then click Continue:

 

4) Choose where to save the CSR file - I just use the default name, CertificateSigningRequest.certSigningRequest, saved to Desktop.

 

Next, we will create the certificate from this CSR file.

1) On your macOS device, open Safari and sign in to your Apple Developer account

2) From your Developer Account home page, click the Certificates, Identifiers and Profiles link:

 

3) Open the Certificates page from the link in the left-hand navigation, if this is not the default page already selected. Click the + icon next to the Certificates header to create a new certificate:


4) Select the Apple Distribution certificate option and click Continue:


5) You should see a page like this that asks you to upload a CSR file, go ahead and choose the file that you created earlier using the Keychain Access utility, then click Continue to upload:


6) You should now see a page that provides certificate details and a Download link. Click this link to download the distribution.cer file for the certificate.

 

We're nearly there - the final set of steps involves installing the certificate file onto your macOS device, then exporting into *.p12 format with password protection:

1) Find the distribution.cer file you just downloaded (goes to Downloads by default), and double-click it, this will launch the Add Certificates dialog:


2) Make sure the login keychain is selected, so that you can export it correctly, then click Add.

3) Ensure Keychain Access is open, and that you have the login keychain currently open. It's also useful to filter to the My Certificates category. You should be able to see the certificate you just imported, including the private key - the expiry date will be 12 months from the certificate creation date, and the name of the private key should match the common name that you specified in your original CSR file. Select BOTH the certificate and private key lines together, then click File > Export Items...:


4) This will bring up a dialog where you can specify the p12 certificate file name and folder location to export to. Ensure that File Format is Personal Information Exchange (.p12), then click Save:


5) Enter a strong password for the file, make sure you write it down as Azure DevOps will need it in order to be able to open the certificate file later during the build process. You will also need to enter an administrator password for your MacBook to finish exporting the file:

 

Phew! That's a lot of work. Now that we have a valid p12 file and the password for that file, we can upload it to Azure DevOps to use in our build pipeline:

1) In your Azure DevOps project, go to Pipelines > Library, and click the Secure files tab:


2) Click + Secure file, and choose the p12 file you exported from your macOS device earlier. Click OK to upload the file.


3) Once uploaded, click on the filename in the list to open its properties. You can change the filename to another value if you wish. The important step here is to authorise the secure file for use in all pipelines by turning the switch for that setting to ON. If you don't do this, then the first time the YAML pipeline tries to access this file it will fail, and you will have to authorise the pipeline access manually before it can run. Click Save to save changes:


4) Go back to Pipelines > Library, then open the CI-Build variable group that we created earlier, and update the AppleCertificatePassword variable to the password you used when exporting the p12 file, as this is what enables the certificate to be installed. Make sure it is set as a secret variable so that it cannot be viewed, and that you maintain a record of that password somewhere else, as you will have to re-export the p12 file if you lose the password. 

5) In the azure-pipelines.yml file, you will need to update the InstallAppleCertificate@2 task to use the filename of the p12 file exactly as it appears in Azure DevOps. As far as I know, it is NOT currently possible to use a variable to refer to the p12 filename indirectly, it must be hardcoded into the YAML file directly, which is pretty annoying. This is an open issue in Azure Pipelines and may be resolved in future - see https://github.com/microsoft/azure-pipelines-tasks/issues/6885 for reference.


We also need to provide a provisioning profile that links to an identifier and signing certificate for our app, and specifies what devices are allowed to run the app, which will also be downloaded and installed on the build agent. This also involves quite a few setup steps if this is the first time you've ever done this!

Before we can create a provisioning profile, we must first create an App ID that identifies our app. We have two basic choices here, we can either create an explicit App ID for the app bundle identifier, or we can create a wildcard App ID where some or all of the bundle ID is replaced by an asterisk (*), meaning it matches any values from that part of the bundle ID onwards (for example, com.sampiper.* will match com.sampiper.devops, com.sampiper.someotherapp.uat, etc, while * by itself will match ANY bundle ID you want to use).

The main difference between these two types is that explicit App IDs can specify ANY capabilities other than the defaults allowed for any app, whereas wildcard App IDs cannot. For example, if you wanted to enable Push Notifications or HealthKit integration as capabilities for your app, you would need to associate these with an explicit App ID that your account owns. If you just wanted Wallet integration, for example, then you could use either type. Here are the steps to follow to set up an App ID, Device registration and Provisioning Profile:

1) Open a web browser and sign in to your Apple Developer account - note that you can do this part on any suitable laptop, Windows or Mac, as we only need to use the online Developer Portal to create a provisioning profile.

2) From your Developer Account home page, click the Certificates, Identifiers and Profiles link

3) Click the Identifiers link in the left-hand sidebar, then click the + button next to the Identifiers title:

 

4) Select the  App IDs option (the default) and click Continue:



5) Select the App option (the default) and click Continue:


6) Enter a simple description for your app, choose the Explicit or Wildcard option, then enter the Bundle ID you want to use. I've entered the value for my sample app, but you should either use an explicit ID that is unique and owned by your account, or a wildcard (*) if you don't care and just want to try things out. Select any platform capabilities that you want your app to be able to use, and note that the capability list is quite restricted if you use the wildcard option. Finally, click Continue:


7) Confirm that your details are correct, and click Register to save your new App ID:


8) We also need to register AT LEAST one physical device that we can test our app on, which will also be linked to our provisioning profile. We will be using automatic provisioning in App Center later on to specify the actual devices we want to be able to test on when deploying to our hosted environments in App Center, but for the purposes of building and packaging our app into an *.IPA file, we just need one device registered first. To do this, click the Devices link in the navigation sidebar from the Certificates, Identifiers & Profiles home page:


9) Enter the device's platform, name and UDID. To get the UDID value, I recommend using iTunes, as per this link, as it's easier to copy and paste the value back into this page. Click Continue. If the device is valid, it should appear in your list of devices.



10) Now that we have all the components we need, we can create a provisioning profile that we can use to build and package our iOS app in Azure DevOps. From the Certificates, Identifiers & Profiles home page, click the Profiles link in the left-hand sidebar, then click the + button next to the Profiles heading:


11) Select the Ad Hoc provisioning profile option as indicated, and click Continue:



12) Select the App ID that you want to associate with the profile from the dropdown, and click Continue. This must match the bundle ID you are using when you build the iOS app in Azure DevOps, using either an explicit or wildcard matching pattern:


13) Select the Distribution certificate that you created earlier, this should be the same certificate that you exported to a *.p12 file and uploaded to Azure DevOps earlier on:



14) Select the devices that you want to include in this profile then click Continue - for an Ad Hoc profile, you can include up to 100 unique devices, but you only need one for this stage:



15) Enter a name for your profile, then click Generate:


16) Download the profile to your local computer.


Once you have downloaded the provisioning profile to your local computer, upload it to Azure DevOps as a secure file, at Pipelines > Library > Secure Files, using the same process as for the p12 certificate file. I won't go over all the steps to do that again. The same restriction applies for the provProfileSecureFile parameter on the InstallAppleProvisioningProfile@1 task - the name you use must be hardcoded to the name of the secure file you uploaded.

Now that the relevant files have been installed, we can continue with the build. Here's the next set of tasks:


The first task just updates NuGet on the build agent to the latest available version. The second task invokes NuGet to restore all packages in the solution.

The third task is used to update the iOS bundle version, using the ios-bundle-version@1 task from James Montemagno's extension, which I already mentioned earlier in this post. The variable AppleSettingsFileName is used to point to the Info.plist file in our iOS project, which will be updated by this task.

The fourth task in this block is used to build and sign an *.IPA package from the iOS project, using the XamariniOS@2 task. We point directly to the iOS project's *.csproj file, and build for Release configuration as we don't want to deploy Debug packages to App Center. The buildForSimulator parameter is also set to false, as we only want to target real devices for these builds (and this is also why we must provide a signing certificate and provisioning profile as this is a requirement for building and deploying to real devices). The packageApp property will be true for main branch builds, meaning an *.IPA file will be produced and signed, but for other branches, the packaging step will be skipped as we won't be publishing any outputs in that case. We also set runNugetRestore to false as we have already done this in an earlier step.

The args property is used to set the OutputPath property in MSBuild to the default Build.BinariesDirectory variable, so that the build output files will be placed into that directory. This makes it easier to publish the output files to build artifacts later on.

Finally, the signingIdentity parameter is set to the APPLE_CERTIFICATE_SIGNING_IDENTITY build variable, which is automatically set for us when we install the Apple certificate onto the build agent. In the same fashion, the signingProvisioningProfileID parameter is set to the APPLE_PROV_PROFILE_UUID variable which is also automatically set when we install the provisioning profile onto the build agent.

Here is the final block of tasks in our iOS job:


Note that both of these tasks use the same expression for their execution condition - if the build succeeds and is executing for the main branch, then we will copy the relevant build outputs to the artifact staging directory from the binaries directory, which is specified by the default Build.ArtifactStagingDirectory build variable, using the CopyFiles@2 task. We are interested in the output *.IPA file, and any debug symbols that have also been produced so that we can symbolicate crash reports in App Center later on.

To complete this job, we publish any content in the artifact staging directory to the build's artifacts repository, using a folder name of iOS, in the same manner as we did in the Android job.

This finishes our YAML build definition as source code, including all the supporting files we need to be able to run the build. Make sure you've pushed all your changes to your repository before continuing. The next step is to create a matching build definition in your Azure DevOps project that references the YAML source file in your repository, which will allow us to trigger a build instance and view its execution progress:


1) In your Azure DevOps project, select the Pipelines menu option from the navigation sidebar, and click the New Pipeline button in the right hand corner of that page:



2) Next, select the type of repository you are using to host your build definition YAML file. For demonstration purposes, I'll be showing how to create a build pipeline that points to my GitHub repository, but you can easily apply the same process to other types of repositories, such as Azure Repos. I'll continue by clicking the GitHub option:


3) You will need to sign in to GitHub via the OAuth prompt shown next, so that Azure Pipelines can obtain a token it can use to access your GitHub repositories. In Azure Repos this is easier, you just select the repository you want to build from without needing to leave Azure DevOps. Once you've signed in, you must then authorise Azure Pipelines to access your repositories, this will then redirect you back to Azure DevOps.


4) Select the repository you want to build. This will redirect you to GitHub again to approve the repository access, then you will be redirected back to Azure DevOps:


5) You should now see a screen that automatically selects your azure-pipelines.yml file, you can click Run to save your pipeline and queue a new build immediately, or use the Save dropdown option to just save the pipeline if you want to try running later on:



6) When you run the pipeline, you'll first see a summary page similar to the following screenshot. You can click the link 2 Published under the Related heading, which will open the artifact explorer for the build, from here you can download the output packages for the build if it was for the main branch:

 

7) Click on either the Android or iOS job to be taken to the build detail screen, where you can watch your build executing live, or see how a build ran in the past. You can also see detailed logs by clicking on individual steps as well:

 

Wow, that's a bit of a journey! Hopefully your build will succeed the first time it runs, however it can often take several iterations to get a successful, stable build pipeline running in Azure DevOps, so don't feel frustrated if you do still need to make several changes to get everything working. 

In my next post, I'll create a multi-stage release pipeline in Azure DevOps that we can use to deploy the artifacts from this build pipeline to our hosted environments in App Center.

Thanks for reading!