This blog post is part one of two blog posts about Infrastructure as Code with Pulumi
1. Introduction to Infrastructure as Code with Pulumi
2. Creating a secure deployment of AWS RDS and Elastic Beanstalk with Pulumi
- Stored in the repository along with your application
- Traceable, changes can be tracked in the same manner as code via commit history and logs
- Empowers DevOps teams
- In-sync with your application
Different types of IaC
Imperative: With the imperative IaC approach, you define how you want to create your infrastructure. You breakdown the end-result into instructions using scripting languages or similar approaches which then create your infrastructure.
Intelligent: The intelligent approach determines the desired state of the system and determines what needs to change before applying changes.
Environment aware desired state is the next generation of IaC.
According to Pulumi website:
In terms of categories of infrastructure as code, Pulumi falls into Declarative category. You can declare the state of your cloud stack in your preferred language, Pulumi analyzes the current state of the environment, determines differences and prepare a set of instructions which transform the current state of your environment to the desired state.
Pulumi currently supports four runtime and multiple languages on those runtimes. For more information, please see Languages.
- Python - Python 3.6 or greater
- .NET Core - C#, F#, and Visual Basic on .NET Core 3.1 or greater
- Go - statically compiled Go binaries
Key features and differences of Pulumi:
- Since Pulimi is using an existing language, you can use techniques in that language or ecosystem of the language for abstraction, code reuse, and defining modules.
- No need to learn a new custom language defined by an IaC framework. For example, if you want to use Terraform, you need to learn new domain-specific-language (DSL) defined by HashiCorp.
- Pulumi figures out dependencies and manages concurrency. For example, when the ID of a resource is used in another resource, Pulumi automatically waits for the creation of the first resource to acquire values required by the other resource and then attempts to create the other resource.
- Most cloud providers have their definition of IaC. This can be in the form of JSON or YAML files which can often result in big bloated files that are hard to read and difficult to reason about your code. However, Pulumi's code is in your preferred language, which is much more familiar and therefore readable than the native option.
You just need to install Pulumi CLI tool on your development environment and CI/CD agents.
Hello world of IaC with Pulumi
I want to go through setting up Pulumi and creating our first stack by what I call "Hello world of IaC with Pulumi". "Hello world" of IaC is going to be something as simple as creating blob storage in your preferred cloud provider platform with your preferred runtime and language. In this post, I am going to create my stack (S3 Bucket) on AWS and also use Node.js runtime with typescript
If you are using macOS, you can use Homebrew to install Pulumi. To install Pulumi on other platforms, you can check Install Pulumi page.
$ brew install pulumi
Login to Pulumi service:
To manage your stack, you need to initialise your Pulumi CLI that sets up storage service which is used to manage your stack's state. Pulumi uses this Service/Storage to keep your current state and history of changes to your stack. So each successful update to your stack by Pulumi also updates this storage to keep track of your stack's changes. There are many options you can use to initialise the Pulumi stack.
- Local storage
- Pulumi SaaS (subscription-based, Software as a Service)
- Cloud blob storage (AWS s3, Azure Blob Storage)
In this example, we use AWS S3 bucket as state storage for Pulumi. To login to the storage, you need to install AWS CLI generate an Access Key and configure your AWS CLI to use that Access Key to communicate with AWS account
Using a terminal, create the Pulumi state bucket
$ aws s3api create-bucket --bucket pulumi-infra-bucket --region ap-southeast-2 --create-bucket-configuration LocationConstraint=ap-southeast-2
$ pulumi login s3://pulumi-infra-bucket
Make sure Node.js is installed on your computer. Run following commands in your terminal.
$ mkdir hello-world-pulumi && cd hello-world-pulumi
$ pulumi new aws-typescript
project name: (hello-world-pulumi)
project description: (A minimal AWS TypeScript Pulumi program)
Created project 'hello-world-pulumi'
Re-enter your passphrase to confirm:
Created stack 'dev'
(set PULUMI_CONFIG_PASSPHRASE to remember):
aws:region: The AWS region to deploy into: (us-east-1) ap-southeast-2
This step could take some time as Pulumi is installing all npm packages and plugins. In the meantime, we can have a look at what Pulumi has created as source code. The folder structure looks like the following image:
Inside this folder, there is
index.ts file which is the primary source code for Pulumi. There is also
package.json which is npm package file. There are two YAML file called
Pulumi.dev.yaml the first file is the general configuration file for Pulumi. Configurations like
description. The other YAML file is per environment. Depending on the number of environments you have you'll have multiple Pulumi.*.yaml files. Currently we only have one environment in the project which is dev therefore we have
Pulumi.dev.yaml. These YAML files are defining the configuration per environment.
In this example, it has aws:region defined for the
dev environment as
These are the main files for the quick hello world example out-of-the-box. But let's have a look in the main file index.ts and see what this sample is building.
There are only three lines of code here. The first line is importing Pulumi/AWS package and alias as AWS. The next line is using this API to create a new S3 bucket called
my-bucket-123. The last line exports the name of the bucket.
Now the initial setup of this sample is done, and we know what it is going to build, let's run this in our connected AWS account and see the result. To run this, you can use the CLI tool and run the following command.
$ pulumi up
Next, Pulumi asks you for the passphrase. Enter the same passphrase you have chosen during the initial setup. Then it analyses the current state of the environment (which in this case is not created yet) and then reports back all the resources missing. It is essential to know that Pulumi does not create the resource straight away; it asks for confirmation if you want to create those resources. This allows you to inspect the details of commands/resource it should create.
Following is the output from Pulumi cli:
Do you want to perform this update? yes
Type Name Status
+ pulumi:pulumi:Stack hello-world-pulumi-dev created
+ └─ aws:s3:Bucket my-bucket-123 created
+ 2 created
Having executed command successfully, Pulumi produces some output. In this case it returns
my-bucket-123-0fe575c. If you notice there is an extra 7 character added to the original name that we gave to the bucket. The reason is that Pulumi can create multiple sets of stacks on the same account/region. If it does not add the postfix to the name, it would be impossible to create the two resources with the same name.
Alright, so that was quite a simple example with Pulumi. This example is what I call the equivalent hello world in Pulumi. In the next blog post we are going to develop a more real-world scenario. We are going to create a secure deployment of AWS RDS and Elastic Beanstalk with Pulumi.