This paper first looks at the pain points and challenges of using a traditional manual setup of infrastructure resources. Here, we look at how Infrastructure as Code, or IaC, addresses those issues. We also examine the added benefits that IaC brings along. Next, we delve into a pragmatic strategy for implementing IaC, and we briefly study some best practices for implementing it. Finally, we introduce Terraform as one of the best tools to implement IaC.
Modern Infrastructure can be Difficult and Costly to Set Up and Maintain
Manual operations have always been the traditional way of deploying computer infrastructure. However, cloud computing and virtualization have dramatically increased the number of infrastructure components to deploy and maintain. Coupled with the agile approach and DevSecOps, this gave rise to more complexities, resulting in an increasing number of new applications required to be more often published and updated.
Here are some challenges and pain points with manual deployment setups. Many of these drawbacks, if not all, may already be familiar to you.
Higher Setup Cost: Setting up each IT environment manually requires dedicated teams of specialized folks responsible for setting up the hardware, middleware, and software. This dramatically increases the deployment and maintenance effort and their associated costs.
Inadvertent Configuration Drift: Configuration settings must be adequately maintained for different deployment environments. However, performing manual tasks each time becomes exceedingly challenging. Eventually, inconsistencies between environments occur, putting the entire infrastructure deployment at risk.
Slow and Tedious Provisioning: Organizations' traditional click-ops deployment methods can consume days or weeks, requiring manual testing at every step. This proves to be tedious, arduous, and slow.
Lack of Automation: Manual interventions made by humans are naturally error-prone. The blunders in the production environment can be unimaginably devastating. In addition, quickly provisioning and scaling infrastructure is a constant challenging demand for the implementation teams to fulfill on time.
Lack of Repeatability and Reusability: Legacy infrastructures struggle to supply the ability to reuse the same codified resources across multiple environments or projects. That is why teams often must create a new one repeatedly, implying more costs.
As a result, propelling demand for more dynamic and flexible infrastructure management techniques began to bloom. This birthed forth the advent of Infrastructure as Code, IaC.
What is IaC, and How can it Solve those Issues?
Infrastructure as Code (IaC) moves from manual operations to automation. It refers to providing and maintaining the deployed resources through machine-readable definition files, i.e., code.
IaC can be understood as recording the desired state of any infrastructure using a declarative language.
IaC can manage physical and virtual resources, including bare-metal servers and the configuration that goes with them. In addition, IaC can also handle high-level components like PaaS (Platform as a service) or SaaS (Software as a Service) services, databases, Kubernetes clusters, etc. It can be applied to any topology: on-premises, hybrid, or full-fledged clouds.
Instead of supporting the deployment through manual processes, the definition files or code uses scripts or declarative definitions to maintain the infrastructure – in a consistent and repeatable way.
Because the productivity gains are exponential, the sooner you define your infrastructure as code, the higher your productivity returns will be. Studies show that the IaC market size is indeed booming.
Source: MarketsAndMarkets, 2022
What are the Benefits of Applying IaC for Your Organization?
IaC gained several outstanding benefits because of its flexibility, including:
Faster Speed: Rather than spending the same time setting up your cloud or on-premises infrastructure in every environment, IaC allows you to quickly build up your whole infrastructure by executing a script. Instead of having discrepancies caused by manual infrastructure management, IaC overcomes this problem by having the same configuration file for each environment, from development to production. Moreover, running the script will quickly generate the same infrastructure in a disastrous event where all infrastructure fails, resulting in fast recovery and less downtime.
Repeatable Services: With IaC, scripts developed for one specific infrastructure can be made as a generic template so that they apply to another infrastructure based on the same topology. This saves money, speeds up the process, and guarantees that the infrastructure is well maintained.
Cost Reduction: Time is money. Rather than completing manual work, engineers can spend more time executing higher-value tasks by automating the infrastructure processes. With IaC, full deployment in multiple environments can take just minutes. Furthermore, the IaC scripts may automatically shut down environments when they are not in use, saving even more money on cloud computing costs.
Risk Reduction and Efficiency: IaC also documents your infrastructure, so you now have a document for every infrastructure step instead of worrying about what will happen when your lead engineer leaves. Every change is recorded, reported, and monitored with IaC. If an issue arises with the new configuration, it can be found and fixed quickly, reducing risk and failure.
Ease of Portability to Other Cloud Scenarios: IaC applies to multi-cloud scenarios, where similar infrastructure is deployed to any cloud provider or local data centers. Developers can use the same tools and configuration files to manage the resources of different cloud providers.
Ease of Integrability in a CI-CD (Continuous Integration and Continuous Delivery) pipeline: IaC enables your team to write, test, and measure those improvements using a CI-CD platform, which empowers your company to deliver value to customers at an increasing rate and to respond with greater agility to changes in development priorities and market trends.
What is a Pragmatic Strategy to Apply IaC?
A simple, deployable framework to introduce IaC in your organization could be as follows.
Get Approval. This is the most important. For your transition to IaC to succeed, you need buy-in from management, your colleagues, and your teams.
Select an IaC Tool. You can opt for the specialized tool tailored to your specific infrastructure (if any). Still, it is best to execute proper due diligence and explore options – the goal here is to select a tool flexible enough to be used in the long run to cover all the IaC concepts.
Organize Training. IaC is easy to understand yet challenging to master. It requires ability and practice for its implementation – and proper knowledge of the tool selected for the implementation. Having dedicated experts/mentors is applicable here for ramping up team ability. The investment will pay off quickly as the teams practice and implement iteratively.
Document the Resources you want to deploy. You cannot automate your deployment if you do not know precisely what to deploy.
Start to Implement the IaC scripts. Once you know what to deploy, you are ready to phase out all your ambiguous, time-consuming, and error-prone manual processes and move to automated, declarative infrastructure code. Start with a few essential resources that you want to experiment with. Version everything to create a single source of truth into a version control system (VCS) like GitHub. Your infrastructure code repositories should reflect the desired state of your infrastructure.
Iteratively Improve your scripts. Time to improve knowledge gradually, carefully perfecting your scripts and their organization. You repeat the process by adding more resources to deploy into your scripts. As the infrastructure to deploy becomes increasingly complex, you will need to learn deeper about the tool you use: its syntax, functions, and tips and tricks. Take your time and consolidate your knowledge, slowly but surely. Make sure to follow the best practices in every single step: once the scripts are completed, the required investment will pay off quickly.
Integrate IaC in a CI-CD pipeline and Apply Dev(Sec)Ops practices. Automate your deployment instead of executing your IaC tool manually. Every time a change is made in the IaC scripts, the CI-CD pipeline should automatically run your IaC tool to apply the necessary change in your infrastructure. It would raise errors and cancel the whole deployment if anything were wrong in the script changes compared to the existing deployment or the enforcing policies.
What about some best practices?
IaC can look simple for small infrastructure, but as the deployment becomes more complex, it is necessary to adhere to best practices to ensure proper implementation while reducing the risks of applying a wrong deployment. The impacts of a misconfiguration on production systems can be devastating. To maximize the benefits of IaC in your organization, learning more and following some proven practices is highly recommended.
In the second part of this blog, we will go into some key implementation details, but here are some examples of best practices:
Provide Version Control for IaC: It is essential when it comes to configuration files. IaC files need to be updated along with the IT environment they help manage. With improved traceability and visibility, you can track, manage, and restore any potential modifications to your infrastructure.
Ensure Modularity in IaC: Small modifications are more straightforward to handle than substantial amounts of work and complex monolithic code. Finding minor, testable script changes that can be implemented one at a time is always helpful. It is also invaluable when reusing groups of resources (modules) shared between multiple parts of your deployment or between different projects.
Be Cautious of hard-coded secrets: Exposed credentials in the IaC code put your organization in significant danger. Having hard-coded secrets might expose passwords to attackers. Do not store such secrets in files submitted to version control systems without using secure secrets storage.
Always Use IaC to Provision any cloud deployment: The IaC scripts are the single source of truth for the state of the deployment. You update the code rather than the infrastructure resources directly to update the infrastructure. Once you use the scripts and carefully design them, you should never return to any manual operation.
Always Use variables: Instead of hard-coding values in the scripts, use custom parameters (or variables) whose values are taken from separate files while executing the scripts. This way, the same scripts can be applied to different deployment environments - simply by supplying the values of the variables for each environment when the IaC tool applies the configuration changes.
Use Composition: The deployment of the infrastructure resources should be seen as multiple individual parts (network, virtual machines, security rules, databases, SaaS, or PaaS Services, etc.) being assembled while taking care of their dependencies. Look at it as playing with a “Lego” toolkit. A good organization of how the scripts are structured can be challenging, but it is needed.
How does Terraform come into the picture of IaC?
The major cloud providers already have specialized products like AWS (Amazon Web Services) CloudFormation, Azure Resource Manager or Azure Bicep, and Google Cloud Deployment Manager… other tools like OpenStack Heat, Cloudify, or Pulumi also propose a specific implementation of IaC.
However, one tool considered to apply the concepts of IaC entirely is Terraform by HashiCorp. Terraform is regarded as the leader in the IaC arena. It is worth noting that, at the same time, many companies still do not have any IaC implementation in place.
Source: DevOps Survey 2022
With Terraform, the lifecycle management and provisioning of all your infrastructure can be done at scale using a standardized method.
Terraform is an automated IaC software that runs on a client. Its key features can be summarized in the following points.
Open-Source: Terraform is free to use, and the open-source version covers most of the requirements. A payable version exists (Terraform Enterprise) for IaC scenarios that require advanced features.
Full Implementation of the IaC lifecycle: Terraform can securely and efficiently build, change, and version management infrastructure. It can also provide custom solutions while managing popular service providers.
Immutability and Execution Plan: Terraform is driven by configuration files that define the infrastructure resources to be managed. Terraform generates an execution plan to create, incrementally change, and continuously manage the specified components by parsing and executing the scripts. If this plan cannot be completed, an error is returned.
Terraform is idempotent: No matter which state you start in, you'll always end up in the same final state.
Extensive Resources to be Deployed: Terraform can manage IaaS-level resources (computing, network, storage, etc.) and higher-level services, such as SaaS applications or PaaS databases.
Support from all major Cloud Providers: Azure, AWS, Google, and Alibaba all provide and publish Terraform modules in a centralized repository called Terraform Registry – those modules can be easily reused and composed together to deploy the whole infrastructure based on IaC concepts.
Vibrant Ecosystem: Terraform can be enhanced with satellite tools that show the cost of the deployments, execute automatic testing and validations, or generate diagrams to document and visualize the current state of the deployed infrastructure easily and automatically.
How to Implement IaC with Terraform?
The second part of this blog will dig deeper into implementing an actual use case and highlight many best practices based on the experience gained from various projects.
Until then, feel free to reach us at Telstra Purple. Our team of expert Terraform consultants can guide you from the introduction of IaC with Terraform into your teams to the implementation and optimization of the scripts – up to the integration of Terraform into a CI-CD pipeline to produce and support a fully automated IaC process.