Begin with Terraform

Completely different 😀 !!

In my recent project I have been involved more into AWS technologies where I need to connect java application with AWS and majorly into infrastructure part but not using AWS console but using TERRAFORM.

What I will be covering in this blog:

• What is terraform?
• Prerequisite - How to connect with AWS?
• Environment information 
• Versioning
• Scripts creating
• AWS Config
• User(s)
• Role(s)
• User-Role relationship
• Group(s)
• Get latest available AMI of linux in your region
• Create EC2 instance using that AMI - and connect using ssh
• Print useful information
• Codebase
• Beginner
• Update 2 - Packer and Modules
• Update 3 - ASG, SSL, ELB, User-data
• Commands
• Glossary
• Images to refer

What is terraform?
Visit terraform-website for details.  But in short it is an automation tool to different cloud providers for managing infrastructure.  

Example - If someone asks you to create 500 EC2 instances or 100 ASGs (Auto Scaling Group), then you need something like terraform.  Make sense?

BTW - I am using terraform version 0.12 for this use case.


Prerequisite - How to connect with AWS?
There are some initial things that you need to do.

• Obviously terraform must be installed
• You must have AWS account - I am using free tier
• For security reasons it is best to have admin group in IAM

Environment information 

• In this use case all terraform (.tf) files are in one single folder.
• All .tf files will be executed/processed when you run any terraform command.
• AWS-CLI and create profile for a user - create config and credentials
• Two files config and credentials must be present under .aws folder of a USER_HOME.
• Terraform picks aws_access_key_id and aws_secret_access_key from credentials file only, so NO need to feed secure information in terraform script.
• Use case carried on OS Win 10 using cmd prompt.

Versioning
Version of software used:

• OS: Win10 Home
• Terraform: v0.12.18
• Packer: 1.5.1
• AWS: aws-cli/1.16.303 Python/3.6.0 Windows/10 botocore/1.13.39

How to set
Let's see various settings.

• AWS Config
• One common file is created which will be used to validate user for all functions/tf-files.
• Refer file aws-config.tf
• User(s)
• I read multiple documents and it is recommended to create-user via console, so I created in same way.  Still refer file create-user-UNUSED.tf
• Group(s)
• User created must belong to group, so whatever group can do that user can do.
• Refer file create-group.tf
• Role(s)
• Roles created and can be assigned to a user.  Each role has its own policy defines which actions are allowed on which resource.
• A user can assume role based on his permissions to perform certain operation. 
• Refer files create-role.tf, create-role-policy.tf and dev-assume-role-policy.json
• User-Role relationship
• Relationship that defines which user has which role.
• Refer file create-user-membership.tf
• Get latest available AMI of linux in your region
• This is important one.  Each region has multiple AMIs.  I created one query which selects latest linux ami created by amazon.  So, file has that logic which connects to AWS and fetches id of AMI and EC2 uses that AMI to create an instance. 
• Refer file get-ami.tf
• Create EC2 instance using that AMI - and connect using ssh
• This file has multiple parts
• First it has logic to create instance from AMI selected in previous step.
• Creates key-pair (you need to create public-private ssh key, so we can login to instance via ssh)
• It passes public key file so that key will be associated to instance and using private key ssh operation should be successful.
• Do not worry if fingerprint generated by your command and AWS is different because logic used by AWS is different, so your private key will still work.
• Important - One last thing that took lot of time for me for ssh - you need to include your specific IP in inbound/outbound rule on all traffic, even though rule of 0.0.0.0/0 is still present.
• I will further analyse it.
• Update (Important) - Now this is no more required.  After creating public subnet and associate subnet with EC2, I can ssh instance using public IP without any other rule.  Cheers 👍 !! 
• Default user/password for ssh is ec2-user/<EMPTY> and you need to pass private-key.  Use public IP to of instance.
• Refer file create-instance.tf
• Print useful information
• This files tells logic of printing any values in case debugging is required.  Only this way I come to know that I misconfigured my region and it was selecting wrong ami.
• Refer file get-user-region.tf

Codebase

• Complete codebase
• https://github.com/svermaji/terraform/
• Beginner
• I would be creating multiple folders in near future for advance scripting, this one is for beginner
• https://github.com/svermaji/terraform/tree/master/1-beginner
• Update 2 - Packer and Modules
• For packer commands are packer validate FILE_NAME and packer build FILE_NAME 
• https://github.com/svermaji/terraform/tree/master/2-modules

• Update 3 - ASG, SSL, ELB, User-data
• Third update has code that will create ASG, and one ELB.  Connects ELB with ASG and EC2 instances with ELB.  Now you can access url via ELB.  
• For HTTPS (SSL), I created my own certificate so if you do same the in browser you might see warning. As per example ELB is listening on 3 ports - 80 (http), 443 (https) and 8000 (http)
• Example also tells about user-data that executes on EC2 startup. 
• https://github.com/svermaji/terraform/tree/master/3-asg
• Update 4 - is about KMS
• Update 5 - is about separting modules
• Update 6 - is about httpd logs configuration
• Update 7 - is about S3 and uploading httpd logs

Commands
Commands used:

• init (only one time - execute in folder where your codebase is present)

Repeatedly 

• plan
• refresh (if you want to print output values)
• apply
• destroy

Commands referred like init, plan, apply destroy etc has below exact commands used:

• To initialize terraform first time
• terraform init
• To create plan that will be applied later
• terraform plan -no-color -refresh=true -out=infra.tfplan
• To apply plan 
• terraform apply -refresh=true -auto-approve "infra.tfplan"
• To refresh, check changes and output values
• terraform refresh
• To destroy resources created by terraform.  
• terraform destroy -auto-approve

• -auto-approve: This flag bypasses manual intervention.
• -no-color: This flag prints all output in single standard color
• Whatever plan is generated should be used in apply command, in above case its infra.tfplan

Glossary

• EC2 - Elastic Compute Cloud
• ASG - Auto Scaling Group
• IAM - Integrated Access Management
• AMI - Amazon Machine Image

Images to refer

Image-1: Result of apply command

Image-2: AWS Instance

Image-3: SSH to an instance

Image-4: Result of destroy command