Deploy a Load Balancer and multiple Web Servers on AWS instances through ANSIBLE!

Shashi Kant

10 min readMar 16, 2021

Tasks -

Provision EC2 instances through ansible.
Retrieve the IP Address of instances using the dynamic inventory concept.
Configure the Web Servers through the ansible role.
Configure the Load Balancer through the ansible role.
The target nodes of the load balancer should auto-update as per the status of web servers.

Summary: In One-Click Instance Launched, Web Servers provisioned and Load Balancer ready!

prerequisites -

Configure Ansible on controller node.
AWS account with IAM role with its access key and secret key.
Basic knowledge of Ansible.

So, lets get started

we’re going to run all the tasks on localhost (controller node), so there is no need to configure static inventory of controller node.

Configure ansible.cfg File

First we have to configure our ansible.cfg file in the “/etc/ansible/” directory.

[defaults]
inventory = /root/ansible/inventory/myhosts.txt
host_key_checking = False
remote_user = ec2-user
ask_pass = false
private_key_file = /root/mykey.pem
roles_path = /root/ansible/roles/

[privilege_escalation]
become = true
become_method = sudo
become_user = root
become_ask_pass = false

In this file, Add your inventory location, host_key_checking, ask_pass

add remote_user - here, ec2-user because in AWS instances by default root login is not enabled.

To configure Instance ansible have to login into ec2-instance so it also need ssh key to login into it. that why in private_key_file pass your ssh key location (ssh key should be in .pem format).

and in roles_path - pass the location where your ansible roles are present.

Ansible need root power to configure ec2-instances (like Installing the software's, starting the services, etc.) so here we are using sudo method to give ansible power of root user.

Install boto python Library

To connect with AWS cloud Ansible need boto python library installed on Host (Here host is localhost) so, we need to install boto library on our controller node.

Here, I’m installing ‘boto3’ library of python.

pip3 install boto3

Ansible Vault

Before writing playbook we have to create Ansible Vault, to store our AWS IAM user credential.

In this vault we can write our access key and secret key to hide them from world. and we lock this file with some password, so that our credentials will be secure.

To create Ansible vault run below command.

## ansible-vault create <name_of_vault.yml> ##   ansible-vault create vault.yml

after running this command, give some password to lock your ansible vault file. and it will open VI editor for you. Inside this file write your access key and secret key of AWS user.

accesskey: <your aws access key>
secretkey: <your aws secret key>

save this file. your credential will be secure.

Now finally Start writing Ansible Playbook

Ansible Playbook

Now, its time to write Ansible playbook to create EC2 Instance for Launching Apache Web server and Load Balancer.

so, create a workspace in controller node and start writing playbook and extension should be “.yml or .yaml” of file, because we write playbook in yaml format.

- hosts: localhost
  vars_files:
          - vault.yml
  tasks:
   - name: "Launching ec2 Instances on AWS for HTTPD web server"
     ec2:
       key_name: "mykey"
       instance_type: t2.micro
       image: "ami-081bb417559035fe8"
       wait: yes
       count: 2
       instance_tags:
               Name: "ansible_HTTPD_Web_Server"
       state: present
       region: "ap-south-1"
       group_id: "sg-035af5c9f3adccc6c"
       aws_access_key: "{{ accesskey }}"
       aws_secret_key: "{{ secretkey }}"
     register: backend

   - name: "Adding Web_Server instances to host group (httpdserver)"
     add_host:
       hostname: "{{ item.public_ip }}"
       groupname: httpdserver
     loop: "{{ backend.instances }}"

we’re running this part of playbook on localhost, that’s why we write localhost in hosts keyword.

then we pass our vault file name in vars_files. so, when ansible go to AWS to launch EC2 instance then it need those credentials(access key and secret key)

In tasks I use ec2 module to launch ec2 instance, here we have to write.

key_name so that ansible can login into instance and configure it as Apache web server.

instance_type - Here we have to pass our instance type whichever we want. here I use t2.micro instance type.

image - here I’m using Amazon Linux 2 ami.

wait - It is very important keyword here, because I want when this playbook run, till the time my instance launched this play will wait then it will go to the next play. Because my next plays are dependent on this Instance launch.

count - Give no. of instances you want to launch. Here I pass 2 because I want multiple web servers to balance the Load between them through Load Balancer, which we gonna launch below.

instance_tags - Give some tags to your Instance so that it will be easily recognizable.

state - Set instance state. here I put ‘present’ means I have to launch it.

region - Select your region where you want to launch your instance.

group_id - Give some security group-id here. and in security group inbound rule SSH(22) and HTTP(80) port should be allowed.

aws_access_key and aws_secret_key - Write the variable name which are created in vault file.

and all the details of Instance are stored in backend variable through register keyword.

Now come to Next Play add_host

After launching the instances we have to add the public IP of instances in the dynamic inventory of Ansible.

Through add_host module we can add new hosts in dynamic inventory.

Here groupname I’ve given httpdserver and hostname is public IP of instances. And I put a loop here so, that if we launch more than 1 instance then all the instances will be added in the httpdserver group.

   - name: "Launching ec2 instance for Load Balancer server"
     ec2:
       key_name: "mykey"
       instance_type: t2.micro
       image: "ami-0a9d27a9f4f5c0efc"
       wait: yes
       count: 1
       instance_tags:
                Name: "ansible_LB_Server"
       state: present
       region: "ap-south-1"
       group_id: "sg-035af5c9f3adccc6c"
       aws_access_key: "{{ accesskey }}"
       aws_secret_key: "{{ secretkey }}"
     register: frontend

   - name: "Adding Load_Balancer instance to host group (lbserver)"
     add_host:
       hostname: "{{ item.public_ip }}"
       groupname: lbserver
     loop: "{{ frontend.instances }}"

Like wise we have launch web server instances above same way we have to launch Load Balancer Instance. but we have to do some changes here like -

here, I use RedHat ami instead of Amazon Linux 2 in image keyword.

count - I need only one LoadBalancer so, that I put 1 here.

group_id - In security group inbound rule SSH(22), HTTP(80) port and (8080) port for Load Balancer should be allowed.

And in last through register keyword frontend variable will store all the details of Load Balancer Instance.

Now come to Next Play add_host

After launching the Load Balancer instance we have to add the public IP of instance in the dynamic inventory of Ansible.

So, In add_host module, Here groupname is lbserver and hostname is public IP of Load Balancer instance. And I put a loop here so, that if we launch more than 1 Load Balancer then all the instances will be added in the lbserver group.

   - name: "Waiting for SSH service in HTTPD_Web_Server"
     wait_for:
           host: "{{ item.public_dns_name }}"
           port: 22
           state: started
     loop: "{{ backend.instances }}"

   - name: "Waiting for SSH service in LB_Server (Load_Balancer)"
     wait_for:
          host: "{{ item.public_dns_name }}"
          port: 22
          state: started
     loop: "{{ frontend.instances }}"

Now after Launching the Load Balancer and web server instances my playbook is waiting for SSH service enable in instances because Ansible need to login into instances to configure them as web server and Load Balancer. that is why we have to allow SSH port in the security group of instances. Once the Instances is ready to do SSH then it will go on next play.

we have launched backend and frontend instances (web servers and Load Balancer respectively) separately that’s why we have to write wait_for play two times. one is for backend instances (web servers) and second is for frontend instance (Load Balancer).

- hosts: httpdserver
  gather_facts: yes
  tasks:
   - name: "Running role for httpdserver"
     include_role:
             name: httpdserver

- hosts: lbserver
  gather_facts: no
  tasks:
   - name: "Running role"
     include_role:
          name: lbserver

Finally we have to run our Ansible roles on httpdserver and lbserver host group which we just created above, we have to run our Ansible Role named as httpdserver and lbserver. which I will tell below how to create this Ansible Role.

Ansible Role

Ansible roles are basically a collection of files, tasks, templates, variables, and modules. It is used for breaking a playbook into multiple files. This will helps you to write a complex playbook and make it easier to reuse.

Creating Ansible Role

we’ve seen in the last plays that we are using two Ansible Roles called ‘httpdserver’ and ‘lbserver’. But first we need to create them.

So, for creating Ansible Role we need to go to ‘roles’ directory that we’ve specified in the ansible.cfg file.

So, go into ‘roles’ directory in your workspace, where roles will be created and run this below mentioned commands and roles will be created.

ansible-galaxy init httpdserver    ## for Web Server ##ansible-galaxy init lbserver       ## for Load Balancer ##

These commands will be initialize the ‘httpdserver’ and ‘lbserver’ Ansible roles.

Write plays inside ‘httpdserver’ role

Now, we need to write the tasks which will configure web server on our EC2 Instance.

For that go into the “httpdserver” role (or we can say directory) and then go inside the ‘tasks’ folder. There we’ve one file called “main.yml”. we have to edit this main.yml file. and write some plays to configure instance as web server.

- name: "Install httpd and php packages"
   package:
       name:
         - "httpd"
         - "php"
       state: present

 - name: "copy code from GitHub"
   get_url:
       url: https://raw.githubusercontent.com/Shashikant17/ansible_task-creating_roles_for_webserver_and_loadbalancer/main/index.php
       dest: "/var/www/html/index.php"

 - name: "Start httpd services"
   service:
       name: httpd
       state: started

Here I used the ‘package’ module to install ‘httpd’ and ‘php’ packages. Next using “get_url” module we fetch code from GitHub and save it in web server default document root path i.e. ‘/var/www/html/’ with our code file name, here it is ‘index.php’. Finally I’m using ‘service’ module to start the service of httpd server.

Edit main.yml file in tasks folder of httpdserver role.

Write plays inside ‘lbserver’ role

Now, we need to write the tasks to configure HAproxy (Load Balancer) on our EC2 Instance.

For that go inside “lbserver” role and then go into the ‘tasks’ folder. now here also we have to edit main.yml file, and write some plays to configure HAProxy in instance to create a Load Balancer.

- name: "Install HAproxy package"
  package:
    name: "haproxy"
    state: present

- name: "Copying configuration file using template module"
  template:
    src: "haproxy.cfg"
    dest: "/etc/haproxy/haproxy.cfg"
  notify: restart LBserver

- name: "Starting Load Balancer service"
  service:
    name: "haproxy"
    state: started

Here, using ‘package’ module install ‘haproxy’ package. then copy a jinja formatted customized ‘haproxy.cfg’ file through template module in ‘/etc/haproxy/haproxy.cfg’ location. and then we use ‘notify’ keyword to trigger ‘restart LBserver’ play which we gonna create in handlers folder of lbserver role. Finally use ‘service’ module to start the service of haproxy server.

Configuring ‘haproxy.cfg’ file -

In the same location where main.yml file is present in ‘task’ folder of lbserver we have to put this file.

we need to do little changes on this “haproxy.cfg” file.

Make sure on line no. 68 of ‘haproxy.cfg’ file the port no. is ‘8080'.

Configuring haproxy.cfg file for Load Balancer

Here we need to mention our backend server IP’s along with the port number. If you notice the actual line that HAProxy require is 89 no. line. On that line we mention about our backend servers.

Now as we need to update this file dynamically, so we are using ‘for’ loop. Ansible works on Python Embedded Programming called ‘jinja’. In ‘jinja’ we mentioned the starting of the for loop using “{% for %}” and end the loop using “{% endfor %}”.

Next in ‘groups’ keyword whose argument is ‘httpdserver’, means it is our host group. Next ‘hosts’ keyword picks the IP Address from the host group.

So, finally using for loop, we are picking IP’s from host group ‘httpdserver’. That means if we launch hundreds of backend web servers, our “httpdserver” host group will contain their IP’s and we can easily connect with them through this HAProxy configuration file.

Creating ‘handlers’ in ‘lbserver’ role -

So, now go to the ‘handlers’ folder of lbserver, and edit there main.yml file.

In above ‘template’ module play, we mentioned one keyword called ‘notify’. Now in Handlers directory we gonna mention that dependent task.

- name: "restart LBserver"
  service:
     name: "haproxy"
     state: restarted

Now let’s deploy our Ansible Playbook

Go to workspace where is our Playbook and Run it.

## ansible-playbook <playbook_name.yml> --ask-vault-pass  ##ansible-playbook task3.yml --ask-vault-pass

It will run your playbook but just before running playbook this command will ask vault password, give the password of vault and you will see your deployment started perfectly fine.