Create a highly scalable Drupal site on AWS

Difficulty : Intermediate


These days organizations using AWS need highly-scalable and highly-available server infrastructures in order to host their web applications securely. They want to be able to quickly deploy applications to servers, connect the same database from multiple places. In order to do this, we will be exploring how to create a highly scalable Drupal site in AWS on a Centos server in this tutorial. This site is an example demo of a site on a highly scalable server architecture described by the article.


Table of Contents:

  1. Getting started
  2. Creating a security group & SSH Keys
  3. Setting up the Virtual Private Cloud (VPC), subnets & bastion host
  4. Altering the route tables for the new VPC
  5. Creating Relational Database System (RDS) Aurora Cluster
  6. Adding an auto-scaling policy to the new Cluster
  7. Creating an Elastic File Storage (EFS) & filemount.config file
  8. Configuring Drupal & creating an .ebextensions folder
  9. Deploying Drupal into a new Elasticbeanstalk environment

1. Getting started

In order to start setting up this highly-available Drupal setup, you will need to have some working knowledge of basic AWS services and the ability to run command line through terminal or putty. We will be using the latest version of Drupal 7 from the public repo as well as php 5.6.

 

2. Creating a security group & SSH keys

  • First make a key-pair in order to be able to SSH into the bastion host that will be created in the following steps by going to “Services” and “EC2” under “Compute
  • On the left-hand side, click “Key Pairs” and then “Create Key Pair” to enter the name of the key pair and create it.

  • Next in the same screen, go to “Security Groups” and “Create Security Group
  • Fill out the name and description, select the correct VPC and then click “Add Rule” on the Inbound tab
  • For Inbound traffic we want components within the security group to have access to each other, so set it up like the following:
  • This allows all traffic within this security group.  (NOTE: you will not have the security group name until you create the security group, so first create the security group with the second rule below)
  • Click  “Add Rule” again to add your local machine ip to the allowed list:
  • The type is SSH, the protocol is TCP, and the Port Range is 22.  For the source, select the “My IP” value in the dropdown.
  • Once you have those two rules added, you can move on to the VPC creation.

 

3. Setting up the VPC, subnets & bastion host

Now we can start creating the VPC itself. The VPC will allow us to define the subnets that live in the public and the private regions of our cloud and designate the proper components to each.

  • Go to “Services” and click “VPC” under “Networking & Content Delivery” Click on “Launch VPC Wizard” & select “VPC with Public and Private Subnets” subnets

  • In the following screen, fill out the fields as below replacing the {vpc name} with your preference, choosing the server type for the bastion, and naming the public and private subnets:

Once you click “Create VPC”, AWS will create:

  • A VPC with a public and a private subnet
  • A routing table and Network Address Translation (NAT)
  • A bastion host with the given key pair in the public subnet
  • An elastic IP for the bastion host to access the internet

We still need to create another subnet in a different availability zone in order to make this a highly available setup, so let’s do that now while we are in the VPC subnet service of AWS. Optionally, we need to make a fourth private subnet to use an application load balancer later on.  So if you plan on using an application load balancer, create an additional subnet in a different availability zone than the ones above.

  • Go to “Services” and “VPC” under “Networking & Content Delivery
  • Click on “Subnets” and then “Create subnet”
  • Fill out the name of the new private subnet, select the VPC you just created, and set the availability zone to something besides the one you used in the creation of the VPC

  • As mentioned above, if you are using an application load balancer, you must make another subnet.  It must be in a different availability zone and can use IPv4 CIDR block ip 10.0.3.0/24.

 

4. Altering the route table for the new VPC

We will need to set the two new private subnets for the main route table for the VPC.

  • Go to “Services” and “VPC” and then to “Route Tables
  • Click the checkbox of the row that has column Main set to “Yes” and the correct VPC name in the VPC column
  • In the pop-up, go to Subnet Associations, click “Edit” and associate both private subnets
  • Everything else should be correct by default:

 

5. Creating Relational Database System (RDS) Aurora Cluster

Once we have the VPC built out, we can build out the RDS.  We will be making an Aurora clustered MYSQL RDS in this setup.

  • Go to “Services” and click “RDS” under “Database
  • Click “Create Database” and select Amazon Aurora which should be the first tab.

  • Choose MySQL 5.6-compatible for this example and click next.

  • In the configuration section, either choose Provisioned or Serverless.  Provisioned allows you to manually select server instance sizes.
  • If you choose Provisioned, you can specify the server type that gets created when scaling up and down:

 

  • If you choose Serverless, specify a minimum and maximum Aurora capacity:

  • Serverless allows you to select the minimum and maximum Aurora capacity units and automatically scales up and down based on application load.  If you choose Provisioned, continue to step 5 below, otherwise you can skip manually adding an auto-scaling policy as the Serverless option does that automatically for you.
  • Click create to start creating the Aurora clustered database.

 

6. Adding an auto-scaling policy to the new Cluster

In order to add an autoscaling policy on a provisioned Aurora cluster database, you need to access the cluster.

  • Go to “Services” and click “RDS” under “Database
  • Click on “Clusters” on the left-hand side and then the correct cluster row.
  • Click the “Actions” dropdown in the top right, and then click “Add Auto Scaling Policy“.

  • Fill out the policy name, the target metric – Average CPU Utilization or Average connections, the target value to threshold, and the cluster capacity.
  • This policy will automatically attach to your cluster when you click “Add Policy

 

7. Creating an Elastic File Storage (EFS)

Next we need to separate the website file system for I/O from the actual codebase to prevent new files being wiped out when a new instance spins up.  We can do this by creating an Elastic File Storage system and a config file to mount the file system on server boot-up.

  • Go to “Services” and click “EFS” under “Storage” and then click “Create file system

  • Select your correct VPC, and you should see the subnets in that VPC listed.
  • Remove the default security group on each subnet and replace with the security group you created earlier that allows traffic within the security group.  This will allow you to access the file system when it is mounted on the EC2 servers.
  • Click “Next Step” and then select a name, the performance mode, throughput mode, and whether or not to encrypt the data:

  • Click “Next Step” and then “Create File System

 

8. Creating an .ebextensions folder & filemount.config file

Whether you are doing a fresh install of Drupal or you have a site you are deploying to elasticbeanstalk, you can configure an .ebextensions hidden folder in the root of the drupal site for installing software on server creation. Optionally, you can add a beanstalk-settings.php file to set some drupal and server settings for AWS.  We won’t set that up in this tutorial.

  • Open terminal or putty and enter mkdir .ebextensions and cd .ebextensions/
  • Once inside that .ebextensions folder, enter sudo vim filemount.config
  • Enter the following code into that filemount.config file:
packages:
 yum:
  fuse: []

files:
 "/opt/elasticbeanstalk/hooks/appdeploy/pre/11_unmount_filemount.sh":
  mode: "000755"
  owner: webapp
  group: webapp
  content: |
  #!/usr/bin/env bash
  if mountpoint -q /var/www/html/sites/default/files; then
   sudo umount /var/www/html/sites/default/files
  fi

 "/opt/elasticbeanstalk/hooks/appdeploy/enact/02_mount_filemount.sh":
  mode: "000755"
  owner: webapp
  group: webapp
  content: |
  #!/usr/bin/env bash
  sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport { file-system }:/ /var/www/html/sites/default/files

This script is fairly straightforward and does a couple of things:

  • Installs the fuse package in case you need it in the future.
  • Allows the file system to be unmounted in case you are deploying to the same servers where it is already mounted
  • Allows the file system to be mounted to the sites/default/files folder of drupal
  • NOTE: You will have to replace the bracketed line with the actual filesystem name you created in the above steps.

 

9. Deploying Drupal into a new Elasticbeanstalk environment

You can now start deploying your drupal application into a new elastic beanstalk container.

  • Go to “Services” and click “Elasticbeanstalk” under “Compute
  • Click the “Actions” dropdown and select “Create environment
  • Select Web server environment and fill out the information in the form.

 

  • For Preconfigured platform, choose PHP and for Application code choose upload your code and select the zip of your configured drupal site
  • Select “Configure more options

  • You will see several boxes for each configuration aspect of the container.
  • Start with the Capacity box to change your environment type to Load Balanced, select the min and max number of instances, and click Save.
  • Next go to the Network box to select your VPC.  For the load balancer, check the public subnet boxes.  For the instances, check the private subnet boxes.
  • Next go to the Instances box to set the default server type and the default security group to be attached to the EC2 instances that get created on scale-up.  For the security group, choose the security group you created earlier that allows traffic within the security group.

  • Next go to the Load Balancer box to select between classic and application load balancers and input ssl listeners.

  • Here you can specify your rules and processes as well for your particular applications.
  • After this, go to the Security box to add the key pair you made earlier to the EC2 instances that will be created when you scale up.

  • Next go to the Software box where you can add environment variables to abstract out the database credentials or any other sensitive variables from your application.  This is highly recommended, and you would have to alter your settings.php to use those environment variables in the database credential section.
  • Finally go to the Notifications box to enter an email which will subscribe to notifications about the status of the container you are about to create.
  • Click Create environment to start creating this container and the rest of the application.  If everything goes well, you should have the container up in a couple of minutes as a green block on Elasticbeanstalk page.

Hopefully, this article has shown you how you can deploy a highly scalable and available drupal application in your own environments.  Let us know your thoughts in the comments section below!

Categorised in: Uncategorized

This post was written by Ajay Alkondon

Leave a Reply

Your email address will not be published. Required fields are marked *