Subscribe
AWS

Setting up Redis on EC2 with AWS CloudFormation

Vishal Ghupta

• 11 min read
Setting up Redis on EC2 with AWS CloudFormation

While working with CloudFormation, most of us might have come across a challenge to set up the Redis Server in EC2 instance, where I have to use the ssh command to connect the instance & change the password of the Redis Server. This process leads to extra steps. Therefore, this needs to be optimized, which will help to reduce and avoid the additional extra steps.

So, we are going to implement Redis Server with Cloud Formation with the user input as a password but before that let’s have a quick understanding of CloudFormation, EC2 & Redis.

Cloudformation + EC2 + Redis

What is CloudFormation?

While configuration & provisioning of any resource in AWS there are two ways through which provisioning & configuration of any resource can be performed. Firstly, it can be done through CLI, or secondly, it can be done through the AWS Console. So, when using the console, you must follow multiple steps to configure resources/architecture. It is not difficult or challenging to configure fewer resources, but in the case of large scale deployments, it is difficult to handle configurations and requires a lot of steps. This is the point where CloudFormation comes into the picture.

  • It provides an Infrastructure as Code (IaC) capabilities which is a process of provisioning & managing your cloud resources by writing a template file that is both human-readable, and machine consumable.
  • It helps to avoid writing down the script to make a bunch of API calls, wait & retry logic, here you just have to describe what you want & tell CloudFormation to do it for you then AWS magically creates it all.
  • It is a free service — AWS only charges for the services you provision via templates.

There are 3 main concepts of CloudFormation which are as follows:

  1. Templates: It is JSON or YAML formatted text file with file extensions as .json, .yaml or .yml, .template or .txt. CloudFormation uses these templates as blueprints for building your AWS resources
  2. Stacks: Collection of resources (AWS Services) in a particular unit is known as a stack. You can create, update, and delete a collection of resources by creating, updating, and deleting stacks. All the resources in a stack are defined by the stack’s AWS CloudFormation template.
  3. Change sets: If you need to make changes to the running resources in a stack, you update the stack. Before making changes to your resources, you can generate a change set, which is a summary of your proposed changes. Change sets allow you to see how your changes might impact your running resources, especially for critical resources, before implementing them.

We will discuss the CloudFormation template in detail further in the blog, till then bear with me 😉

Now moving on to AWS EC2…

What is an AWS EC2?

  1. It is nothing but a virtual server in Amazon Web Services terminology.
  2. It stands for Elastic Compute Cloud.
  3. By using Amazon EC2 eliminates your need to invest in hardware up-front, so you can develop and deploy applications faster.
  4. It enables you to scale up or down to handle changes in requirements or spikes in popularity, reducing your need to forecast traffic.
  5. It is a web service where an AWS subscriber can request & provide a compute server in the AWS cloud.
  6. The instance will be charged per hour with different rates based on the type of the instance chosen.

There are many more details & features of EC2 instance, if you want to explore more about it you can visit the official AWS documentation, but for now, you just need to know a high-level overview of EC2 instance.

What is Redis?

  1. Redis which stands for REmote DIctionary Server is a fast, open-source, in-memory key-value data structure store for use as a database, cache, message broker.
  2. It supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyper log, geospatial indexes with radius queries, and streams
Introduction to Redis
Learn about the Redis open source project
Redis
Modules API for native types
How to use native types in a Redis module
Redis

Create your first CloudFormation template

Since now we have an overview of CloudFormation, EC2 & Redis, we will proceed with the implementation of Self-hosted EC2 with the CloudFormation template.

Now we will create a template.yml file where we will define the resources required for our stack.

Starting with the AWSTemplateFormatVersion and Description:

AWSTemplateFormatVersion: "2010-09-09"
Description: Provisioning Bitnami Redis Server with cloudFormation
  1. It identifies the capabilities of the template & it is an optional attribute to include in the script.
  2. The latest template format version is 2010–09–09 and is currently the only valid value.
  3. Description: is an optional attribute that must consider the string value between 0 to 1024 bytes in length.

Note: AWSTemplateFormatVersion is an implication — sometimes amazon in the future might change the syntax of the CloudFormation template work. Running an old format template in a system that uses a new format by default would break the things, So AWSTemplateFormatVersion would be needed so the CloudFormation service could understand how the instructions are to be processed.

Parameters:

Parameters:
RedisPassword:
Type: String
Description: Redis password

  1. It’s an optional attribute that helps to customize the template.
  2. It enables us to input custom values into a template each time when we create or update a particular stack.
  3. We can have a maximum of 60 parameters for a particular CloudFormation template.
  4. Each parameter must have(mandatory) to have a value at runtime for AWS CloudFormation to successfully provision the stack.
  5. Each parameter must be given a logical name (also known as logical ID). In our case, we have “RedisPassword’’ as a logical name
Resources:
  Logical ID:
    Type: Resource type
    Properties:
      Set of properties

Format:

Resources:
  RedisEc2Instance:
    Type: AWS::EC2::Instance
    Properties:
      InstanceType: t2.small
      ImageId: ami-064e916063a94c38f # ami bitnami redis image id
      Monitoring: true
      SecurityGroupIds:
        - !Ref EC2SecurityGroup
      UserData:
        "Fn::Base64": !Sub |
          #!/bin/bash
          /opt/bitnami/ctlscript.sh stop redis
          password="$(grep -oP '(?<=requirepass )\w+' /opt/bitnami/redis/etc/redis.conf)"
          echo $password
          sed -i "s/$password/${RedisPassword}/g" /opt/bitnami/redis/etc/redis.conf
          /opt/bitnami/ctlscript.sh start redis

AWS CloudFormation templates that declare an Amazon Elastic Compute Cloud (EC2) instance must specify an Amazon Machine Image (AMI) ID, which includes operating system & other software configuration information used to launch instances.

In our case, we will be using Bitnami Redis instance, where Redis is already installed & configured with the default configuration.

InstanceType: t2.small — InstanceType determines the hardware of the host computer used for your instance. Each instance type offers different types of computing, memory, & storage capabilities & based on this are grouped instance families based on these capabilities.

For now, we will be using a low-cost t2.small instance which is designed to provide a baseline level of CPU performance.

Monitoring: true — These attributes specify whether detailed monitoring is enabled for the instance.

SecurityGroupIds — This field accepts a list of security group id. The Security Group acts as a firewall for our instance to control inbound (incoming traffic to our instance) & outbound (outgoing traffic from our instance) traffic. We will discuss further in the blog when we are creating the security blog

UserData —

  1. When we launch an instance in Amazon EC2, we have the option of passing user data to the instance that can be used to perform common automated configuration tasks and even run scripts after the instance starts.
  2. You can pass two types of user data to Amazon EC2: shell scripts and cloud-init directives. You can also pass this data into the launch wizard as plain text, as a file, or as base64-encoded text.
  3. This process happens when the instance is launched for the first time.

It is executed as the root user, so do not use the sudo command in the script.

# stop the redis server
/opt/bitnami/ctlscript.sh stop redis

#extract the redis from the configuration file & store it in 
# password variable
password="$(grep -oP '(?<=requirepass )\w+' 
/opt/bitnami/redis/etc/redis.conf)"

#output the password
echo $password

#restart the redis server
sed -i "s/$password/${RedisPassword}/g" 
/opt/bitnami/redis/etc/redis.conf
/opt/bitnami/ctlscript.sh start redis

The above snippet is a bash script that is encoded in Base64 which would be passed to EC2 instance while launching the instances.

Whenever Bitnami Redis is configured, it has authentication pre-configured in the configuration file which is located at /opt/bitnami/redis/etc/redis.conf.

The trick over here is with the help of the above script will be modifying the Redis password which is passed as a parameter to the CloudFormation script & also we don’t require to login to EC2 & manually change the password

RedisEc2EIP:
Type: AWS::EC2::EIP
Properties:
InstanceId: !Ref RedisEc2Instance

Elastic IP:

  1. An Elastic IP address is a static IPv4 address designed for dynamic cloud computing.
  2. An Elastic IP address is associated with your AWS account.
  3. IPs get allocated to our account and stay the same — it’s up to us to attach them to an instance or not. We could say they are static public IP addresses.

EC2SecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Redis securtiy group
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 6379
ToPort: 6379
CidrIp: 0.0.0.0/0

Here we are creating a security group, where AWS::EC2::SecurityGroup is required as a type to create a security group. For incoming(inbound) traffic we need to define SecurityGroupIngress as a field type which as 4 parameters which are a follows:

  1. IpProtocol: It accepts IP protocol name (TCP, UDP, ICMP, icmpv6) or number
  2. FromPort: We can define the port range in the security group. So FromPort acts as a start range for TCP and UDP protocols, or an ICMP/ICMPv6 type number. So, we will add 6379 as the port number to be accessible.
  3. ToPort: This attribute act as endRange for TCP and UDP protocols, or an ICMP/ICMPv6 type number.

Note: In our case, we need only one port to be open so we will keep the FromPort & ToPort number the same. If you can define range port numbers such as 6379–6388.

4. CidrIp: — This field accepts the IPV4 address range in CIDR format. 0.0.0.0/0 means we are allowing anyone to connect from any IP.

Outputs:

Outputs:
EC2PublicIp:
Description: Public IP address of the specified instances
Value: !GetAtt RedisEc2Instance.PublicIp
EC2PublicDnsName:
Description: Public DNS name of the specified instances
Value: !GetAtt RedisEc2Instance.PublicDnsName

  1. This response can be used to import into another stack or view on the AWS CloudFormation console.
  2. Here we are outputting the public IP & DNS which will be used to connect with Redis.

Since now we have created & understood each aspect of the CloudFormation template.

The final template should look like:

To execute the CloudFormation template, make sure you have installed & configured AWS CLI, if not please install it from the link. Also, we can execute the CloudFormation template from the AWS CloudFormation console.

But for now, we will be using AWS CLI since as a developer it’s considered to be best practice to use the CLI commands. But before executing the template let’s validate our template to avoid the syntax error in the template

$ aws cloudformation validate-template --template-body file://template.yml

The output of the above command will be:

{
"Parameters": [
{
"ParameterKey": "RedisPassword",
"NoEcho": false,
"Description": "Redis password"
}
],
"Description": "Provision Bitnami Redis Server with cloudFormation"
}

Which means our template is valid.

To create a stack in CloudFormation execute the following command:

$ aws cloudformation deploy --template-file template.yml --stack-name test-redis --parameter-overrides RedisPassword=admin123

Once the stack is created, we can check the output by executing the following command:

$ aws cloudformation describe-stacks --stack-name test-redis

Output:

{
"Stacks": [
{
"StackId": "arn:aws:cloudformation:us-east-1:288642165525:stack/test-redis/fa69cfb0-efbc-11ea-8ec8-1245bb6cedee",
"StackName": "test-redis",
"ChangeSetId": "arn:aws:cloudformation:us-east-1:288642165525:changeSet/awscli-cloudformation-package-deploy-1599340550/212515a0-c8b4-40d7-a9fb-f4be44d8a490",
"Description": "Provision Bitnami Redis Server with cloudformation",
"Parameters": [
{
"ParameterKey": "RedisPassword",
"ParameterValue": "admin123"
}
],
"CreationTime": "2020-09-05T21:15:52.380Z",
"LastUpdatedTime": "2020-09-05T21:15:58.224Z",
"RollbackConfiguration": {},
"StackStatus": "CREATE_COMPLETE",
"DisableRollback": false,
"NotificationARNs": [],
"Outputs": [
{
"OutputKey": "EC2PublicDnsName",
"OutputValue": "**.amazonaws.com",
"Description": "Public DNS name of the specified instances"
},
{
"OutputKey": "EC2PublicIp",
"OutputValue": "**.**.***.**",
"Description": "Public IP address of the specified instances"
}
],
"Tags": [],
"EnableTerminationProtection": false,
"DriftInformation": {
"StackDriftStatus": "NOT_CHECKED"
}
}
]
}

Now we have a successfully deployed a CloudFormation stack, we can now test the Redis connection to verify the password has been changed against the input provided.

For this, we need a redis-cli installed on our local machine. If it’s not installed yet, you can download & install it from this link.

Execute the following command to connect with the Redis server:

$ redis-cli -h <public_dns> -p 6379 -a admin123 ping
PONG

Params:

  • h: PublicDNS or Public IP. You will get public DNS after executing the describe stack command.
  • -p: port number of Redis server
  • -a: password to authenticate with the server.

If you receive PONG as the response after executing the command which means you have successfully connected with the Redis Server.

That’s It!

Further, If you want to delete the stack, you can execute the below command:

$ aws cloudformation delete-stack --stack-name test-redis

Note: If you have multiple AWS accounts configured in your local machine, don’t forget to add --profile <profile_name> to each command.

Hi folks, this was my first blog that I am writing down. I am not an expert writer (I’m aspiring to be). I am an IT professional, working from the past 3–4 years. After being immersed in the technology I thought of writing some blogs, on which I am currently working on. I believe after going through this blog, you will get to learn something new, if not please put down your thoughts in the comments section, so that it will help me to improve my skills, as well as it will help me to understand the need of the hour and therefore, I could be better next time. Hope you enjoy it! 😉

Some obvious but helpful links:

What is AWS CloudFormation? - AWS CloudFormation
Introduces the AWS CloudFormation service in this AWS CloudFormation User Guide.
AWS CloudFormation
What is Amazon EC2? - Amazon Elastic Compute Cloud
Use Amazon EC2 for scalable computing capacity in the AWS Cloud so you can develop and deploy applications without hardware constraints.
Amazon Elastic Compute Cloud
Documentation
Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache, and message broker
Redis
AWS::EC2::EIP - AWS CloudFormation
Specifies an Elastic IP (EIP) address and can, optionally, associate it with an Amazon EC2 instance.
AWS CloudFormation
Template anatomy - AWS CloudFormation
Describes the details of the components that you need for an AWS CloudFormation template.
AWS CloudFormation

Interested in working with Axioned as a client or as an employee? Then hit us up! Go here www.axioned.com for further details, including contact details.

Sign up for more like this.

Enter your email
Subscribe