Website fail over to S3

On a good day everything works fine. Occasionally it is not a good day. Website is something that should be accessible on every day. With AWS that can be reached rather easily. I am going to discuss a solution that utilizes S3 and Route 53.

Steps are not super hard and there are lots of examples around. Yet, those might not cover all the aspects, at least I found that during my endeavour. Below is illustrated steps for basic case (HTTP, no LB).

S3

Naturally we need a static website that will be used in a case of failure at the primary.

  1. Create bucket with the domain name. If your domain is example.com, then create bucket example.com.
  2. Upload static files to your bucket
  3. Turn S3 Website Hosting on
  4. Make uploaded files public (recap: select the file, go to “More”, -> Make public)
  5. Confirm that http://<example.com>.s3-website.<region>.amazonaws.com is working

Route 53

The second AWS service is Route53. Following steps assumes that there is already a domain and hosted zone.  First  somekind of measurements have to be created and then we use to build automation to change the traffic from primary to secondary.

Health Check

  1. Create Health Check

    1. Choose a fancy name
    2. What to monitor: Endpoint
    3. Specify endpoint by : IP*
    4. Protocol: HTTP
    5. IP address: <ip address of your instance>
    6. Host name: <empty>
    7. Port: 80
    8. Path / : <what ever is preferred…>
    9. Advanced configuration -> Changes are not necessary required
    10. SNS notifications: if preferred.

*) Selecting Domain won’t fit here as Domain should be healthy all the time, just the routing should change according to the health check (either to Primary or Secondary).

Hosted Zone

  1. Select your domain’s A-record and modify it

    1. Alias -> no
    2. Routing policy -> Failover
    3. Failover Record Type : Primary
    4. Set ID: Primary (or similar phrase whatever you prefer)
    5. Health Check: yes
  2. Create new Record Set

    1. Alias -> yes
    2. Alias -> s3-website.<region>.amazonaws.com    (Notice that there is no bucket name)
    3. Routing Policy -> Failover
    4. Failover Record Type: Secondary
    5. Set ID: Secondary (or similar phrase)
    6. Health Check: no

And that’s it pretty much. When Route 53 finds that Primary is not ok, then it routes traffic to Secondary. Steps are similar when loadbalancer is in the front of instances. Route53 can utilize loadbalancer’s health check, so configuring a Route 53 Health Check to check health of the instance is not required…

-Tero

AWS reached all time high in the Q4/17

AWS performed extremely strongly last year. Once again, they hit the record. The public cloud service providers -in general- performed very well in 2017, all the big players increased their revenues with double digits. Buzz around cloud remains on the high level. It seems that cloud services are not appealing just for certain industry but all of them. Fast paced business environments are more and more depending on dynamic possibilities of x-as-a-service based solutions.

Figure 1. A section from Amazon’s quarterly result

Based on the quarterly result, the income generated by AWS is highly important for Amazon. Just for the last quarter AWS was responsible for income of $1,35B that was a huge part of the total Amazon’s income. Furthermore, AWS generated 1/10 of all Amazon’s net sales in 2017.  If the trend continues, there is a possibility that AWS becomes bigger than SAP within a few years.

What those financial numbers means for the current year? AWS is starting once again the new year from the leading position. 2017 they introduced 1430 new services and features and it is expected that they will introduce even more this year.  It has begun to be hard to find a nice-to-have service that is not provided by AWS. Yet, the trend is to increase the number of managed services and I expect that trend to continue. Who really likes to be able to look under the hood, people just like to ride.

 

-Tero

 

Loadbalancers

As we know, there are two of them (yes, one might say that three but I ignore the classic one here…). Application LB and Network LB. What is the difference and most of all, when one should prefer the former and when the latter. There are good pack of info provided by AWS that we all should be aware of. Yet, I share some of my own findings…

If the internet traffic is proper packets (HTTP/HTTPS), ALB provides such nice tools to control the traffic among various target groups. One can utilize either host or path routing or both. That enables lots of nice ways to route traffic  for example for containers. Also as the ALB operates on HTTP/HTTPS level, one can let it handle the SSL termination. SSL certifications are pretty easy task to handle in AWS, by the way (more about SSL).

The figure 1 below is a snap of the scenario that single ALB routes traffic for two domains . With host based routing it is possible to route traffic to various target groups accordingly.  And of course it is possible to add also path based routing to it.

Figure1. ALB and host based routing

Network LB is on the other hand the preferred one in a scenario that traffic destination must be IP address, not domain. That could be for example SMS messages that your application is receiving / sending through SMS proxy. Or the traffic is not HTTP/HTTPS… Or something else interesting. Well the bottom line is that the NLB works with TCP level not HTTP/HTTPS level. There is a catch, of course. The IP-address is AZ specific, i.e. for example in Ireland Region there can be defined 3 IP addresses; one for each AZ. Furthermore, AZ-specific IP address more or less  connects application to be only one AZ.  That means that multi-AZ designs for HA and DR should require extra effort…

On architecture point of view, the best way is to place LB in public subnets (in all AZs) and webservers in private subnets (figure 2). LBs will use the private IP-addresses of the EC2. Actually all communication inside AWS is between private IPs, so EC2 point of view the traffic comes from LBs private IP address. That might be a problem in some applications as private IP addresses of loadbalancers are not easy to find out.

Figure 2. Private and public subnets

 

-Tero

Aurora cluster – Writer to handle writes and Reader to handle reads

Aurora cluster is the combination of two kinds of endpoints; writer and reader. Sometimes cluster contains lots of readers. Those read replicas are handy to scale out read performance of the database. Typically, database receives lots of read requests and scarce write requests, so possibility to add new read replicas is a nice feature. As indicated, Aurora cluster contains a cluster endpoint (pointing to writer) and reader endpoint (no surprise here, it points to reader(s) ). Writer is the primary endpoint, so that is used when only one endpoint is preferred. Even though the Writer endpoint can handle both writes and reads, the fundamental idea of the utilizing reader endpoint is to route reads to Reader endpoint…

It is extremely quick to ramp up new Aurora read replicas, and as those are all behind cluster’s Reader endpoint, the utilization is fast. AWS take care of loadbalancing between them. Possibility to scale out read performance is good to have but how about scaling out writes? It has not been possible until just recently AWS announced forthcoming Aurora Multi-master and Aurora serverless -services. Those are going to be available in near future. Meanwhile we focus on read replicas and scaling out the read performance. To utilize separated Writer and Reader one must tell the application those endpoints. For Python/Django the story goes as follows.

File to be tuned: settings.py

File to be generated: dbrouters.py

Yes, only one file to be tuned and only one new file to be generated. Sounds piece of cake.

In settings.py, add new database endpoint to the database section and add an additional line to tell Django how to route database traffic: dbrouters = [path.to.dbrouters].

So settings.py should include something like (Django 2.0):

DATABASE_ROUTERS = ['myapp.dbrouters.DBRouter']
DATABASES = {
    'default': {
        'ENGINE''django.db.backends.sqlite3',
        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
    },
    'Writer': {
        'ENGINE''django.db.backends.mysql',
        'OPTIONS': {
            'read_default_file''myapp/databaseWriter.cnf',
        },
    },
    'Reader': {
        'ENGINE''django.db.backends.mysql',
        'OPTIONS': {
            'read_default_file''myapp/databaseReader.cnf',
        },
    }
}

 

In this example myapp/databaseReader.cnf and myapp/databaseWriter.cnf -files are located in the same folder than settings.py and those contain information for databases, such as database name, endpoints, ports, and authentications. [for example syntax ->Django docs]

And dbrouters.py that is also located in the same folder as settings.py, should include something like: 

from django.conf import settings

class DBRouter(object):
    def db_for_read(self, model, **hints):
        """
        All reads are routed to Reader
        """
    return 'Reader'

    def db_for_write(self, model, **hints):
        """
        All writes are routed to Writer
        """
    return 'Writer'

    def allow_relation(self, obj1, obj2, **hints):
        “””
        Returning None means that no specific routing
        “””
    return None


    def allow_migrate(self, db, app_label, model_name=None, **hints):
        """
        Returning None means that no specific routing
        """

    return None


Please be aware of that in case you have used ‘.using(“<database>”)’ -notation in code, that will overwrites dbrouters.py. Therefore, all database related routings should be in dbrouters.py. One way to get rid of ‘.using(“<database>”)’, could be for example sed-command to trim the code. [something like sed -i /s<source>/<target>/g *.py ]

-Tero

SSL Termination

Application loadbalancers are handy tools to handle SSL termination. It is not a huge task for webservers but still, if one can let AWS to do the job, why not?

Basically, it means that LB will have HTTP and HTTPS listeners but it routes traffic to target groups as HTTP. Target groups should be in private subnets and inbound rules in target groups’ security groups should allow traffic only from LB’s security group.

Figure1. Loadbalancer in public subnets and EC2 instances in private subnets

SSL-termination task begins with SSL-certification generation. With AWS that task is easy, especially when premium DNS service Route53 is used. During LB creation, select (add) HTTPS listener, pick all your AZs with public subnets, and then moving forward to SSL & ACM. Click “Request a new certificate from ACM”.

Figure2. Configuring HTTPS for ALB

On ACM screen, write your domain name (don’t forget to add *.domain).

Figure3. Request a certificate

On Step 2, select DNS validation, and then Review -> Confirm&Request.

Figure4. DNS validation during the new certification request process 

On Validation phase, just click on Create record in Route 53, and voila, you will have SSL certification for your domain. You will find a new Record set (CNAME) in your domain’s hosted zone. The SSL-certification is added to ACM and you don’t have to remember to renew the certification. Life becomes a bit easier.

Figure5. Validation phase

Now when you go back to LB creation, refresh the Certification name list and you should find the brand new certification on the list. Select that and move forward selecting Security groups and target groups.

Now your ALB is listening HTTP (80) and HTTPS (443) and routing traffic to target group with HTTP (80 and 443) and target group don’t have to take care of SSL termination.  The final touch will be on webservers. Then Rewrite rules & Rewrite conditions should be defined for Apache or NGINX to make sure that they are on the same page (accepting HTTP).

For example something like (Apache):

<VirtualHost *:80>
...
ReWriteEngine on
RewriteCond %{HTTP:X-Forwarded-Proto} !https
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]
...
</VirtualHost>

-Tero

AWS DevDay Nordics 25.1.2018

Full house at Clarion

When I arrived at the scene the  further orienteering was simple. Just follow the queue. After registration, I wrapped some snacks and decided to reserve my seat in the containers session. The decision between tracks was not simple, though. There were three simultaneous sessions and two workshops… Lots of content for a single day!

The containers track was hosted by AWS gurus Abby Fuller and Paul Maddox. All the empty seats disappeared well before the official start.  The first  session included good insights to ECS, EKS and Fargate services. Especially EKS part was waited for as it includes some demonstration of the new service. Slides and recoded videostream will be available by AWS some day soon (link to Slides and Recordings). That enables not only a possibility to refresh own notes from all those sessions but provides also opportunity to get insights from interesting overlapping tracks that I was not able to attend.

ECS

ECS is the oldest service among these three, and unlike the new ones, it is used in production at the moment. The new insight was relating to Fargate, as new ECS clusters can be launched in Fargate mode (Fargate’s preview mode is only available in us-east-1). Fargate mode might be interesting option in those scenarios where there are no need to have SSH access to EC2.

Fargate

Fargate is, as previously indicated, managed service that relief some of the pain that relates to backend infrastructure configurations. Using Fargate, one really have only to define memory and cpu for the task. AWS will takes care of the rest. The default values (at the moment) for memory is 0,5G and CPU 0,25vCPU. And naturally define how many healthy tasks there have to be up and running at all the time. Fargate seems to be one of those services that I will look deeper. 

EKS

Kubernetes is will be integrated into AWS, at least for orchestrating perspective. EKS was and is one of the most waited for service, for me at least. Ramping up EKS with AWS Console is pretty quick and neat. The sneakpeak revealed that during the preview phase the process includes a couple of Cloudformation stacks but those are expected to be hided under the hood later on. After those initial setups, there is not much to do than a couple of clicks and kubernetes cluster is up and running. EKS provides a cluster endpoint, something like my-cluster.EKS.<region>.amazonaws.com. Behind that endpoint there is the kubernetes master and etcd. Migration to EKS seems to be piece of cake as kuberctl works as before.

I also attended AWS meet-up Finland at Siili Solutions. Nicely the event was on the same day than AWS DevDay Nordic. It was fully buuked event and there were also people on the wait list. I was expecting a full house but reason beyond my knowledge, for some reason a quite many people had faced force majeure that not only hindered them to participate but also to inform organizators about their absence. Hopefully people that were on the wait list don’t read this… Anyhow the panel discussion with the AWS gurus really wrapped up the AWS day.

-Tero