Tag: RDS

Aurora cluster is the combination of two kinds of endpoints; writer and reader. Sometimes cluster contains lots of readers. Those read replicas are handy to scale out read performance of the database. Typically, database receives lots of read requests and scarce write requests, so possibility to add new read replicas is a nice feature. As indicated, Aurora cluster contains a cluster endpoint (pointing to writer) and reader endpoint (no surprise here, it points to reader(s) ). Writer is the primary endpoint, so that is used when only one endpoint is preferred. Even though the Writer endpoint can handle both writes and reads, the fundamental idea of the utilizing reader endpoint is to route reads to Reader endpoint…

It is extremely quick to ramp up new Aurora read replicas, and as those are all behind cluster’s Reader endpoint, the utilization is fast. AWS take care of loadbalancing between them. Possibility to scale out read performance is good to have but how about scaling out writes? It has not been possible until just recently AWS announced forthcoming Aurora Multi-master and Aurora serverless -services. Those are going to be available in near future. Meanwhile we focus on read replicas and scaling out the read performance. To utilize separated Writer and Reader one must tell the application those endpoints. For Python/Django the story goes as follows.

File to be tuned: settings.py

File to be generated: dbrouters.py

Yes, only one file to be tuned and only one new file to be generated. Sounds piece of cake.

In settings.py, add new database endpoint to the database section and add an additional line to tell Django how to route database traffic: dbrouters = [path.to.dbrouters].

So settings.py should include something like (Django 2.0):

DATABASE_ROUTERS = ['myapp.dbrouters.DBRouter']
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
    },
    'Writer': {
        'ENGINE': 'django.db.backends.mysql',
        'OPTIONS': {
            'read_default_file': 'myapp/databaseWriter.cnf',
        },
    },
    'Reader': {
        'ENGINE': 'django.db.backends.mysql',
        'OPTIONS': {
            'read_default_file': 'myapp/databaseReader.cnf',
        },
    }
}

In this example myapp/databaseReader.cnf and myapp/databaseWriter.cnf -files are located in the same folder than settings.py and those contain information for databases, such as database name, endpoints, ports, and authentications. [for example syntax ->Django docs]

And dbrouters.py that is also located in the same folder as settings.py, should include something like: 

from django.conf import settings

class DBRouter(object):
    def db_for_read(self, model, **hints):
        """
        All reads are routed to Reader
        """
    return 'Reader'

    def db_for_write(self, model, **hints):
        """
        All writes are routed to Writer
        """
    return 'Writer'

    def allow_relation(self, obj1, obj2, **hints):
        “””
        Returning None means that no specific routing
        “””
    return None


    def allow_migrate(self, db, app_label, model_name=None, **hints):
        """
        Returning None means that no specific routing
        """

    return None

Please be aware of that in case you have used ‘.using(“<database>”)’ -notation in code, that will overwrites dbrouters.py. Therefore, all database related routings should be in dbrouters.py. One way to get rid of ‘.using(“<database>”)’, could be for example sed-command to trim the code. [something like sed -i /s<source>/<target>/g *.py ]

-Tero