The Database Bottleneck
The following article will discuss issues related to database performance and approaches that can be adopted to mitigate them using read-replicas.
As applications scale, the database often becomes the bottleneck. Read-heavy applications can benefit significantly from read replicas.
What Are Read Replicas?
Read replicas are copies of your primary database that stay synchronized through replication:
┌─────────────┐ ┌─────────────┐
│ Primary │──────▶│ Replica │
│ (writes) │ sync │ (reads) │
└─────────────┘ └─────────────┘
│ │
▼ ▼
INSERT, UPDATE SELECT
DELETE (read-only)Benefits
- Reduced load on primary: Distribute read queries
- Improved read performance: Scale reads horizontally
- Geographic distribution: Place replicas closer to users
- Failover capability: Promote replica if primary fails
Django Database Configuration
Configure multiple databases in settings.py:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'myapp',
'USER': 'myapp',
'PASSWORD': 'password',
'HOST': 'primary.db.example.com',
'PORT': '5432',
},
'replica': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'myapp',
'USER': 'myapp_readonly',
'PASSWORD': 'password',
'HOST': 'replica.db.example.com',
'PORT': '5432',
}
}Database Router
Create a router to direct queries:
# myapp/db_routers.py
class PrimaryReplicaRouter:
"""
Route reads to replica, writes to primary.
"""
def db_for_read(self, model, **hints):
"""
Reads go to replica.
"""
return 'replica'
def db_for_write(self, model, **hints):
"""
Writes go to primary.
"""
return 'default'
def allow_relation(self, obj1, obj2, **hints):
"""
Relations between objects are allowed.
"""
return True
def allow_migrate(self, db, app_label, model_name=None, **hints):
"""
Migrations only on primary.
"""
return db == 'default'Configure the router in settings:
DATABASE_ROUTERS = ['myapp.db_routers.PrimaryReplicaRouter']Handling Replication Lag
Replicas may be slightly behind the primary. Handle this:
class ReplicationAwareRouter:
"""
Router that considers replication lag.
"""
def db_for_read(self, model, **hints):
# Check if we need fresh data
if hints.get('fresh', False):
return 'default'
return 'replica'Usage in views:
# Normally reads from replica
users = User.objects.all()
# Force read from primary for fresh data
user = User.objects.using('default').get(pk=user_id)Context Manager for Transactions
When you need writes and immediate reads:
from django.db import transaction
class UsePrimaryDatabase:
"""
Context manager to force primary database.
"""
def __enter__(self):
# Implementation to route to primary
pass
def __exit__(self, exc_type, exc_val, exc_tb):
# Restore normal routing
pass
# Usage
with UsePrimaryDatabase():
user = User.objects.create(name="John")
# Immediately read the user we just created
user.refresh_from_db()Multiple Replicas
For high read loads, use multiple replicas:
import random
class MultiReplicaRouter:
replicas = ['replica1', 'replica2', 'replica3']
def db_for_read(self, model, **hints):
return random.choice(self.replicas)
def db_for_write(self, model, **hints):
return 'default'Health Checking
Monitor replica health:
from django.db import connections
def check_replica_health(replica_name):
try:
cursor = connections[replica_name].cursor()
cursor.execute("SELECT 1")
return True
except Exception:
return False
class HealthAwareRouter:
def db_for_read(self, model, **hints):
for replica in ['replica1', 'replica2']:
if check_replica_health(replica):
return replica
return 'default' # Fallback to primaryBest Practices
- Monitor replication lag: Alert if lag exceeds threshold
- Test failover: Practice promoting replicas
- Use connection pooling: PgBouncer or similar
- Index replicas: Ensure proper indexes for read queries
- Plan for writes: Primary must handle all writes
Conclusion
Read replicas are a proven pattern for scaling database performance. Django's database routing makes implementation straightforward.
Need help scaling your Django application? Contact Datamart to discuss your architecture.
