Pagination and filtering in DRF: lessons from production

We've all been there: a junior engineer ships a Django endpoint that returns 50,000 rows, the frontend times out, and users start complaining. Pagination and filtering in DRF: lessons from production shows us how to avoid this trap entirely. In this article, we'll walk through the patterns that keep API response times predictable and databases healthy, drawing from real incidents we've debugged in trading platforms, e-commerce systems, and SaaS products.

Set a default pagination class globally

The first rule we learned the hard way: never let an endpoint return unbounded result sets by default. When you don't specify pagination, DRF returns everything, and that scales with your data, not with your user's patience.

Set pagination at the REST framework level in your settings file. This gives you a sensible baseline across all list endpoints without repeating yourself:

REST_FRAMEWORK = {
    'DEFAULT_PAGINATION_CLASS': 'rest_framework.pagination.PageNumberPagination',
    'PAGE_SIZE': 25,
    'DEFAULT_FILTER_BACKENDS': [
        'django_filters.rest_framework.DjangoFilterBackend',
        'rest_framework.filters.OrderingFilter',
    ],
}

Now every list view defaults to 25 items per page. Clients can override with ?page_size=50 if they set it up properly, but the default protects you. We've found 25 to 100 is the sweet spot depending on your serializer weight—heavier serializers lean toward lower page sizes.

If you need different pagination for specific views, override it:

class OrderListView(generics.ListAPIView):
    queryset = Order.objects.all()
    serializer_class = OrderSerializer
    pagination_class = LargeResultsSetPagination  # custom class for this view

The key insight: a default exists so developers don't have to think about it. They still can, but they won't accidentally forget.

Cursor pagination for large tables

Page-based pagination works fine up to a point, but we hit a wall with one of our ride-hailing services. Users were hammering GET /trips?page=500, and the database was executing offset 12,500 queries. Offset queries force the database to scan past thousands of rows it's not using—it gets slower the further back you go.

Enter cursor pagination. Instead of skipping rows, cursor pagination passes an opaque token representing your position in the result set. The database uses that token (usually an indexed field like a timestamp or ID) to fetch the next batch in constant time.

Switch to it in settings:

REST_FRAMEWORK = {
    'DEFAULT_PAGINATION_CLASS': 'rest_framework.pagination.CursorPagination',
    'PAGE_SIZE': 50,
}

Then define a custom cursor class with an ordering field:

from rest_framework.pagination import CursorPagination

class TripCursorPagination(CursorPagination):
    page_size = 50
    ordering = '-created_at'
    template = None

Apply it to your view:

class TripListView(generics.ListAPIView):
    queryset = Trip.objects.all()
    serializer_class = TripSerializer
    pagination_class = TripCursorPagination
    filter_backends = [DjangoFilterBackend, OrderingFilter]
    filterset_fields = ['status', 'driver_id']

Clients will get responses with next and previous cursors. They always follow the link, they never construct pagination manually. That's the safety model—clients can't request page 10,000 because the API doesn't expose that concept.

Trade-off: cursor pagination doesn't tell you the total count without an expensive query. If your UI absolutely needs "showing 1 to 50 of 1,234 items," accept the cost or show "more results available" instead.

django-filter for query params

Filtering should be declarative and testable. In production, we've seen filtering logic scattered across view methods, leading to inconsistent behavior and security holes. django-filter centralizes this.

Install it and add to settings:

INSTALLED_APPS = [
    # ...
    'django_filters',
]

REST_FRAMEWORK = {
    'DEFAULT_FILTER_BACKENDS': [
        'django_filters.rest_framework.DjangoFilterBackend',
    ],
}

Define filters on your model or use the simpler approach in your view:

from django_filters import rest_framework as filters
from rest_framework import generics

class OrderFilter(filters.FilterSet):
    status = filters.CharFilter(field_name='status', lookup_expr='exact')
    created_after = filters.DateTimeFilter(field_name='created_at', lookup_expr='gte')
    created_before = filters.DateTimeFilter(field_name='created_at', lookup_expr='lte')
    
    class Meta:
        model = Order
        fields = ['status', 'customer_id']

class OrderListView(generics.ListAPIView):
    queryset = Order.objects.all()
    serializer_class = OrderSerializer
    filterset_class = OrderFilter

Now clients can query GET /orders?status=completed&created_after=2024-01-01 and it works safely. django-filter handles SQL injection concerns and validates input types.

One practice we enforce: only filter on fields you've explicitly declared. It prevents accidental exposure of hidden fields and makes your API contract explicit. If a field isn't in Meta.fields, it's not filterable.

Order by an indexed field

Ordering without indexes is silent performance death. A customer service rep runs GET /tickets?ordering=-customer_email on a table with 2 million rows, and suddenly queries take 30 seconds.

Use DRF's OrderingFilter and ensure the ordering field has a database index:

class Ticket(models.Model):
    id = models.BigAutoField(primary_key=True)
    customer_email = models.EmailField(db_index=True)
    created_at = models.DateTimeField(auto_now_add=True, db_index=True)
    status = models.CharField(max_length=20, choices=STATUS_CHOICES)
    
    class Meta:
        indexes = [
            models.Index(fields=['-created_at']),
            models.Index(fields=['status', 'created_at']),  # for filtered + sorted queries
        ]

Then allow ordering on specific fields in your view:

class TicketListView(generics.ListAPIView):
    queryset = Ticket.objects.all()
    serializer_class = TicketSerializer
    filter_backends = [OrderingFilter]
    ordering_fields = ['created_at', 'updated_at']  # only these, not everything
    ordering = '-created_at'  # default sort

Key rule: ordering_fields is a whitelist. If you don't list it, it's not sortable. This prevents users from sorting on expensive computed fields or unindexed columns.

We learned this in a travel booking system where someone ordered by a calculated "price with fees" field on every request—the database was computing it for half a million rows just to sort. Now we pre-compute and index what matters.

Common mistakes

• Forgetting select_related and prefetch_related with pagination. Pagination doesn't magically fix the N+1 problem. Use select_related for foreign keys and prefetch_related for reverse relations. Paginating 25 items doesn't help if each item needs a database query.
• Setting page_size too high by default. We've seen teams set PAGE_SIZE=1000 thinking it's an "optimization." It moves the problem to JSON serialization and network transfer. Smaller pages are faster for users and more responsive.
• Allowing unlimited page_size override. If you let clients do ?page_size=999999, they will. Add a ceiling:

class PageNumberPagination(rest_framework.pagination.PageNumberPagination):
    page_size = 25
    page_size_query_param = 'page_size'
    max_page_size = 100  # hard limit

• Filtering on unindexed fields in high-volume queries. Always check your database indexes when you add a filterable field. Run EXPLAIN on a production-sized dataset to confirm the index is used.
• Using cursor pagination without a suitable ordering field. Cursor pagination needs a unique or quasi-unique ordering field. If you use status as your cursor field and 50% of rows share the same status, the cursor gets unstable. Combine it with ID: ordering = ['-status', '-id'].

Key takeaways

• Set DEFAULT_PAGINATION_CLASS in REST_FRAMEWORK settings so no endpoint accidentally returns unbounded data.
• Use cursor pagination for large datasets where clients may deep-page; offset pagination is fine for typical UI navigation.
• Declare filters explicitly with django-filter and whitelist what's filterable—never expose unintended fields.
• Whitelist ordering fields and ensure they have database indexes; prevent sorting on computed or missing columns.
• Pair pagination with select_related and prefetch_related to avoid multiplying N+1 problems across pages.
• Set a max_page_size to prevent clients from requesting absurd batch sizes.
• Test pagination and filtering on production-scale data; behavior changes at scale.