Pagination in Go APIs: a practical guide for developers

Pagination in Go APIs: a practical guide for developers

Most APIs need to return large lists—orders, users, logs, transactions. Fetching millions of records at once will crash your database and client. Pagination in Go APIs is a practical guide for developers who need to return large lists efficiently, and choosing the right pagination strategy can mean the difference between a responsive API and one that degrades under load.

Why pagination matters in production

We've all seen the naive approach: fetch everything, return it all, hope the client has enough memory. In production, this doesn't work. A customer with 100,000 orders shouldn't download all of them in one request. Your database shouldn't scan the entire table every time someone asks for page 50. Your network bandwidth is finite.

Pagination solves this by splitting results into manageable chunks. But not all pagination strategies are equal. Some are simple but perform terribly at scale. Others require careful implementation but reward you with predictable query performance regardless of how deep into the dataset a user scrolls.

Limit and offset — Simple, slow for deep pages

Limit-offset pagination is what most developers reach for first. You send a limit (how many records) and an offset (how many to skip), and the database returns that slice.

type ListOrdersRequest struct {
    Limit  int `query:"limit"`
    Offset int `query:"offset"`
}

func (s *OrderService) ListOrders(ctx context.Context, req ListOrdersRequest) ([]Order, error) {
    query := "SELECT id, customer_id, total, created_at FROM orders ORDER BY id DESC LIMIT $1 OFFSET $2"
    rows, err := s.db.QueryContext(ctx, query, req.Limit, req.Offset)
    if err != nil {
        return nil, err
    }
    defer rows.Close()

    var orders []Order
    for rows.Next() {
        var o Order
        if err := rows.Scan(&o.ID, &o.CustomerID, &o.Total, &o.CreatedAt); err != nil {
            return nil, err
        }
        orders = append(orders, o)
    }
    return orders, rows.Err()
}

This works fine for the first few pages. But imagine a user asking for page 1000 with a limit of 50. The database must scan and discard 50,000 rows before returning the result. At page 10,000, you're scanning 500,000 rows. This gets exponentially slower, and it happens on every request.

On a busy platform, offset pagination on large tables becomes a DoS vector. A single user repeatedly requesting deep pages can consume significant database resources. We've seen this in production: offset-heavy APIs become the bottleneck during reporting season when analysts run deep pagination queries.

When to use limit-offset: Small datasets (a few thousand rows), internal APIs where clients are trusted, or situations where random access to any page is mandatory. For everything else, cursor pagination is better.

Cursor pagination — WHERE id < $cursor ORDER BY id DESC LIMIT $n

Cursor pagination fixes the performance problem by using a marker (cursor) to tell the database where you left off, rather than how many rows to skip.

The concept is simple. On the first request, you return the first N records. You also return a cursor—typically the ID of the last record. On the next request, the client sends that cursor back. Your query becomes: "Give me the next N records where ID is less than the cursor."

type ListOrdersRequest struct {
    Limit  int    `query:"limit"`
    Cursor string `query:"cursor"`
}

type ListOrdersResponse struct {
    Orders     []Order `json:"orders"`
    NextCursor string  `json:"next_cursor,omitempty"`
}

func (s *OrderService) ListOrders(ctx context.Context, req ListOrdersRequest) (*ListOrdersResponse, error) {
    limit := req.Limit
    if limit <= 0 || limit > 100 {
        limit = 20
    }

    var cursorID int64
    if req.Cursor != "" {
        decoded, err := decodeCursor(req.Cursor)
        if err != nil {
            return nil, fmt.Errorf("invalid cursor: %w", err)
        }
        cursorID = decoded
    }

    var query string
    var args []interface{}

    if cursorID == 0 {
        // First request
        query = "SELECT id, customer_id, total, created_at FROM orders ORDER BY id DESC LIMIT $1"
        args = []interface{}{limit + 1}
    } else {
        // Subsequent requests
        query = "SELECT id, customer_id, total, created_at FROM orders WHERE id < $1 ORDER BY id DESC LIMIT $2"
        args = []interface{}{cursorID, limit + 1}
    }

    rows, err := s.db.QueryContext(ctx, query, args...)
    if err != nil {
        return nil, err
    }
    defer rows.Close()

    var orders []Order
    for rows.Next() {
        var o Order
        if err := rows.Scan(&o.ID, &o.CustomerID, &o.Total, &o.CreatedAt); err != nil {
            return nil, err
        }
        orders = append(orders, o)
    }
    if err = rows.Err(); err != nil {
        return nil, err
    }

    var nextCursor string
    if len(orders) > limit {
        orders = orders[:limit]
        nextCursor = encodeCursor(orders[len(orders)-1].ID)
    }

    return &ListOrdersResponse{
        Orders:     orders,
        NextCursor: nextCursor,
    }, nil
}

Notice we fetch limit + 1 records. If we get that extra record, we know there are more results, and we encode the last record's ID as the next cursor. If we don't get the extra record, we've reached the end of the list.

The database can use an index on the id column to find the cursor position in constant time, then scan forward linearly. No matter whether you're on page 2 or page 200, the query takes roughly the same amount of time. This is the performance win.

Encode cursors — Base64 an opaque cursor for clients

Never expose raw IDs or database internals in your API. Cursors should be opaque from the client's perspective. If you expose id: 12345 as a cursor, clients might try to manipulate it. They might send id: 1 to "go back," breaking your pagination contract. They might send random numbers and hammer your database.

Encode cursors in Base64. It's not encryption, but it's enough to signal to clients that this is an opaque value they shouldn't interpret.

func encodeCursor(id int64) string {
    return base64.StdEncoding.EncodeToString([]byte(fmt.Sprintf("%d", id)))
}

func decodeCursor(cursor string) (int64, error) {
    decoded, err := base64.StdEncoding.DecodeString(cursor)
    if err != nil {
        return 0, err
    }
    id, err := strconv.ParseInt(string(decoded), 10, 64)
    if err != nil {
        return 0, err
    }
    return id, nil
}

Some teams use more sophisticated cursor encoding—including timestamps, sort order, or multiple fields—so the cursor becomes self-describing. For simple single-sort pagination, this is overkill. For complex sorts or multi-column cursors, it's worth the extra complexity.

Cap page size — Never trust client-sent limits

A client might send limit=999999 thinking they're being clever. Your API should reject or clamp this. Always validate the limit on the server.

const (
    DefaultLimit = 20
    MaxLimit     = 100
)

func validateLimit(limit int) int {
    if limit <= 0 || limit > MaxLimit {
        return DefaultLimit
    }
    return limit
}

This serves two purposes. First, it protects your database from expensive queries. Second, it makes your API contract predictable. Clients can't force unexpected performance characteristics by tuning the limit.

We typically set a max limit between 50 and 200 depending on the payload size. For lightweight endpoints, 200 is reasonable. For endpoints returning large nested objects, 50 is safer.

Common mistakes

• Forgetting to validate the limit. Clients will send absurd values. Clamp them server-side every time.
• Using offset pagination for very large tables. The query time degradation is real. Benchmark at 100,000 rows and you'll see it.
• Exposing raw IDs as cursors. Clients will try to game it. Encode everything.
• Not handling the "last page" case. If you fetch limit+1 and get exactly limit, you're not at the end. If you get fewer than limit, you are. Make this explicit.
• Assuming cursors are portable across API versions. If your schema changes, old cursors might break. Document the cursor format or version it.
• Forgetting indexes on sort columns. Cursor pagination is fast only if the database can index the WHERE clause efficiently. Verify your indexes are in place and being used.

Key takeaways

• Limit-offset pagination is simple but performs poorly on large datasets. Use it only for small tables or internal APIs.
• Cursor pagination uses a marker to find your place, avoiding expensive row skips. It scales linearly regardless of pagination depth.
• Encode cursors in Base64 to make them opaque and prevent client manipulation.
• Always validate and cap the page size server-side. Never trust client inputs.
• Ensure your database has indexes on the columns you use in WHERE clauses. Without them, cursor pagination offers no advantage.