Git internals: packfiles and garbage collection explained

Opening: Why Git Internals Matter

You clone a repository with millions of commits, and Git responds in seconds. You push a feature branch, and the server computes differences instantly. This performance doesn't happen by accident—it's the result of how Git organizes and compresses objects on disk. Understanding Git internals: packfiles and garbage collection explained will help you optimize repositories, troubleshoot storage issues, and make informed decisions about repository maintenance in production environments.

Most developers interact with Git through high-level commands: git add, git commit, git push. But beneath those commands lies a storage system that evolves as your repository grows. When you first commit changes, Git creates individual object files. Over time, without intervention, these loose objects accumulate and slow down operations. Git's solution involves two key concepts: packfiles for compression and git gc for cleanup. Let's explore how they work together to keep your repository fast.

Loose Objects — The Foundation of Git Storage

Every time you create a commit, blob, tree, or tag in Git, it's stored as a separate object. These are called loose objects, and they live in the .git/objects directory, organized by their SHA-1 hash.

When you run git commit on a new repository, Git creates:

• A blob object for each file's content
• A tree object representing the directory structure
• A commit object linking the tree, parent commit, author info, and message

Each object is stored individually, compressed with zlib, and named after its hash. For example:

$ find .git/objects -type f
.git/objects/a1/b2c3d4e5f6... (blob)
.git/objects/f7/e8d9c0a1b2... (tree)
.git/objects/2c/3d4e5f6a7b... (commit)

This design is simple and works well for small repositories. Each object is self-contained and can be independently verified by its hash. However, a real-world scenario reveals the limitation: imagine a large project with 100,000 commits. If each commit modifies a few files, you'll have hundreds of thousands of loose objects scattered across your .git/objects directory. Disk I/O becomes the bottleneck—reading object metadata requires many filesystem calls.

Additionally, loose objects offer no deduplication for similar content. If a large file is modified slightly across 50 commits, Git stores 50 nearly-identical versions independently. Over time, your repository's disk footprint grows linearly with history size.

Packfiles — Compression Through Deltas

Packfiles solve the loose object problem by bundling multiple objects together and storing deltas—differences between related objects—instead of full copies.

A packfile (typically named with a .pack extension) contains a binary format that:

• Groups many objects into a single file
• Stores deltas between similar objects to save space
• Includes an index file (.idx) for fast object lookup

In a real production scenario, consider a repository with a 50 MB JSON configuration file that's modified in 100 commits. With loose objects, you'd store 100 copies (~5 GB total). With packfiles, Git stores one full copy and 99 deltas, reducing size to perhaps 100 MB. That's a 50x reduction for a common workflow.

You can inspect your packfiles directly:

$ ls -lh .git/objects/pack/
-rw-r--r-- 1 user staff 12M pack-abc123def456.pack
-rw-r--r-- 1 user staff 2.3M pack-abc123def456.idx

$ git verify-pack -v .git/objects/pack/pack-abc123def456.idx | head -20
abc123def456 commit 245 bytes 150 offset
def456abc123 tree 1024 bytes 395 offset 2 levels
... (delta encodings shown for related objects)

The .idx file is critical for performance. It's a binary index that allows Git to locate any object in the packfile without scanning the entire file. When you request a commit by hash, Git uses the index to jump directly to its location in the pack.

Git automatically creates packfiles when you interact with remote servers. When you git fetch or git push, the remote server (or your local Git in some cases) creates a temporary packfile to transfer objects efficiently over the network. These transient packs are usually cleaned up, but long-lived packs accumulate as your repository grows.

You can manually create a packfile for all loose objects:

$ git repack -a -d
Counting objects: 145000, done.
Compressing objects: 100% (98750/98750), done.
Writing objects: 100% (145000/145000), done.
Building bitmaps: 100% (145000/145000), done.
Total 145000 (delta 85200), reused 145000 (delta 85200), pack 145000

The -a flag packs all objects; the -d flag deletes loose objects after packing. The result: a single packfile containing all history, with deltas computed between related objects.

git gc — Automatic Garbage Collection and Optimization

git gc (garbage collection) runs repack automatically and removes unreachable objects. It's your repository's maintenance routine.

When you delete a branch or reset a commit, the associated objects become unreachable—they're no longer pointed to by any ref. These orphaned objects accumulate over time, wasting disk space. git gc identifies and removes them:

$ git gc
Counting objects: 150000, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (105000/105000), done.
Writing objects: 100% (150000/150000), done.
Building bitmaps: 100% (150000/150000), done.
Pruning objects: 100%
Total 150000 (delta 87500), reused 145000 (delta 85200)

In a production scenario, consider a CI/CD pipeline that creates hundreds of temporary branches daily. Each failed test run creates refs that are later deleted. Without garbage collection, your repository's .git directory would grow indefinitely. Running git gc periodically removes those orphaned objects.

You can control garbage collection behavior with configuration:

$ git config gc.aggressiveWindow 250
$ git config gc.aggressiveDepth 250
$ git config gc.pruneExpire "now"

These settings control how aggressively Git searches for delta opportunities and when to consider objects eligible for deletion. aggressiveWindow increases the search window for related objects (higher = slower but better compression). pruneExpire sets the grace period for unreachable objects; "now" means delete immediately instead of waiting 2 weeks (the default).

Running git gc manually before major operations (like repository migrations) is a best practice. On shared hosting or managed Git services, aggressive garbage collection may run in the background without your input.

Maintenance — Scheduled Background Optimization with git maintenance

Modern Git (version 2.32+) introduces git maintenance, a more sophisticated alternative to one-off git gc calls. Instead of a single heavy operation, it schedules incremental background tasks.

Enable automatic maintenance:

$ git maintenance start
Created /home/user/.git/config.d/maintenance
Auto-running maintenance:

This configures your system to run maintenance tasks in the background via launchd (macOS), systemd (Linux), or Task Scheduler (Windows). The maintenance tasks include:

• gc — Full garbage collection (runs weekly)
• prefetch — Fetches commits from remotes in the background
• commit-graph — Builds a commit graph for faster log traversal
• loose-objects — Packs loose objects incrementally (runs daily)
• incremental-repack — Repacks packfiles efficiently (runs daily)

In a team environment, this is transformative. Developers no longer experience unexpected lag when Git suddenly decides to garbage-collect. Operations run incrementally, keeping the repository responsive. The commit graph, in particular, accelerates git log, git merge-base, and related operations—critical when working with monorepos or projects with deep histories.

Check maintenance status:

$ git maintenance run --auto
$ git config --show-origin maintenance.auto
file:/home/user/.git/config.d/maintenance	maintenance.auto = true

For server-side repositories (bare repos on a CI/CD system or git server), you can configure maintenance in the global config:

$ git config --global maintenance.auto true
$ git config --global maintenance.strategy incremental

The incremental strategy prioritizes background optimization over one-time aggressive packing—ideal for production systems where responsiveness matters.

Common Mistakes

Running git gc too frequently: A single aggressive git gc computes deltas across all objects, consuming CPU and I/O. Running it hourly on a busy repository is counterproductive. Use git maintenance instead, which increments the work gradually.

Ignoring packfile corruption: If a .pack or .idx file becomes corrupted (rare but possible after a crash), git operations fail silently. Run git fsck periodically to detect issues early. On critical servers, incorporate this into monitoring.

Forgetting to repack after large deletions: If you rewrite history to remove a large file (e.g., git filter-branch or git filter-repo), the old object still exists until you gc. Users pulling the new history will still download the old object from the server. Always push a new packfile after rewriting history.

Misconfiguring pruneExpire on shared repositories: Setting gc.pruneExpire = "now" on a shared server can delete objects that other users are still working with (if they have unpushed commits or open refs). Use the default 2-week grace period on production systems.

Key Takeaways

• Loose objects are Git's base storage unit, created when you commit. They're simple but don't scale well as repositories grow.
• Packfiles bundle objects and use delta encoding to compress similar content, reducing disk size dramatically—often by 50% or more.
• git gc repacks loose objects into packfiles and prunes unreachable objects, keeping your repository lean and fast.
• git maintenance start schedules incremental background optimization, replacing the need for manual gc calls and improving developer experience on large codebases.
• Understanding these internals helps you diagnose slow repositories, optimize storage costs, and make informed decisions about repository maintenance in production.