Monorepo Performance
Monorepo slow hai toh Git kaa dosh nahi, strategy ka dosh hai.
Monorepo means ALL your projects live in ONE repository — frontend, backend, mobile, shared libraries, infrastructure code, everything together.
The benefits are powerful: atomic commits across projects (change API and frontend in one commit), shared versioning (all packages always in sync), easy refactoring (search and replace across the entire codebase), and a single source of truth.
The challenge: the repository becomes HUGE. Real monorepos at Google, Meta, and Microsoft range from 10GB to over 100GB. Even smaller companies hit 5-10GB easily.
Performance problems multiply: slow clone (downloading gigabytes of data), slow git status (scanning millions of files), slow git log (processing millions of commits), slow CI (checking out the entire repo every build).
Git was designed for smaller repositories. Monorepos push Git to its limits and require special strategies to stay fast.
# The monorepo problem
git clone https://github.com/huge-company/monorepo.git
# Cloning into 'monorepo'...
# Receiving objects: 12% (52384/423892), 1.23 GiB | 2.50 MiB/s
# (45 minutes later...)
# Receiving objects: 100% (423892/423892), 9.87 GiB | 3.10 MiB/s
# Even after cloning, daily operations are slow
time git status
# real 0m8.234s ← 8 SECONDS just for status!
time git log --oneline -1
# real 0m2.145s ← 2 seconds for a single log entry
time git diff
# real 0m5.678s ← 5 seconds for a diff
# The root causes:
# 1. Millions of files in working directory
# 2. Millions of commits in history
# 3. Large binary assets (images, videos, ML models)
# 4. Bloated .git directory with full history
git clone --depth 1 creates a shallow clone — only the latest commit, no history.
It is dramatically faster for large repos: you download only the files at the latest state, not the entire history. A 10GB repo might only download 500MB with a shallow clone.
Use cases: CI/CD builds, deployment scripts, automated testing — anywhere you need the code but not the history.
Limitations: you cannot view history beyond the depth, you usually cannot push from a shallow clone, and git blame and git log do not work for older commits.
--depth N gets the last N commits. You can deepen later with git fetch --unshallow.
# Shallow clone — only the latest commit
git clone --depth 1 https://github.com/huge-company/monorepo.git
# Cloning into 'monorepo'...
# Receiving objects: 100% (1243/1243), 45.23 MiB | 12.50 MiB/s
# Done in 30 seconds instead of 45 minutes!
# Limitations
git log --oneline
# abc1234 (HEAD -> main) latest commit
# (only 1 commit visible — no history!)
git blame src/app.js
# fatal: no such path in the commit history ← doesn't work for deep history
git push origin main
# ERROR: cannot push from a shallow clone
# Deepen if needed
git fetch --unshallow # get full history
git fetch --depth=100 # get last 100 commits
# Shallow clone with single branch (even faster)
git clone --depth 1 --single-branch --branch main https://github.com/huge/repo.git
# Only downloads main branch, not all branches
--depth 1 for CI where you only need to build and test. Use --depth 2 if you need to diff against the previous commit. Never use shallow clone for daily development — you lose git blame, git log, git bisect, and other debugging tools that depend on history.Sparse checkout lets you clone a repository but only check out SPECIFIC directories.
You still get the full commit history, but your working directory only contains the files you care about. This is the perfect balance for monorepo developers.
Perfect for monorepos where you work on one package but need to see full history and make cross-project commits.
Git 2.25+ introduced cone mode (--cone) which is much faster for large monorepos because it uses directory-level patterns instead of complex glob patterns.
The workflow: git sparse-checkout init --cone then git sparse-checkout set packages/frontend
# Step 1: Create a sparse checkout clone
git clone --filter=blob:none --sparse https://github.com/huge/monorepo.git
cd monorepo
# Initially, almost no files are checked out
ls
# (only root-level files like README.md)
# Step 2: Set the directories you want
git sparse-checkout init --cone
git sparse-checkout set packages/frontend packages/shared
# Now only those directories exist
ls packages/
# frontend/ shared/
# (backend/, mobile/, infra/ are NOT checked out!)
# Step 3: Add more directories later
git sparse-checkout add packages/backend
# Step 4: Remove directories you no longer need
git sparse-checkout set packages/frontend
# (removes packages/shared and packages/backend from working dir)
# Step 5: See what is checked out
git sparse-checkout list
# packages/frontend
# Full history is available even for unchecked-out dirs
git log -- packages/backend/api.js # still works!git clone --filter=blob:none downloads the commit and tree objects but NOT the file content (blobs).
Git downloads blobs on demand — when you check out a file or run git diff, it fetches only the needed blobs from the server.
This combines the best of both worlds: full history is available, but you do not download gigabytes of data you do not need.
--filter=blob:limit=1m — only download blobs smaller than 1MB; large files are fetched on demand.
--filter=tree:0 — do not even download tree objects; fetch everything on demand (ultra-thin clone).
Requires server support — GitHub, GitLab, and Azure DevOps all support it.
# Blobless clone — full history, no file content initially
git clone --filter=blob:none https://github.com/huge/monorepo.git
# Cloning into 'monorepo'...
# Receiving objects: 100% (423892/423892), 234.56 MiB
# (Only commits and trees — much smaller than 9.87 GiB!)
# File contents are fetched on demand
cat src/app.js
# (Git automatically fetches the blob for this file)
git diff HEAD~1
# (Git fetches the needed blobs to compute the diff)
# Full history works!
git log --oneline | head -20 # all commits visible
git blame src/app.js # works perfectly
# Treeless clone — even thinner
git clone --filter=tree:0 https://github.com/huge/monorepo.git
# Doesn't even download tree objects
# Fetches everything on demand — smallest initial download
# Size-limited clone
git clone --filter=blob:limit=1m https://github.com/huge/monorepo.git
# Downloads all blobs < 1MB
# Large files (images, videos) are fetched on demand
# Combine with sparse checkout for maximum efficiency
git clone --filter=blob:none --sparse https://github.com/huge/monorepo.git
# Blobless + sparse = minimal download
CI pipelines for monorepos need special optimization to avoid checking out the entire repo every time.
Strategy 1: Shallow clone — use fetch-depth: 1 in GitHub Actions. Fastest checkout, but no history.
Strategy 2: Path-based triggers — only run CI for directories that changed. Saves massive CI time in monorepos.
Strategy 3: Caching — persist the .git directory between CI runs. Incremental fetches instead of full clones.
Strategy 4: Sparse checkout in CI — only check out the directories being tested. Smaller working directory, faster operations.
Strategy 5: Monorepo tools (Nx, Turborepo, Lerna) — intelligent task scheduling based on dependency graphs. Only build and test affected packages.
# GitHub Actions optimized for monorepo
name: CI
on: [push, pull_request]
jobs:
# Strategy 1: Shallow clone (fastest)
fast-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 1 # shallow clone
# Strategy 2: Path-based triggers
frontend-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 2 # need 2 for diff
- name: Check if frontend changed
run: |
if git diff --name-only HEAD~1 HEAD | grep -q "^packages/frontend/"; then
echo "Frontend changed, running tests..."
cd packages/frontend && npm test
else
echo "Frontend not changed, skipping."
fi
# Strategy 3: Sparse checkout in CI
backend-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
sparse-checkout: |
packages/backend
packages/shared
sparse-checkout-cone-mode: true
# Strategy 4: Cache .git between runs
cached-checkout:
runs-on: ubuntu-latest
steps:
- uses: actions/cache@v3
with:
path: .git
key: git-cache
- run: git fetch # incremental update instead of full clone
fetch-depth: 0 only when you need full history (e.g., for git blame or semantic-release). For everything else, fetch-depth: 1 is 10x faster. If you need to detect which files changed, use fetch-depth: 2 and run git diff HEAD~1 --name-only.Lo kar liya — Key Points:
- ✅ Monorepos store all projects in one repository but face performance issues with size: slow clone, slow status, slow CI
- ✅ Shallow clone (--depth 1) downloads only the latest commit — fastest but no history, cannot push, cannot blame
- ✅ Sparse checkout lets you check out only specific directories while retaining full commit history
- ✅ Partial clone (--filter=blob:none) downloads commit and tree objects but fetches file content on demand
- ✅ Combine sparse checkout with partial clone for maximum efficiency: minimal download, full history on demand
- ✅ CI/CD optimization includes shallow clones, path-based triggers, caching, and sparse checkout
- ✅ Choose clone strategy based on needs: shallow for CI, sparse for monorepo dev, blobless for large repo dev
Want to track your progress?
Log in to save your place and pick up where you left off.
Progress track karna chahte ho?
Login karo apni progress save karne ke liye aur jahan chhoda tha wahan se shuru karo.
Login