Fixing Slow Git Clone on Linux

Git clone can be painfully slow on large repositories. When I tried cloning a 10GB repository on my Lisbon server, it took over an hour. This guide covers multiple optimization techniques to dramatically speed up git operations.

Environment

• OS: Ubuntu 22.04 LTS
• Git version: 2.34.1
• Repository size: ~10GB with 500K+ commits
• Network: 100Mbps connection

$ git --version
git version 2.34.1

# Measure clone time
$ time git clone https://github.com/large-org/big-repo.git
Cloning into 'big-repo'...
...
real    65m23.456s
user    12m34.567s
sys     8m45.678s

The Problem

# Clone takes over an hour
$ time git clone https://github.com/large-org/big-repo.git
# 65 minutes for 10GB repo

# Network speed test shows fine
$ speedtest-cli --simple
Testing download... 95.45 Mbit/s
Testing upload... 45.67 Mbit/s

Analysis

Step 1: Check Git Configuration

# Check current settings
$ git config --global --get-all --list | grep -E "(compression|packthreads|http)"
http.postBuffer=524288
core.compression=0

# Check SSH vs HTTPS
$ git remote -v
origin  https://github.com/large-org/big-repo.git (fetch)
# Using HTTPS - may be slower than SSH for large repos

Step 2: Check Repository Characteristics

# Count commits
$ git rev-list --all --count
523456

# Count objects
$ git count-objects -v
count: 1234567
size: 10234567890
in-pack: 8765432
packs: 1

Solution

Fix 1: Use Shallow Clone

# Clone only recent history (fastest option)
$ time git clone --depth 1 https://github.com/large-org/big-repo.git
# 5 minutes instead of 65!

# Clone with specific depth
$ git clone --depth 100 https://github.com/large-org/big-repo.git

# Fetch more history later if needed
$ git fetch --unshallow

Fix 2: Optimize Git Configuration

# Optimize network buffer
$ git config --global http.postBuffer 524288000

# Enable compression
$ git config --global core.compression 6

# Use multiple threads for packing
$ git config --global pack.threads 0

# Optimize delta cache
$ git config --global core.deltaBaseCacheLimit 256m

# Increase window size for pack
$ git config --global pack.windowMemory 256m
$ git config --global pack.packSizeLimit 256m

Fix 3: Use SSH Instead of HTTPS

# Change remote URL to SSH
$ git remote set-url origin git@github.com:large-org/big-repo.git

# Verify
$ git remote -v
origin  git@github.com:large-org/big-repo.git (fetch)
origin  git@github.com:large-org/big-repo.git (push)

# Test connection
$ ssh -T git@github.com
Hi large-org! You've successfully authenticated, but GitHub does not provide shell access.

Fix 4: Use a Git Mirror

# Clone from a mirror (faster CDN)
$ git clone https://github.com/large-org/big-repo.git --mirror

# Or use a local mirror server
$ git clone git://mirror.internal/big-repo.git

Fix 5: Parallel Downloads

# Use git-lfs for large files
$ git lfs install
$ git clone https://github.com/large-org/big-repo-with-lfs.git

# Or use partial clone
$ git clone --filter=blob:none https://github.com/large-org/big-repo.git

Fix 6: Optimize SSH Connection

# Add to ~/.ssh/config
Host github.com
    User git
    Hostname github.com
    IdentityFile ~/.ssh/id_ed25519
    ServerAliveInterval 60
    ServerAliveCountMax 3
    IPQoS=throughput
    Compression yes

Fix 7: Network Optimization

# Increase TCP buffer sizes
$ sudo sysctl -w net.core.rmem_max=16777216
$ sudo sysctl -w net.core.wmem_max=16777216
$ sudo sysctl -w net.ipv4.tcp_rmem="4096 87380 16777216"
$ sudo sysctl -w net.ipv4.tcp_wmem="4096 65536 16777216"

# Make persistent
$ cat >> /etc/sysctl.conf << 'EOF'
net.core.rmem_max=16777216
net.core.wmem_max=16777216
net.ipv4.tcp_rmem=4096 87380 16777216
net.ipv4.tcp_wmem=4096 65536 16777216
EOF
$ sudo sysctl -p

Comparison

Quick Commands

# Shallow clone
$ git clone --depth 1 URL

# Clone specific branch
$ git clone --depth 1 --branch main URL

# Resume interrupted clone
$ git clone --depth 1 URL
$ cd repo
$ git fetch --unshallow

# Show clone progress
$ GIT_CURL_VERBOSE=1 git clone URL

Lessons Learned

1. Shallow clone is the fastest option when you don't need full history.
2. SSH is faster than HTTPS for large repository operations.
3. Increase http.postBuffer for repositories with large files.
4. Use --filter=blob:none to skip downloading large binary files initially.
5. Network tuning can help but is secondary to shallow clone optimization.

This blog does not accept any external sponsorships, affiliate marketing, or ad revenue.