Git LFS for Large Files

What is Git LFS?

Git LFS (Large File Storage) is an open-source Git extension that replaces large files in your repository with tiny pointer files. Instead of storing the actual binary content directly in Git, LFS stores a lightweight text pointer that references the real file, which is kept on a separate storage server. When you checkout a branch, Git LFS automatically downloads the actual file content from that remote server.

Here's what a Git LFS pointer file looks like internally:

version https://git-lfs.github.com/spec/v1
oid sha256:4d7a214614ab2935c26f93e1e75c2a9b7e4c6e5a0b4c2e8d9f1a3b5c7d8e9f01
size 1234567

This tiny pointer (just a few hundred bytes) lives in your Git history instead of the multi-megabyte or gigabyte binary file. The actual file content is stored on a separate LFS server — either GitHub's built-in LFS storage, GitLab's LFS service, or a self-hosted solution.

Why Git LFS Matters

🚀 Deploy your AI agent in 10 minutes

Managed Hermes hosting. Zero DevOps. 100M tokens/mo included.

Try it free →

Standard Git was designed for text-based source code. When you commit large binary files — images, videos, compiled assets, datasets, game assets, or design files — several problems emerge:

The Core Problems Without LFS

Repository bloat: Every clone pulls down the complete history of every large file, even deleted ones. A repository with a few hundred MB of game assets can balloon to multiple gigabytes over time.
Slow operations: git clone, git fetch, and git push become painfully slow as Git must transfer and process massive binary diffs.
Merge conflicts on binaries: Binary files cannot be meaningfully merged. Two designers editing the same Photoshop file will always cause conflicts.
Server storage limits: Platforms like GitHub impose repository size limits (often around 100GB). Large teams with binary-heavy projects hit these ceilings quickly.

How LFS Solves These Issues

Faster cloning: Only the current version of a large file is downloaded by default, not every historical version.
Smaller repository size: The Git history contains only lightweight pointers, keeping the main repository lean and fast.
Bandwidth savings: You only download the large files you actually need for your current checkout.
Locking support: LFS includes file locking to prevent conflicts on unmergeable binary files.

Installing Git LFS

Git LFS is a separate binary that integrates with your existing Git installation. Here's how to get it set up on different platforms:

macOS (via Homebrew)

brew install git-lfs

Ubuntu / Debian

sudo apt-get update
sudo apt-get install git-lfs

Windows (via Chocolatey)

choco install git-lfs

Manual Installation

You can also download the installer directly from the Git LFS releases page. Choose the appropriate binary for your operating system and architecture.

One-Time Setup Per User

After installing the package, you must initialize Git LFS globally. This step registers the LFS filters in your global Git configuration:

git lfs install

You should see output similar to:

Git LFS initialized.

This command modifies your ~/.gitconfig file to add the necessary smudge and clean filters that Git LFS relies on. You only need to run this once per machine.

Getting Started: Tracking Large Files

Once Git LFS is installed and initialized, you need to tell it which file types to manage. This is done inside your repository using the git lfs track command.

Basic File Tracking

Navigate to your existing repository (or create a new one) and specify patterns for the files you want LFS to handle:

cd my-project

# Track individual files by exact name
git lfs track "assets/big-texture.png"

# Track all files matching a glob pattern in a directory
git lfs track "assets/*.psd"

# Track by extension across the entire repository
git lfs track "*.zip"

# Track multiple patterns
git lfs track "*.png" "*.jpg" "*.mp4" "*.fbx"

# Track everything in a directory recursively
git lfs track "raw-data/**"

After running these commands, a new file appears in your repository: .gitattributes. Git LFS registers your tracking patterns there. Let's inspect it:

cat .gitattributes

# Output:
# *.png filter=lfs diff=lfs merge=lfs -text
# *.jpg filter=lfs diff=lfs merge=lfs -text
# *.mp4 filter=lfs diff=lfs merge=lfs -text
# *.fbx filter=lfs diff=lfs merge=lfs -text
# raw-data/** filter=lfs diff=lfs merge=lfs -text

The .gitattributes file must be committed to your repository. This ensures everyone on your team uses the same LFS tracking rules:

git add .gitattributes
git commit -m "Configure Git LFS tracking for large binary files"
git push

Adding Large Files After Tracking

Now add and commit your large files as usual. Git LFS intercepts the process automatically:

# Add a large Photoshop file
git add hero-banner.psd

# Commit it
git commit -m "Add hero banner design"

# Push — this uploads the actual binary to the LFS server
git push origin main

When you push, you'll see LFS-specific output showing the upload progress:

Uploading LFS objects: 100% (1/1), 45 MB | 12 MB/s, done.

Cloning and Pulling LFS Repositories

When someone clones your repository, Git LFS handles the large files transparently — but there are nuances worth understanding.

Standard Clone

A normal clone works fine, but it downloads all LFS files for every branch by default:

git clone https://github.com/user/my-lfs-repo.git
cd my-lfs-repo

During clone, you'll see LFS download progress. This pulls down every version of every tracked file referenced in the default branch history.

Faster Clone with Skipping LFS Files

If you want a fast clone and plan to fetch LFS content later, use the GIT_LFS_SKIP_SMUDGE environment variable:

# Clone without downloading any LFS file contents
GIT_LFS_SKIP_SMUDGE=1 git clone https://github.com/user/my-lfs-repo.git

# Or set it persistently for the session
export GIT_LFS_SKIP_SMUDGE=1
git clone https://github.com/user/my-lfs-repo.git

With this flag, your working directory will contain only the pointer files (those tiny text references). The actual binary content is not downloaded. This is extremely useful for CI/CD pipelines or when you're on a slow connection and only need to work with a subset of assets.

Fetching LFS Content on Demand

After a smudge-skipped clone, pull down the actual files you need:

# Pull all LFS objects for the current checkout
git lfs pull

# Pull LFS objects for a specific file
git lfs fetch --include="assets/hero-banner.psd"

# Pull only files changed since a particular commit
git lfs pull --include="*.fbx" origin main

Checking Out a Branch with LFS Files

When you switch to a branch that references different versions of LFS-tracked files, Git automatically fetches the needed content:

git checkout feature/new-assets
# LFS downloads the specific versions referenced on this branch

Migrating an Existing Repository to LFS

If you already have a repository full of large binary files committed with plain Git, you can migrate them to LFS retroactively. This rewrites history, replacing the actual binary blobs with LFS pointers.

Step 1: Identify Large Files

First, find out what's taking up space:

# List the largest objects in your repository history
git rev-list --objects --all | git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize) %(rest)' | awk '/^blob/ {print substr($0, 6)}' | sort --numeric-sort --key=2 --reverse | head -20

Step 2: Use git-lfs-migrate

The git lfs migrate command rewrites your entire history, converting blobs matching your patterns into LFS pointers. This is a destructive operation — coordinate with your team before doing this on a shared repository.

# Migrate all PNG files throughout the entire history
git lfs migrate import --include="*.png" --everything --verbose

# Migrate multiple patterns in one pass
git lfs migrate import --include="*.psd,*.mp4,*.fbx,*.tga" --everything

# Migrate files larger than a certain size
git lfs migrate import --above="50MB" --everything --verbose

The --everything flag tells LFS to process all branches and tags. Without it, only the current branch is migrated.

Step 3: Verify and Force Push

After migration, verify that the pointers are correct:

# Check that tracked files are now LFS pointers
git lfs ls-files

# Verify the repository state
git fsck

Because history has been rewritten, you must force push all branches and tags. Coordinate this carefully with your team:

# Force push all branches
git push --force --all origin

# Force push all tags
git push --force --tags origin

Every team member will need to do a fresh clone or use git pull --rebase with care after this operation.

File Locking for Binary Collaboration

One of LFS's most valuable features is file locking. Since binary files cannot be merged, locking prevents two people from simultaneously editing the same asset.

Enabling Locking

First, ensure your tracking patterns support locking by adding the lockable flag:

git lfs track "*.psd" --lockable

This updates .gitattributes to include the lockable attribute:

*.psd filter=lfs diff=lfs merge=lfs -text lockable

Locking a File

# Lock a file before editing it
git lfs lock assets/main-banner.psd

# Lock with a descriptive message for your team
git lfs lock assets/main-banner.psd --message "Redesigning hero section for Q4 launch"

Listing and Unlocking

# See all current locks on the repository
git lfs locks

# Output shows who locked what and their message:
# assets/main-banner.psd  alice  Redesigning hero section for Q4 launch

# Unlock after you're done
git lfs unlock assets/main-banner.psd

# Force unlock someone else's lock (requires admin permissions)
git lfs unlock assets/main-banner.psd --force

Locking Workflow in Practice

# 1. Designer locks the file
git lfs lock characters/dragon.fbx -m "Rigging adjustments for animation"

# 2. Designer makes changes locally
# ... edit dragon.fbx in Maya or Blender ...

# 3. Stage, commit, and push
git add characters/dragon.fbx
git commit -m "Update dragon rigging weights"
git push

# 4. Release the lock
git lfs unlock characters/dragon.fbx

Inspecting and Managing LFS Objects

Git LFS provides several commands to understand and manage what's stored in LFS.

Listing Tracked Files

# Show all LFS-tracked files in the current checkout
git lfs ls-files

# Show files including their sizes
git lfs ls-files --size

# Show all LFS files across all branches (requires fetching first)
git lfs fetch --all
git lfs ls-files --all

Checking LFS Status

# Show the current state of LFS in the repository
git lfs status

# Show files that differ between working tree and index
git lfs status --porcelain

Pruning Local LFS Cache

LFS maintains a local cache of downloaded file objects. Over time, this cache can grow. Prune objects that are no longer referenced by your current checkout:

# Remove local LFS objects not needed by current HEAD
git lfs prune

# Dry run to see what would be removed without actually deleting
git lfs prune --dry-run

# Aggressively prune, keeping only objects from recent commits
git lfs prune --recent --verbose

Fetching Specific Objects

# Fetch LFS objects for a specific branch without checking it out
git lfs fetch origin feature-branch

# Fetch only certain file types
git lfs fetch --include="*.png,*.jpg"

# Exclude certain patterns
git lfs fetch --exclude="*.mp4"

Configuring Git LFS

Git LFS behavior can be tuned through Git configuration options.

Global Configuration

# Set the LFS remote URL (if self-hosting)
git config --global lfs.url https://lfs-server.company.com

# Configure parallel transfer threads for faster uploads/downloads
git config --global lfs.concurrenttransfers 8

# Set a custom cache directory
git config --global lfs.storage /mnt/fast-ssd/lfs-cache

# Enable or disable progress bars
git config --global lfs.progress true

Per-Repository Configuration

# Override LFS URL for a specific repository
git config lfs.url https://lfs-staging.company.com

# Set batch transfer size (objects per batch)
git config lfs.batch true
git config lfs.batchsize 100

Environment Variables

# Skip downloading LFS files on clone/checkout
export GIT_LFS_SKIP_SMUDGE=1

# Disable progress meters (useful in CI logs)
export GIT_LFS_PROGRESS=false

# Set custom temporary directory for downloads
export GIT_LFS_TMP_DIR=/scratch/lfs-temp

Best Practices for Git LFS

1. Track by Extension, Not by Individual File

Use broad extension-based patterns rather than listing each file individually. This keeps your .gitattributes clean and ensures new files of that type are automatically handled:

# Good: catch-all by extension
git lfs track "*.psd" "*.fbx" "*.tga" "*.wav"

# Less ideal: manually adding each file
git lfs track "assets/hero.psd"
git lfs track "assets/logo.psd"
git lfs track "assets/banner.psd"

2. Commit .gitattributes Early

Make the .gitattributes file one of the first commits in your repository. If you introduce LFS tracking later, team members pulling before the migration will have inconsistent states. For new projects, set up LFS tracking before adding any binary assets.

3. Use Locking for Unmergeable Files

Any binary format that can't be text-merged — Photoshop files, 3D models, video projects, large datasets — should use the lockable flag. Establish a team convention: always lock before editing, unlock after pushing.

git lfs track "*.psd" --lockable
git lfs track "*.fbx" --lockable
git lfs track "*.blend" --lockable

4. Don't Track Files That Belong in Regular Git

Git LFS is for large binary files. Do not track text files, small images (like UI icons under 100KB), or generated files that can be reproduced from source code. A good threshold: if a file is under 1MB and changes frequently, regular Git handles it fine. Reserve LFS for files over 10-50MB or binary formats that benefit from pointer storage.

# Don't do this — these are small and text-based
git lfs track "*.svg"     # SVGs are text XML files
git lfs track "*.json"    # JSON is text and diffable
git lfs track "*.csv"     # CSVs are text

# Do this instead
git lfs track "*.psd"     # Large binary Photoshop files
git lfs track "*.fbx"     # 3D model binaries
git lfs track "*.mp4"     # Video files

5. Manage Storage Quotas Proactively

LFS storage is not infinite. GitHub provides a limited amount of LFS storage per plan (typically 1GB free, with additional purchased in data packs). Regularly audit your LFS usage:

# Check what's taking up LFS space
git lfs ls-files --all --size | sort -k2 -n -r | head -20

# Prune old local objects
git lfs prune --recent

# On GitHub, check Settings > Billing > Git LFS Data for usage stats

6. Use in CI/CD Pipelines Efficiently

In continuous integration, you often don't need every large file. Use selective fetching to speed up builds:

# In your CI script (e.g., GitHub Actions, Jenkins)
git lfs fetch --include="build-assets/**"
git lfs checkout build-assets/

Or skip LFS entirely if the build doesn't need binary assets:

GIT_LFS_SKIP_SMUDGE=1 git clone --depth=1 https://github.com/org/repo.git

7. Regularly Verify LFS Integrity

Periodically check that your LFS pointers match the actual objects stored on the remote server:

# Verify all LFS objects for the current checkout are present locally
git lfs fsck

# Verify and attempt to recover missing objects from remote
git lfs fsck --verify

8. Set Up a Pre-Commit Hook for Large Files

Prevent accidental commits of large files that should be tracked by LFS. Add this script to .git/hooks/pre-commit:

#!/bin/bash
# Prevent committing files over 10MB that aren't LFS-tracked

MAX_SIZE=10485760  # 10MB in bytes
LFS_TRACKED=$(git lfs ls-files --name-only 2>/dev/null)

for file in $(git diff --cached --name-only); do
  # Skip LFS-tracked files
  if echo "$LFS_TRACKED" | grep -q "^$file$"; then
    continue
  fi
  
  size=$(git cat-file -s :"$file" 2>/dev/null)
  if [ "$size" -gt "$MAX_SIZE" ]; then
    echo "ERROR: $file is $(numfmt --to=iec $size) and not tracked by Git LFS."
    echo "Run: git lfs track \"$file\" and retry."
    exit 1
  fi
done

Make it executable:

chmod +x .git/hooks/pre-commit

Troubleshooting Common Issues

Problem: "This repository is over its data quota"

This GitHub-specific error means you've exceeded your LFS storage allowance. Solutions:

# Remove old LFS objects from history (destructive — coordinate with team)
git lfs migrate import --include="*.psd" --everything
git push --force --all

# Or purchase additional LFS data from your hosting provider

Problem: LFS Files Show as Pointer Text

If working directory files appear as the raw pointer text instead of the actual binary, LFS smudging isn't working:

# Verify LFS is installed and initialized
git lfs version
git lfs install

# Force re-smudge all files in working directory
git lfs checkout

# If using a CI environment, ensure GIT_LFS_SKIP_SMUDGE is not set
unset GIT_LFS_SKIP_SMUDGE
git lfs pull

Problem: Push Rejected Due to Missing LFS Objects

# Sometimes local LFS objects get lost. Re-download them:
git lfs fetch --all
git lfs checkout

# Then retry the push
git push

Self-Hosting Git LFS

While GitHub, GitLab, and Bitbucket all include built-in LFS servers, you can also run your own. This is useful for organizations with security requirements or massive storage needs.

Using the Reference LFS Server

Git LFS ships with a reference server implementation. You can run it as a standalone service:

# Clone the LFS server reference implementation
git clone https://github.com/git-lfs/lfs-test-server.git
cd lfs-test-server

# Build and run
go build
./lfs-test-server -port 8080 -storage /data/lfs-storage

Then configure your clients to use it:

git config lfs.url http://lfs-server.internal:8080

Commercial Self-Hosted Options

Several products offer enterprise-grade self-hosted LFS servers with authentication, replication, and management UIs — including Artifactory, MinIO with an LFS gateway, and Azure DevOps Server. Consult each product's documentation for setup instructions tailored to your infrastructure.

Conclusion

Git LFS transforms Git from a source-code-only tool into a capable version control system for projects of any size and type. By replacing large binaries with lightweight pointers stored on a separate backend, it keeps repositories fast, clones efficient, and team collaboration smooth. The key to success with LFS is thoughtful planning: track files by extension, commit your .gitattributes early, use locking for unmergeable assets, and regularly audit your storage usage. Whether you're building a game with gigabytes of art assets, managing a machine learning project with massive datasets, or collaborating on video production with multi-gigabyte source files, Git LFS provides the scalable storage layer that makes version control practical for large binary workflows. Install it, configure your tracking patterns, and let LFS handle the heavy lifting while you focus on building.