git remove binary files

See current file size. First need to git gc

Alistair@MONET ~/Documents/Research/Projects/Chromatin/Code (master)
$ git gc
Counting objects: 713, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (570/570), done.
Writing objects: 100% (713/713), done.
Total 713 (delta 257), reused 459 (delta 142)

Now we can count objects. size-pack is the size of our database in Kb (70+ Mb here!)

Alistair@MONET ~/Documents/Research/Projects/Chromatin/Code (master)
$ git count-objects -v
count: 7
size: 22
in-pack: 713
packs: 1
size-pack: 70827
prune-packable: 0
garbage: 0

Here we get rid of the offending file. For more details on all of this, including a good guide on how to find the offending big file(s), see this part of the Git Pro Book.

Alistair@MONET ~/Documents/Research/Projects/Chromatin/Code (master)
$ git filter-branch --force --index-filter \ 'git rm --cached --ignore-unmatch
mRNAdata.dax' HEAD
Rewrite 4c0556f7efab2e9e23fa09dfd247edabfc6ef301 (1/46)rm 'mRNAdata.dax'
Rewrite 7ec96fb9c5ba3d23d4a55022f2a094635835901e (2/46)rm 'mRNAdata.dax'
Rewrite bba4dfa4d06cf51b06a8ba937a84ef4700f44094 (3/46)rm 'mRNAdata.dax'
Rewrite d6a21db5bd63f1c1a23f5633278e84db8763348e (4/46)rm 'mRNAdata.dax'
Rewrite c25106ae199f3a84024a9609434ca9b6818cf632 (5/46)rm 'mRNAdata.dax'
Rewrite 3214522b942f96b579d15bf1b95c2e8daeed0d36 (6/46)rm 'mRNAdata.dax'
Rewrite 94f2d457c83b45ef14b01b479c59e8a94c262d2f (7/46)rm 'mRNAdata.dax'
Rewrite 43d60c2c03fa9c1125148e0b8ca22b8c2d7d622d (8/46)rm 'mRNAdata.dax'
Rewrite a0c181e158a125549b2ac80a48058bb7f075a9ec (9/46)rm 'mRNAdata.dax'
Rewrite d63f8f7ff072f836685eaadbb71128be79ea4186 (10/46)rm 'mRNAdata.dax'
Rewrite 9f64ccff4b1de60bea49b3e74020305768568df2 (11/46)rm 'mRNAdata.dax'
Rewrite 83c96e3fbe80cc3ca17b2b5c847488576081aeb6 (46/46)
Ref 'refs/heads/master' was rewritten

Now some cleanup. We remove the refs and logs

Alistair@MONET ~/Documents/Research/Projects/Chromatin/Code (master)
$ rm -Rf .git/refs/original

Alistair@MONET ~/Documents/Research/Projects/Chromatin/Code (master)
$ rm -Rf .git/logs/

And we want to remove the rewrite backup so that everything is back to normal. Otherwise if we need to do this again we will get errors objecting to our editing the rewrite folder. I haven’t found as good references to support this move so maybe do with caution.

Alistair@MONET ~/Documents/Research/Projects/Chromatin/Code (master)
$ rm -rf .git-rewrite/

Now let’s see how we’re doing on space

Alistair@MONET ~/Documents/Research/Projects/Chromatin/Code (master)
$ git gc
Counting objects: 712, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (463/463), done.
Writing objects: 100% (712/712), done.
Total 712 (delta 257), reused 655 (delta 248)

Alistair@MONET ~/Documents/Research/Projects/Chromatin/Code (master)
$ git count-objects -v
count: 65
size: 68423
in-pack: 712
packs: 1
size-pack: 3040
prune-packable: 0
garbage: 0

Much better!

This entry was posted in Software Development and tagged , , . Bookmark the permalink.