By Matt Fedorovich / 7 Feb 2017 / Topics: Application development
As my scenes and assets grew, so did the file sizes: 3D assets, movies files, large 360-degree photos. All of these files are multiple megabytes, or gigabytes. As I would push them to our GitLab, I would come across broken pipes, timeout issues and failure errors due to file size.
After doing a bit of research, I came across documentation that GitHub and GitLab wouldn't accept general files over 100MB. Most of our files reach well over that.
In April of 2015, GitHub introduced an open-source tool called Git Large File Storage (LFS). It was developed specifically for large files such as audio, video, data sets and graphics that exceed the maximum file size.
Instead of forcing the file up onto the server or simply bypassing the max setting, Git LFS actually pushes the large file to another server (large file storage) and creates a text file that points to it on that server. The text file is then pushed to your Git as a placeholder.
The process of getting these files to the large file storage is fairly simple to set up. I'm going to use Sourcetree to show you how to do it because it has Git LFS built right in.
For your Git repository to push to the LFS, it needs to know which files are going to be more than 100MB or ones that are already more than 100MB. Luckily, you don’t need to provide the exact file name but just the file extension it's associated with (i.e., a *.PSD file).
First, let’s open Sourcetree and pick a repo that has large files in it. Go to the top menu bar and select Repository > Git LFS > Initialize Repository.
Then this popup shows up. Click Start using Git LFS.
Next, add the file extensions you think, or know, are going to be large files. When it comes to Unity, there are four files that are usually pretty big: *.a, *.asset, *.resS, and *.assetbundle. The first three are generated based on your scenes, scene links and the assets associated with those scenes. AssetBundles are like ZIP files with assets compressed inside them, making them huge. Once in game, Unity is able to decompress and use those assets on the fly, making them a great resource for assets that need to be downloaded (i.e., large skyboxes, high-poly models).
Once you hit OK, Sourcetree will create, or add to, a file called .gitattributes. Be sure to commit and push that file up to GitHub before pushing the rest of your files.
And that's it. Now you can commit your large files, and GitHub will automatically track which ones get sent over to LFS.
Quick note: Git LFS will also warn you when it notices a file that may be too large for the Git. It gives you a list of files, and you can add their extensions by going to Repository > Git LFS > Track/Untrack Files.
National Lead for Immersive Technology, Insight
With more than 10 years of experience in gaming, mobile and spatial computing, Matt focuses on bringing the latest virtual, augmented and mixed reality solutions to enterprise clients. He believes eliminating the barrier to entry for the next dimension of computing is key to realizing impactful business value and promoting a deep integration of natural spatial interfaces.
