git lfs at autodesk - meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · git lfs tips...

49
Git LFS at Autodesk LARS SCHNEIDER @kit3bus

Upload: others

Post on 27-Sep-2020

59 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

Git LFS at Autodesk

LARS SCHNEIDER

@kit3bus

Page 2: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

Lars Schneider Autodesk Inc.

Git and Git LFS contributor

Technical Lead forGit at Autodesk

Page 3: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

• Best known for AutoCAD2D and 3D computer-aided design

• 33 years in business

• 4000 engineers, hundreds of products, terabytes of code and asset data

Lars Schneider Autodesk Inc.

Page 4: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

Architecture, Engineering & Construction

Image by Dave Tyner, Autodesk Plant 3D - P&ID

Page 5: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

Manufacturing

Page 6: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

Media & Entertainment

Page 7: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

3D Printing

"Future of Making Things"

Image courtesy of Local Motors Inc.

Page 8: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

Git Git LFS

+

Page 9: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

Why are large files a problem?

All history is local. Good for source files.

Problem for large files.

Page 10: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

Elephant-Video.mp4

Why are large files a problem?

Page 11: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

What files are not suited for Git?

Files that do not compress well and

change frequently....

Page 12: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

Mon Tue Wed

What files are not suited for Git?

Files that do not compress well and

change frequently.

Page 13: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

Example: Git repowith large files

codecode

Git Repo

mastercodecode

301MB

codecode

Page 14: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

Git LFS to the rescue!

git lfs track "*.mp4"

Page 15: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus 1 MB

Example: Git LFS repo with large files

master

LFS Server

LFS Ptr

code

Git LFS Repo

LFS Ptr

codecode

codecode LFS Ptr

code

Page 16: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus 1 MB

Example: Git LFS repo with large files

master

LFS Server

LFS Ptr

code

LFS Ptr

codecode

codecode LFS Ptr

code

Git LFS Repo

Clo

ne

Page 17: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus 1 MB

Example: Git LFS repo with large files

master

LFS Server

LFS Ptr

code

LFS Ptr

codecode

codecode LFS Ptr

code

Git LFS Repo

Clo

ne +

Che

ckou

t

Page 18: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

Technical Details

https://autode.sk/git-lfs-talk

Page 19: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

What do we use Git LFS for?

Integration Test Data(3D Models, ...)

Auxiliary Data(Documentation, Images, Videos, ...)

Build Artifacts(not recommended)

Page 20: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

D E V E L O P E R

M I G R ATO R

A D M I N I S T R ATO R

- Git LFS - What have we learned?

Page 21: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

Migrator

Page 22: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

Migration Process

1 Identify an engineer with deep code knowledge

Create a "demo" migration on Git migration server

Iterate on "demo" migration until repo and CI are OK

Ask broader team to "play" with the "demo" migration

Perform migration on Git production server

2

3

4

5

Page 23: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

How to migrate?

+

git-svn / git-p4 / git-tfs ...

git filter branch / git-lfs-migrate

Page 24: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

How to migrate?

+

git-p4

git-svn / git-p4 / git-tfs ...

git filter branch / git-lfs-migrate

Page 25: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

Git LFSmigration gotchas

Discard large file history

1998 2007 2016

code code code +++

Page 26: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

Git LFSmigration gotchas

master LFS Ptr

code

LFS ServerGit LFS Repo

Avoid "orphaned" LFS files

Page 27: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

Git LFSmigration gotchas

rebasedmaster code

LFS ServerGit LFS Repo

?

Avoid "orphaned" LFS files

Page 28: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

( INCLUDES DESIGNER, TESTER, . . . )

Developer

Page 29: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

Mon Tue Wed

Teach why "Large" files are a problem!

Files that do not compress well and

change frequently.

Page 30: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

Files smaller than 500kb are OK.

Rule of Thumb

Teach why "Large" files are a problem!

Page 31: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

git lfs track "*.png"How to track Git LFS files?

Page 32: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

git lfs track "*.lfs.*"

e.g. /images/elephant.lfs.png

How to track Git LFS files?

Page 33: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

git lfs track "/big/*"

e.g. /big/elephant.png

How to track Git LFS files?

Page 34: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

git lfs track "/xxl.png"How to track Git LFS files?

Page 35: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

Less than 1000 files in LFS are OK.

Rule of Thumb

How to track Git LFS files?

Page 36: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

Less than 1000 files in LFS are OK.

Rule of Thumb

How to track Git LFS files?

https://autode.sk/git-filter

"Git checkout for a test repo with 12k LFS files on Windows takes 57 seconds instead of 55 minutes!"

Git 2.11 + GitLFS 1.5

Page 37: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

git lfs track "*.png"

git lfs track "*.[pP][nN][gG]"

Case sensitive:

Case in-sensitive:

Git LFS Tips & Tricks

Page 38: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

No line ending conversions on

LFS files!Git LFS Tips & Tricks

FirstLineSecondLine

FirstLine▯SecondLineWindowsmacOS

Page 39: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

git lfs clone <URL>Git LFS Tips & Tricks

Page 40: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

Setup your Git credential helper

(or use SSH)!

Git LFS Tips & Tricks

Page 41: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

Use the latestGit / Git LFS

version!

Git LFS Tips & Tricks

Page 42: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

Administrator

Page 43: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

How to make sure Git LFS is used properly?

Setup Git LFS on all dev machines!

Enterprise Config for Githttps://autode.sk/git

LFS Ptr

Page 44: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

How to make sure Git LFS is used properly?

"What happens in Git, stays in Git."

Page 45: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

How to make sure Git LFS is used properly?

Rewriting history can cause a lot of

trouble!

Page 46: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

How to make sure Git LFS is used properly?

Configure file size limit on Git server!

Page 47: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

How to make sure Git LFS is used properly?

Configure file size limit with localGit pre-commit

hooks!

Page 48: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

How to make sure Git LFS is used properly?

Initially, perform Git LFS usage reviews.

Page 49: Git LFS at Autodesk - Meetupfiles.meetup.com/20767392/larsxschneider-gitlfs-at... · Git LFS Tips & Tricks. @kit3bus No line ending conversions on LFS files! Git LFS

@kit3bus

Takeaways

• Git LFS works at scale

• Use the latest Git/Git LFS versions

• Automate Git LFS setuphttps://autode.sk/git

• Use fast clone commandgit lfs clone <URL>

• Use smart tracking patternsgit lfs track "*.lfs.*"

• Reject large files in Git