devops+data: working with source control
TRANSCRIPT
![Page 1: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/1.jpg)
DEVOPS+DATA:WORKING WITH SOURCE CONTROL
Ed Leighton-Dick
![Page 2: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/2.jpg)
ABOUT MEEmail: [email protected]: edleightondick.comTwitter: @eleightondick
![Page 3: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/3.jpg)
GOALS• Understand DevOps• Understand source control• How to apply source control to data
![Page 4: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/4.jpg)
DEVOPS
![Page 5: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/5.jpg)
WHAT IS DEVOPS?• “DevOps is the practice of operations and development
engineers participating together in the entire service lifecycle, from design through the development process to production support.”• Corollary: “DevOps is also characterized by operations staff
making use [of] many of the same techniques as developers for their systems work.”
Source: https://theagileadmin.com/what-is-devops/
![Page 6: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/6.jpg)
WHAT IS DEVOPS?• Incorporates• Structured processes favored by operations• Flexibility favored by developers
• Built around consistent processes and practices• Evolved from need to better support Agile• Often includes Continuous Integration/Continuous Delivery
![Page 7: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/7.jpg)
WHAT’S THE BIG DEAL?• Improved communication and collaboration• Improved productivity and efficiency•More regular, more consistent software updates• Improved software quality• Reduced time to recover from failed releases• Lower risk
![Page 8: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/8.jpg)
STAGES• Code• Build• Verify• Package• Release• Configure•Monitor
Image Credit: By Kharnagy - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=51215412
![Page 9: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/9.jpg)
DEVOPS+DATA• Databases are harder to automate• Shared resource• Code mixed in with data• Mix of static and dynamic data• Objects must be modified, not simply replaced
• Databases are integral components of most applications
![Page 10: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/10.jpg)
SOURCE CONTROL
![Page 11: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/11.jpg)
SOURCE CONTROL• Practices and supporting systems that support the
management of changes to application code• Stores history of changes• Facilitates merging of conflicting changes
• AKA: version control, source management, revision control
![Page 12: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/12.jpg)
TYPES OF SOURCE CONTROL• Centralized• PVCS• Microsoft Visual Source Safe• Microsoft Team Foundation Server• Subversion (SVN)
• Distributed• Git• Mercurial
![Page 13: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/13.jpg)
COMPARISON
Centralized
• One snapshot + deltas•Working copy =
Most recent version•Work solely on
central repository• One source of truth
Distributed
• Git: Snapshots•Working copy =
Full copy of repository•Work primarily with
local repository• ??
![Page 14: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/14.jpg)
TRADITIONAL• Repository•Working folder• Check-out and check-in• Commits• Branching and merging
![Page 15: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/15.jpg)
DISTRIBUTED• Repository• Local repository• Cloning• Pull• Push• Branching and merging
![Page 16: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/16.jpg)
VCS+DATA
![Page 17: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/17.jpg)
VCS+DATA• Databases should be included in source control• Database code = Application code• Static data = Application code• Most applications cannot run without their database(s)
![Page 18: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/18.jpg)
VCS+DATA• Database professionals have been slow to adopt source
control• Many DBAs have no development experience• Lack of communication about why• Lack of communication/inclusion with development teams
• Migration vs deployment• Lack of pre-built tools to facilitate complex database changes
![Page 19: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/19.jpg)
VCS+DATA: WHY NOW?• The situation has changed• Modern tools now exist to facilitate technical side• DevOps changes the conversation
![Page 20: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/20.jpg)
VCS+DATA MODELS• Schema-only• Changes-only• Hybrid
![Page 21: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/21.jpg)
THE SCHEMA-ONLY MODEL• Code for all database objects stored•Mirrors model used by most application code• Can produce a clean copy of a specific version of a database• Cannot migrate to a new version of a database without help• Used by…• Roll-your-own• Most third-party tools* - Microsoft, Redgate, ApexSQL, DBMaestro
![Page 22: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/22.jpg)
THE CHANGES-ONLY MODEL• Scripts to upgrade between versions are stored• Scripts to create a clean copy may not be stored• Used by…• Roll-your-own (very common)
![Page 23: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/23.jpg)
THE HYBRID MODEL• Stores scripts used to create objects• Stores scripts used to migrate between versions• Used by…• Redgate SQL Source Control (with migration scripts)• Redgate ReadyRoll• Others?
![Page 24: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/24.jpg)
FIRST THINGS FIRST…• All changes must be saved to source control for source
control to be useful• Database cannot enforce this – Unaware of source control• Source control cannot enforce this – Unaware of changes• External tools may be able to monitor but cannot yet enforce –
No control over database or source control• Cannot move further in DevOps without good source control
![Page 25: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/25.jpg)
FIRST THINGS FIRST…• Decide on your process before implementing tools• What objects are saved to source control?• What changes are saved?• Who saves changes?• How often?• Who/what will enforce process? How?
![Page 26: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/26.jpg)
SHARED VS DEDICATED• Two models for working with development databases• Shared• Dedicated
![Page 27: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/27.jpg)
SHARED DEVELOPMENT MODEL• Data kept in central location•More efficient from technical perspective• Less disk space• Less management overhead
• Developers must be concerned with conflicts with each others’ work• Integration done at time of development• Refreshing can be difficult
![Page 28: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/28.jpg)
DEDICATED DEVELOPMENT MODEL• Each developer has their own copy of the database• Developer is able to work more efficiently• No conflicts with other developers’ work• Copy of database is stored locally in many cases• Data is more easily refreshed
• Integration occurs between development and testing• Concerns• Transporting and storage of large databases• Security of data• Management overhead
![Page 29: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/29.jpg)
WHICH IS BETTER?• No right answer• Often dictated by circumstances
•My preference: As close to dedicated as possible• Redgate SQL Clone can help facilitate refreshing,
management• Solves some problems• Still requires connection to central image
![Page 30: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/30.jpg)
RELEASING CHANGES• The problem:
How do we generate the deltas that matter to us?• Source control deltas = Differences between stored versions
(Not useful for migrations)• Databases require change scripts of some sort
![Page 31: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/31.jpg)
TOOLS•Microsoft Visual Studio Database Projects• Feature of Visual Studio• Developers works with creation scripts, rather than directly with
database• “Declarative database development” model• Schema Comparison used to generate change scripts
![Page 32: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/32.jpg)
TOOLS• Redgate approaches• SQL Source Control• More “behind the scenes” than database projects• Integrated into SQL Server Management Studio• Uses SQL Compare to generate change scripts at time of release• Manually-created migration scripts override generated change scripts
• ReadyRoll• Integrated into Visual Studio• Uses SQL Compare to generate migration scripts at time of commit• Edit generated migration scripts to override
![Page 33: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/33.jpg)
DEMO
![Page 34: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/34.jpg)
BEST PRACTICES•Make database source control part of your team’s standard
development process• Don’t make it only the DBA’s responsibility
• One object/change per file• Check-in changes on a regular basis• Find a solution that integrates with what your developers already
use• Don’t allow code changes outside of the process• Store the database build number in the database
![Page 35: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/35.jpg)
WRAP-UP
![Page 36: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/36.jpg)
GOALS• Understand DevOps• Understand source control• How to apply source control to data
![Page 37: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/37.jpg)
QUESTIONS?Email: [email protected]: edleightondick.comTwitter: @eleightondick
![Page 38: DevOps+Data: Working with Source Control](https://reader035.vdocuments.mx/reader035/viewer/2022062900/58ea8c911a28ab983e8b5505/html5/thumbnails/38.jpg)
THANK YOU FOR ATTENDING!~ Please remember to fill out your comment cards ~