andrás belokosztolszki 6/10/2007andras.belokosztolszki@red-gate.com managing change in the database...
Post on 26-Mar-2015
219 Views
Preview:
TRANSCRIPT
András BelokosztolszkiAndrás Belokosztolszki6/10/20076/10/2007
Andras.Belokosztolszki@red-gate.comAndras.Belokosztolszki@red-gate.com
Managing change in Managing change in the database worldthe database world
www.sqlbits.com
Group BY: [Food and Drink at Reading Bowl, see you there!]
Feedback Forms: [Voucher for £30 book on return of Form]
Lunch Time Sessions: [Idera in Everest, Quest in Memphis, Grok in Chic 1 and 2]
Learn & Enjoy [Put your phone on Vibrate!]
Ask The Experts[Sessions need to finish on time, take questions to the ATE area]
BackgroundBackground
Working for Red Gate SoftwareWorking for Red Gate Software Software Architect/PMSoftware Architect/PM
– SQL Log RescueSQL Log Rescue (reads the transaction log, and allows fine (reads the transaction log, and allows fine
grained recovery)grained recovery)
– SQL CompareSQL Compare Compares and synchronizes databasesCompares and synchronizes databases
– SQL RefactorSQL Refactor Productivity tool for DBAs and developersProductivity tool for DBAs and developers
AgendaAgenda
MotivationMotivation Where is the schema storedWhere is the schema stored
– Database vs. Creation scriptsDatabase vs. Creation scripts– Advantages/disadvantagesAdvantages/disadvantages
Explore the database/make it Explore the database/make it explorableexplorable
Compare (scripts/db, db/db)Compare (scripts/db, db/db) Synchronize (scripts/db, db/db)Synchronize (scripts/db, db/db)
The Ideal WorldThe Ideal World
Requirements ChangeRequirements Change Natural growth of the databaseNatural growth of the database
– More data and physical limitationsMore data and physical limitations– Expansion: more information is storedExpansion: more information is stored– Access control restrictionsAccess control restrictions– Database merges (two perfect Database merges (two perfect
databases need to be consolidated)databases need to be consolidated) Turnover, new people? New ideas?Turnover, new people? New ideas? Change for the sake of change!Change for the sake of change! Poor documentation -> Re-Poor documentation -> Re-
implementing existing functionalityimplementing existing functionality
Changes Lead to Database Changes Lead to Database EvolutionEvolution
Database schemata change, i.e. Database schemata change, i.e. existing schemata need to be existing schemata need to be extended and changedextended and changed
Generally there is a deployed Generally there is a deployed production database and one or many production database and one or many “freely modifiable” development “freely modifiable” development databasesdatabases
There is a need to identify or track There is a need to identify or track changes!changes!
Database DevelopmentDatabase Development
DB Development Problems:DB Development Problems: It is NOT compiled code with public/private qualifiersIt is NOT compiled code with public/private qualifiers
– Dependencies can be brokenDependencies can be broken Where are these? What is using this object? Public APIWhere are these? What is using this object? Public API Can I modify this?Can I modify this? What is in this object?What is in this object? Who changed this the last time?Who changed this the last time?
– Problems are detected during production?Problems are detected during production? Garbage (unused, possibly unusable code, possibly Garbage (unused, possibly unusable code, possibly
maintained)maintained) Documentation: Where to store? How to retrieve?Documentation: Where to store? How to retrieve? Legacy codeLegacy code No versioningNo versioning No auditNo audit DataData
Database Development Database Development With ProblemsWith Problems
Where Is the Schemata Where Is the Schemata Stored?Stored?
??
Source Control For Source Control For DatabasesDatabases
Source control works with files, the Source control works with files, the database schema is not in filesdatabase schema is not in files
Problem: identifying the difference Problem: identifying the difference between the files and the database between the files and the database schema itselfschema itself
Problems: keeping the files in sync Problems: keeping the files in sync with the databasewith the database
What Is In the Database?What Is In the Database?
Processed objects Processed objects [tables]: [tables]: – syntax not preservedsyntax not preserved
Textual objects Textual objects [views, triggers]:[views, triggers]:– Syntax preservedSyntax preserved– Meta data not perfectly preserved -> Meta data not perfectly preserved ->
runtime problemsruntime problems Generated objects Generated objects [symmetric keys][symmetric keys] CLR:CLR:
– hard to explore, just a DLLhard to explore, just a DLL
Explore: TablesExplore: Tables
Tables are not textual objectsTables are not textual objects Comments and formatting in the creation Comments and formatting in the creation
scripts are lostscripts are lost The information is stored in several system The information is stored in several system
tables and are accessible via these or via tables and are accessible via these or via system viewssystem views
Compare: TablesCompare: Tables
Scripts: Textual comparisonScripts: Textual comparison Documentation: Scripts or extended Documentation: Scripts or extended
propertiesproperties Generated constraint names Generated constraint names
(defaults, foreign keys, check (defaults, foreign keys, check constraints)constraints)
2000 vs. 2005: users vs. schema2000 vs. 2005: users vs. schema
Synchronization: TablesSynchronization: Tables
Tables contain data, so a drop/create Tables contain data, so a drop/create is not an optionis not an option
Certain operations cannot be Certain operations cannot be performed using an alter statement:performed using an alter statement:– Change filegroupChange filegroup– Change identity propertyChange identity property– Certain data type changes (image)Certain data type changes (image)
Sync Problem: Alter ObjectSync Problem: Alter Object
In certain cases you cannot alter an objectIn certain cases you cannot alter an object– Tables need to be rebuilt when identity Tables need to be rebuilt when identity
columns need changecolumns need change– Table returning functions when the returning Table returning functions when the returning
table schema changestable schema changes– Underlying CLR assembly needs to be rebuiltUnderlying CLR assembly needs to be rebuilt
Be careful, because during the rebuild you Be careful, because during the rebuild you may lose your:may lose your:– PermissionsPermissions– Extended propertiesExtended properties
Explore: Stored Procedures, Explore: Stored Procedures, Views, …Views, …
Non-CLR stored procedures, functions, views and triggers Non-CLR stored procedures, functions, views and triggers are textual objects, i.e. they are stored as textare textual objects, i.e. they are stored as text
Comments and formatting in the creation scripts are Comments and formatting in the creation scripts are preservedpreserved
The information is stored in several system tables and are The information is stored in several system tables and are accessible via these or via system views, but meta accessible via these or via system views, but meta information may be inconsistent!information may be inconsistent!
This is a This is a problemproblem
Problem: Stored Procedures Problem: Stored Procedures (1)(1)
They are stored as textThey are stored as text Dependencies may change:Dependencies may change:
– Stored procedures can return bad resultStored procedures can return bad result– Stored procedures can break (and we learn Stored procedures can break (and we learn
this only when we try to execute them)this only when we try to execute them)– This is very different from compiled This is very different from compiled
software!software!
TableA
Stored procedure
B
Schema binding
Schema binding
Problem: Stored Procedures Problem: Stored Procedures (2)(2)
Sysdepends and Sysdepends and sys.sql_dependencies track sys.sql_dependencies track dependencies dependencies (DEMO)(DEMO)
– No, they do No, they do NOTNOT– Alters are not updating sysdepends Alters are not updating sysdepends
(2000)(2000) Stored procedures can reference Stored procedures can reference
nonexistent stored proceduresnonexistent stored procedures
Understanding Stored Understanding Stored Procedures & Textual Procedures & Textual
ObjectsObjects Legacy code “someone” wrote sometimeLegacy code “someone” wrote sometime Task: understand it fast and modify itTask: understand it fast and modify it
– Formatted code easier to readFormatted code easier to read– Documentation can be inlined unlike in tablesDocumentation can be inlined unlike in tables
Modification is by a simple alter Modification is by a simple alter statement?statement?– This is often sufficient, but there are This is often sufficient, but there are
problems with CLR and object dependenciesproblems with CLR and object dependencies
Problems: ViewsProblems: Views
Views are also stored as textual objectsViews are also stored as textual objects The owner (user or schema) if not The owner (user or schema) if not
specified is resolved differently for various specified is resolved differently for various users! users! – Solution: fully qualified names Solution: fully qualified names (DEMO)(DEMO)
Underlying tables may change!Underlying tables may change!select * from dbo.tableA select * from dbo.tableA – Columns can change -> sp_refresh!!Columns can change -> sp_refresh!!– Columns may be droppedColumns may be dropped– Solution: Expand * and schemabindingSolution: Expand * and schemabinding
Renaming ObjectsRenaming Objects
In scripts: Search and replaceIn scripts: Search and replace In database:In database:
– Need to identify dependencies (but Need to identify dependencies (but sysdepends is wrong)sysdepends is wrong)
– Rename in Management Studio uses Rename in Management Studio uses sp_rename sp_rename Sp_rename is evil! It does update only the Sp_rename is evil! It does update only the
sysobjects/sys.objects tables/views.sysobjects/sys.objects tables/views. Do not use it, or use it with care!Do not use it, or use it with care!
– Drop and create the textual objectDrop and create the textual object
Roles, Users, PermissionsRoles, Users, Permissions
The syntax for creating roles is different for The syntax for creating roles is different for 2000 and 2005 (sp_addrole vs. CREATE 2000 and 2005 (sp_addrole vs. CREATE ROLE)ROLE)– Which one should be in the scripts?Which one should be in the scripts?– Sp_adduser in 2005 creates a schemaSp_adduser in 2005 creates a schema
Good policy: grant permissions to roles onlyGood policy: grant permissions to roles only– Do not compare user membersDo not compare user members
Problem: create role transactional, but Problem: create role transactional, but sp_addrole and sp_addrolemember are NOTsp_addrole and sp_addrolemember are NOT
Explore: DefaultsExplore: Defaults
Two types of defaults, and they must be Two types of defaults, and they must be handled differently during synchronizationhandled differently during synchronization– DRI default (in the column)DRI default (in the column)– Bound default – can be reusedBound default – can be reused
2000: syscomments, sysobjects, 2000: syscomments, sysobjects, syscolumnssyscolumns
2005: sys.columns, 2005: sys.columns, sys.default_constraintssys.default_constraints
Processed Objects: DefaultsProcessed Objects: Defaults Default values for tables are parsed and processed by Default values for tables are parsed and processed by
SQL Server:SQL Server:
Note that there are:Note that there are:– Different casesDifferent cases– Different function callsDifferent function calls– Extra parametersExtra parameters– Extra parenthesesExtra parentheses
See my blog: See my blog: http://www.simple-talk.com/community/blogs/andras/default.aspx
Input SQL Server 2000 SQL Server 2005
2 (2) ((2))
1+2+3 (1+2+3) (((1)+(2))+3)
cast(3 as int) (convert(int,3)) (CONVERT([int],(3),0))
Compare and Sync:Compare and Sync:XML Schema CollectionsXML Schema Collections
SQL Server 2005 supports XML with SQL Server 2005 supports XML with schema validationschema validation
XML schema collection is a set of XML schema collection is a set of XML schemataXML schemata
Problem: it is difficult to alter this Problem: it is difficult to alter this object, i.e. you can only add to it!object, i.e. you can only add to it!
If you want to modify it, you must If you want to modify it, you must unbind it from all dependent objects, unbind it from all dependent objects, like table columns, functions, etclike table columns, functions, etc
CLRCLR
One can write .Net code and execute One can write .Net code and execute it from the database it from the database
It is compiled code It is compiled code Well, it can be very fastWell, it can be very fast
Where are the CLR objects?Where are the CLR objects?
The CLR assemblies are stored in the The CLR assemblies are stored in the database as binary files. See database as binary files. See sys.assemblies and sys.assembly_filessys.assemblies and sys.assembly_files
Strongly named assemblies can be Strongly named assemblies can be easier identified based on their easier identified based on their version numberversion number
Dependencies are much better Dependencies are much better tracked for CLR objectstracked for CLR objects
Problem: CLR AssemblyProblem: CLR Assembly
My database has assemblyXYZ, what My database has assemblyXYZ, what is in it?is in it?– Documentation?Documentation?– Get it out of the database and use Get it out of the database and use
Reflector?Reflector? What about the dependencies (Which What about the dependencies (Which
tables does it reference? It is like tables does it reference? It is like dynamic SQL)dynamic SQL)
CLR UDTCLR UDT
One can create custom user defined One can create custom user defined types using .Nettypes using .Net
These types can be used in a table as These types can be used in a table as column typescolumn types
Problem: How to update an assembly?Problem: How to update an assembly?– Alter assembly works sometimes!Alter assembly works sometimes!– What is the alternative? (See SQL What is the alternative? (See SQL
Compare)Compare)AssemblyAssembly
AssemblyAssembly
UDT1UDT1
CLR Procedure
CLR Procedure
Table with DATA
Table with DATA
Synchronize: RoutesSynchronize: Routes
DECLARE @RouteLifetime INTDECLARE @RouteLifetime INTDECLARE @CreateRouteCommand Nvarchar(4000)DECLARE @CreateRouteCommand Nvarchar(4000)SET @RouteLifetime = CASE WHEN (DATEDIFF(s, GETUTCDATE(), CAST('20070110 SET @RouteLifetime = CASE WHEN (DATEDIFF(s, GETUTCDATE(), CAST('20070110
09:23:45.823' AS DATETIME)) < 0)09:23:45.823' AS DATETIME)) < 0)THEN 1 ELSE DATEDIFF(s, GETUTCDATE(), CAST('20070110 09:23:45.823' AS DATETIME)) THEN 1 ELSE DATEDIFF(s, GETUTCDATE(), CAST('20070110 09:23:45.823' AS DATETIME))
ENDENDset @CreateRouteCommand = N'CREATE ROUTE [routeA]set @CreateRouteCommand = N'CREATE ROUTE [routeA]AUTHORIZATION [routeA_User] AUTHORIZATION [routeA_User] WITH ADDRESS=N''TCP://localhost'',WITH ADDRESS=N''TCP://localhost'',SERVICE_NAME=N''serviceA'',SERVICE_NAME=N''serviceA'',BROKER_INSTANCE=N''broker_instance_identifier'',BROKER_INSTANCE=N''broker_instance_identifier'',LIFETIME='+ cast(@RouteLifetime as nvarchar(12))+''LIFETIME='+ cast(@RouteLifetime as nvarchar(12))+''exec sp_executesql @CreateRouteCommandexec sp_executesql @CreateRouteCommand
Routes are used by the Service BrokerRoutes are used by the Service Broker Routes have a lifetime which is:Routes have a lifetime which is:
Specified in seconds that indicate time to Specified in seconds that indicate time to livelive
Stored as an absolute valueStored as an absolute value
Explore: Certificates And Explore: Certificates And KeysKeys
SQL Server 2005 supports Certificates SQL Server 2005 supports Certificates and (A)symmetric keysand (A)symmetric keys
These are mostly generated by their These are mostly generated by their create statementscreate statements
Can be used for encryption and Can be used for encryption and authorizationauthorization
Note that there is nothing about key Note that there is nothing about key rotation!!!rotation!!!
The system tables do not contain The system tables do not contain enough information to recreate theseenough information to recreate these
Sync: PartitionsSync: Partitions
A table that is stored on A table that is stored on several filegroups based several filegroups based on a partitioning columnon a partitioning column
Indexes are also Indexes are also partitionedpartitioned
Some filegroups can be Some filegroups can be read onlyread only
Two objects that control Two objects that control this: partition schemes this: partition schemes and partition functionsand partition functions
Sync: Partitions (2)Sync: Partitions (2)
Partition functions specify the Partition functions specify the intervalsintervals
Partition schemes specify the Partition schemes specify the filegroups for these intervalsfilegroups for these intervals
E.g. E.g. CREATE PARTITION FUNCTION pf1 ( INT ) CREATE PARTITION FUNCTION pf1 ( INT )
AS RANGE LEFT FOR VALUES ( 2000, 2001 )AS RANGE LEFT FOR VALUES ( 2000, 2001 )
CREATE PARTITION SCHEME ps1 AS PARTITION pf1 CREATE PARTITION SCHEME ps1 AS PARTITION pf1
TO ( [PRIMARY], [PRIMARY], [PRIMARY] )TO ( [PRIMARY], [PRIMARY], [PRIMARY] )
Sync Problem: PartitionsSync Problem: Partitions
The alter operation splits and merges The alter operation splits and merges a partition scheme, but this affects a partition scheme, but this affects both the partition function and the both the partition function and the partition schemepartition scheme
A split and a merge is a very resource A split and a merge is a very resource intensive operation because data is intensive operation because data is involved (moving rows from one involved (moving rows from one filegroup to another)filegroup to another)
Space limitationsSpace limitations
QuestionsQuestions
?? Andras.Belokosztolszki@red-gate.comAndras.Belokosztolszki@red-gate.com http://www.red-gate.com http://www.simple-talk.com/community/blogs/andras/defaul
t.aspx http://www.simple-talk.com/author/andr%c3%a1s-belokoszt
olszki/
www.sqlbits.comwww.SQLBits.com[Conference Web site]
www.SQLBlogCasts.com[Becoming the premier Blogging site for SQL professionals]
www.SQLServerFAQ.com[UK SQL Server Community Website]
UK SQL Bloggerscwebbbi.spaces.live.comsqlblogcasts.com/blogs/simonssqlblogcasts.com/blogs/tonyrogerson
Andras.Belokosztolszki@red-gate.comAndras.Belokosztolszki@red-gate.comhttp://www.red-gate.comhttp://www.red-gate.comhttp://www.simple-talk.com/community/blogs/andras/http://www.simple-talk.com/community/blogs/andras/default.aspxdefault.aspxhttp://www.simple-talk.com/author/andr%c3%a1s-http://www.simple-talk.com/author/andr%c3%a1s-belokosztolszki/belokosztolszki/ Feedback Forms!!
top related