SQL Server Stored Procedures
Architecture & Performance
SQL Server Stored Procedures
Architecture & Performance
Victor IsakovVictor IsakovMCT, CTT, MSCE, MCDBAMCT, CTT, MSCE, [email protected]@informare.com.au
© Victor Isakov [email protected]
Victor Isakov ([email protected])Victor Isakov ([email protected])
Victor Isakov has been involved with SQL Server Victor Isakov has been involved with SQL Server since Version 4.21a, enjoying a reputation as one of since Version 4.21a, enjoying a reputation as one of the best "SQL Gurus" in the countrythe best "SQL Gurus" in the country
He has taught the Administration, Development, He has taught the Administration, Development, Data Warehousing and Performance Tuning courses Data Warehousing and Performance Tuning courses since 1995 in the Australasia-Pacific region and since 1995 in the Australasia-Pacific region and provided consultancy services to companies like provided consultancy services to companies like the University of NSW, Department of Education and the University of NSW, Department of Education and Training, Westpac and the United Nations Training, Westpac and the United Nations
He specialises as a consultant in Optimising, He specialises as a consultant in Optimising, Troubleshooting and Architecting Database Troubleshooting and Architecting Database Solutions based on Microsoft technologySolutions based on Microsoft technology
© Victor Isakov [email protected]
AgendaAgenda
Stored Procedure ArchitectureStored Procedure Architecture
Stored Procedure OptimizationStored Procedure Optimization
Managing Query Plan PerformanceManaging Query Plan Performance
Undocumented Microsoft Stored ProceduresUndocumented Microsoft Stored Procedures
© Victor Isakov [email protected]
AgendaAgenda
Stored Procedure ArchitectureStored Procedure Architecture
Stored Procedure OptimizationStored Procedure Optimization
Managing the Query Plan for Managing the Query Plan for Performance.Performance.
Undocumented Microsoft Stored Undocumented Microsoft Stored ProceduresProcedures
© Victor Isakov [email protected]
Stored Procedure ArchitectureStored Procedure Architecture
SPs are SQL Server programsSPs are SQL Server programsTraditionally written in Transact-SQLTraditionally written in Transact-SQLIn Yukon will include VB.NET, C#In Yukon will include VB.NET, C#
ProvideProvideQuery ManagementQuery ManagementData ModificationsData ModificationsData / Referential IntegrityData / Referential IntegrityMaintenance TasksMaintenance TasksProtect DataProtect Data
Provide Gateway to Database ObjectsProvide Gateway to Database Objects
© Victor Isakov [email protected]
Stored Procedure ArchitectureStored Procedure Architecture
Stored in Database Stored in Database Protected like other DB objectsProtected like other DB objects
Logic moved to serverLogic moved to server
Increased Developer Productivity Increased Developer Productivity Reusable CodeReusable Code
More structured 3-Tier solutionsMore structured 3-Tier solutions
Eliminate needless query plan Eliminate needless query plan constructionconstruction
Cached query plans Cached query plans cancan improve improve performance performance
© Victor Isakov [email protected]
Compiled Compiled Execution Plan Execution Plan
QueryQuery
ChangesChanges
Stored Procedure ArchitectureStored Procedure Architecture
SQL Server
Security
UnauthorizedUnauthorized
Stored Procedure
SELECT logic SELECT logic
Business Rules Business Rules
Constraints Constraints
© Victor Isakov [email protected]
Execution Plan in DetailExecution Plan in Detail
Query PlanQuery PlanRe-Entrant, Read-Only StructureRe-Entrant, Read-Only Structure
SharedShared
Separate Parallel / Serial PlansSeparate Parallel / Serial Plans
Execution ContextExecution ContextUser ContextUser Context
Parameter Values, etcParameter Values, etc
Not SharedNot Shared
Can be Re-UsedCan be Re-Used
© Victor Isakov [email protected]
AgendaAgenda
Stored Procedure ArchitectureStored Procedure Architecture
Stored Procedure OptimizationStored Procedure Optimization
Managing the Query Plan for Managing the Query Plan for Performance.Performance.
Undocumented Microsoft Stored Undocumented Microsoft Stored ProceduresProcedures
© Victor Isakov [email protected]
CREATE PROCEDURE…
Stored Procedure OptimizationStored Procedure Optimization
Parse TSQL Syntax
Resolve References
Save in Database
EXEC @RC=MyProc…
Resolve references
Optimize
Compile
Cache (RAM)
Execute
© Victor Isakov [email protected]
Compilation in DetailCompilation in Detail
© Victor Isakov [email protected]
RAM Cache
Stored Procedure OptimizationStored Procedure Optimization
SQL Server
1st instance compiled and QP loaded2nd instance shares loaded QP (new EC)
3rd instance shares first QP (& 1st EC)
1st instance finishes
Data Pages
EXEC @RC=MyProc…
© Victor Isakov [email protected]
Compilation / Execution in DetailCompilation / Execution in Detail
© Victor Isakov [email protected]
Stored Procedures OptimizationStored Procedures Optimization
Ad-hoc QueriesAd-hoc QueriesCompiled when first usedCompiled when first used
EP EP potentiallypotentially cached cached
Subsequent referencesSubsequent referencesQuery Optimizer compares Query Optimizer compares query with query with Query-TemplatesQuery-Templates
Conservative Conservative
Use cached plan Use cached plan if if it’s it’s recognizedrecognized
Auto-ParameterizationAuto-Parameterization
Cach
e (R
AM
)
Data Pages
© Victor Isakov [email protected]
Processing T-SQL in DetailProcessing T-SQL in Detail
© Victor Isakov [email protected]
Stored Procedure OptimizationStored Procedure Optimization
SPs compiled on first useSPs compiled on first useQuery plan cached in RAMQuery plan cached in RAM
Subsequent calls generally Subsequent calls generally use cached planuse cached plan
Recompiled if….Recompiled if….
Cach
e (R
AM
)
Data Pages
© Victor Isakov [email protected]
RecompilationRecompilation
WITH RECOMPILE in CREATE WITH RECOMPILE in CREATE PROCEDURE or EXECUTE statementPROCEDURE or EXECUTE statement
Schema changes to referenced objectsSchema changes to referenced objectsTable referenced by stored procedureTable referenced by stored procedure
Adding / dropping constraints, defaults, Adding / dropping constraints, defaults, or rulesor rules
sysobjects.schema_versysobjects.schema_ver
Restoring the database or any objects Restoring the database or any objects procedure referencesprocedure references
© Victor Isakov [email protected]
RecompilationRecompilation
Running sp_recompile on tableRunning sp_recompile on table
SET OptionsSET OptionsMay affect query resultMay affect query result
ANSI_DEFAULTSANSI_DEFAULTS
ANSI_NULLSANSI_NULLS
ANSI_PADDINGANSI_PADDING
ANSI_WARNINGSANSI_WARNINGS
CONCAT_NULL_YIELDS_NULLCONCAT_NULL_YIELDS_NULL
Certain operations on temporary tablesCertain operations on temporary tables
© Victor Isakov [email protected]
RecompilationRecompilation
Interleaving DDL and DML in procedureInterleaving DDL and DML in procedure
Server activity ages plan out of cacheServer activity ages plan out of cache
Table TypeTable Type Empty ConditionEmpty Condition Threshold When EmptyThreshold When Empty Threshold When Not EmptyThreshold When Not Empty
PermanentPermanent < 500 rows< 500 rows Changes >= 500Changes >= 500 Changes >= 500 + (20% of Cardinality)Changes >= 500 + (20% of Cardinality)
TemporaryTemporary < 6 rows< 6 rows Changes >= 6Changes >= 6 Changes >= 500Changes >= 500
VariableVariable Change in Cardinality does not affect Automatic RecompilationChange in Cardinality does not affect Automatic Recompilation
© Victor Isakov [email protected]
Profiler Trap SP:RecompileProfiler Trap SP:Recompile
CodeCode ReasonReason
11 Schema, bindings, or permissions changed Schema, bindings, or permissions changed between compile or execute between compile or execute
22 Statistics changedStatistics changed
33 Object not found at compile time, deferred Object not found at compile time, deferred check to run timecheck to run time
44 Set option changed in batchSet option changed in batch
55 Temp table schema, binding, or permission Temp table schema, binding, or permission changedchanged
66 Remote rowset schema, binding, or permission Remote rowset schema, binding, or permission changed changed
© Victor Isakov [email protected]
Avoiding RecompilationAvoiding RecompilationUse sp_executesqlUse sp_executesql
Preferred MethodPreferred Method
Calling Stored Procedure unaffectedCalling Stored Procedure unaffected
Use Sub-Procedures Use Sub-Procedures Only smaller sub-procedure recompiledOnly smaller sub-procedure recompiled
Use KEEP PLAN OptionUse KEEP PLAN Option
SET OptionsSET Options
© Victor Isakov [email protected]
AgendaAgenda
Stored Procedure ArchitectureStored Procedure Architecture
Stored Procedures OptimizationStored Procedures Optimization
Managing Query Plan PerformanceManaging Query Plan Performance
Undocumented Microsoft Stored Undocumented Microsoft Stored ProceduresProcedures
© Victor Isakov [email protected]
Query OptimizationQuery Optimization
All input parametersAll input parametersWhether used or notWhether used or not
Suitable indexesSuitable indexes
Data statistics Data statistics
AllAll logic in the procedure logic in the procedureWhether or not the code is Whether or not the code is executedexecuted
Query ComplexityQuery Complexity
IF
IF
© Victor Isakov [email protected]
Query Optimization in DetailQuery Optimization in Detail
© Victor Isakov [email protected]
Managing Query Plan PerformanceManaging Query Plan Performance
Generated query plan based on parametersGenerated query plan based on parametersProvided by Provided by firstfirst query (arbitrary) query (arbitrary)
Cached and reused for all subsequent useCached and reused for all subsequent useRegardless of suitabilityRegardless of suitability
Some queries Some queries run normally, run normally, others do notothers do not
© Victor Isakov [email protected]
Managing the CacheManaging the Cache
Flushing the CacheFlushing the CacheDBCC FREEPROCCACHEDBCC FREEPROCCACHE
DBCC DROPCLEANBUFFERSDBCC DROPCLEANBUFFERSTip: Should CHECKPOINT first to ensure all data is Tip: Should CHECKPOINT first to ensure all data is flushed, otherwise “dirty” data pages will remain in the flushed, otherwise “dirty” data pages will remain in the cache.cache.
Restart SQL ServerRestart SQL Server
Power-cycle systemPower-cycle system
© Victor Isakov [email protected]
SQL Server CacheSQL Server Cache
Least-Frequently-Used strategyLeast-Frequently-Used strategyLeast used objects flushed when cache fillsLeast used objects flushed when cache fills
Unified cache stores data and procedure pagesUnified cache stores data and procedure pagesLazywriter does know the differenceLazywriter does know the difference
Monitor cache with PerfmonMonitor cache with Perfmon
© Victor Isakov [email protected]
Inserting QP into CacheInserting QP into Cache
© Victor Isakov [email protected]
Retrieving QP From CacheRetrieving QP From Cache
© Victor Isakov [email protected]
Lazywriter Cache Processing FlowLazywriter Cache Processing Flow
© Victor Isakov [email protected]
Managing Query Plan Performance Managing Query Plan Performance
Recompilation might Recompilation might helphelp performance performance When “optimized” query is not optimalWhen “optimized” query is not optimal
Cost of recompile insignificant compared to Cost of recompile insignificant compared to poorly running querypoorly running query
Test all parameter combinationsTest all parameter combinationsCheck for consistent plans, performanceCheck for consistent plans, performance
If query plan varies based on parametersIf query plan varies based on parametersRecompile for each execution?Recompile for each execution?
Best to redesign procedureBest to redesign procedure
© Victor Isakov [email protected]
Managing Query Plan Performance Managing Query Plan Performance
Recompiling on DemandRecompiling on DemandCREATE PROCEDURE … WITH RECOMPILECREATE PROCEDURE … WITH RECOMPILE
Compiles QP each time stored procedure is executedCompiles QP each time stored procedure is executed
EXECUTE … WITH RECOMPILEEXECUTE … WITH RECOMPILEWhen parameters are not “typical” When parameters are not “typical”
sp_recompilesp_recompileForces Forces allall plans to be recompiled (very cheap) plans to be recompiled (very cheap)
Point to stored procedure, table…Point to stored procedure, table…
Statement-based recompileStatement-based recompileDynamic string execution (dangerous, but powerful)Dynamic string execution (dangerous, but powerful)
Smaller, more-focused proceduresSmaller, more-focused procedures
© Victor Isakov [email protected]
Managing Query Plan Performance Managing Query Plan Performance
Use Query Analyzer to view Query PlanUse Query Analyzer to view Query PlanExecute query with a range of input Execute query with a range of input parametersparameters
Clear procedure and data cacheClear procedure and data cache
View IO StatisticsView IO Statistics
Enable “Show Execution Plan”Enable “Show Execution Plan”
© Victor Isakov [email protected]
Managing Query Plan Performance Managing Query Plan Performance
© Victor Isakov [email protected]
Managing Query Plan PerformanceManaging Query Plan Performance
© Victor Isakov [email protected]
Managing Query Plan Performance StrategyManaging Query Plan Performance Strategy
Flush cache DBCC FREEPROCCACHEFlush cache DBCC FREEPROCCACHE
Force recompile WITH RECOMPILEForce recompile WITH RECOMPILE
Avoid “all-purpose” stored proceduresAvoid “all-purpose” stored procedures
SET OptionsSET Options
© Victor Isakov [email protected]
Managing Query Plan Performance StrategyManaging Query Plan Performance Strategy
Re-engineer stored proceduresRe-engineer stored proceduresBreak up larger, more complex proceduresBreak up larger, more complex procedures
Each sub-procedure gets its Each sub-procedure gets its own query planown query plan
Design procedures to work Design procedures to work with “typical” parmswith “typical” parms
Build special case proceduresBuild special case procedures
Use templates!Use templates!
© Victor Isakov [email protected]
PerformancePerformance
Construct Command object
Execute query
Parse, resolve, build QP
Execute query
Return resultsets
Client-side processing
Don’t sweat the small stuff…Don’t sweat the small stuff…
Client-side preparation
Client-side consumption
Server-side execution
© Victor Isakov [email protected]
AgendaAgenda
Stored Procedure Architecture.Stored Procedure Architecture.
Stored Procedures OptimizationStored Procedures Optimization
Managing Query Plan PerformanceManaging Query Plan Performance
Undocumented Microsoft Stored Undocumented Microsoft Stored ProceduresProcedures
© Victor Isakov [email protected]
Undocumented Microsoft Stored ProceduresUndocumented Microsoft Stored Procedures
Stored Procedure Architecture.Stored Procedure Architecture.
Stored Procedures OptimizationStored Procedures Optimization
Managing Query Plan PerformanceManaging Query Plan Performance
Undocumented Microsoft Stored Undocumented Microsoft Stored ProceduresProcedures
© Victor Isakov [email protected]
Undocumented Microsoft Stored ProceduresUndocumented Microsoft Stored Procedures
Seven for the DBASeven for the DBAsp_EnumErrorLogssp_EnumErrorLogs
sp_ReadErrorLogsp_ReadErrorLog
sp_TempDbSpacesp_TempDbSpace
sp_EnumOleDbDatasourcessp_EnumOleDbDatasources
sp_MSForEachDbsp_MSForEachDb
sp_MSLoginMappingssp_MSLoginMappings
sp_MSTableSpacesp_MSTableSpace
© Victor Isakov [email protected]
sp_EnumErrorLogssp_EnumErrorLogs
© Victor Isakov [email protected]
sp_ReadErrorLogsp_ReadErrorLog
© Victor Isakov [email protected]
sp_TempDbSpacesp_TempDbSpace
© Victor Isakov [email protected]
sp_EnumOleDbDataSourcessp_EnumOleDbDataSources
© Victor Isakov [email protected]
sp_MSForEachDBsp_MSForEachDB
© Victor Isakov [email protected]
sp_MSLoginMappingssp_MSLoginMappings
© Victor Isakov [email protected]
sp_MSTableSpacesp_MSTableSpace
© Victor Isakov [email protected]
Undocumented Microsoft Stored ProceduresUndocumented Microsoft Stored Procedures
Seven for the DeveloperSeven for the Developersp_GetTypeStringsp_GetTypeString
sp_MSCheck_UId_Owns_Anythingsp_MSCheck_UId_Owns_Anything
sp_MSGet_Qualified_Namesp_MSGet_Qualified_Name
sp_MSForEachTablesp_MSForEachTable
sp_MSIs_PK_Colsp_MSIs_PK_Col
sp_MSTable_Has_Unique_Indexsp_MSTable_Has_Unique_Index
sp_MSTableRefssp_MSTableRefs
© Victor Isakov [email protected]
sp_GetTypeStringsp_GetTypeString
© Victor Isakov [email protected]
sp_MSCheck_UId_Owns_Anythingsp_MSCheck_UId_Owns_Anything
© Victor Isakov [email protected]
sp_MSGet_Qualified_Namesp_MSGet_Qualified_Name
© Victor Isakov [email protected]
sp_MSForEachTablesp_MSForEachTable
© Victor Isakov [email protected]
sp_MSIs_PK_Colsp_MSIs_PK_Col
© Victor Isakov [email protected]
sp_MSTable_Has_Unique_Indexsp_MSTable_Has_Unique_Index
© Victor Isakov [email protected]
sp_MSTableRefssp_MSTableRefs
© Victor Isakov [email protected]
SummarySummary
Audience:Audience:““Who cares, let us go home! Please!”Who cares, let us go home! Please!”
Victor: Victor: ““But I need to put up something…”But I need to put up something…”
Understand how SQL Server worksUnderstand how SQL Server works
Helps you help SQL Server toHelps you help SQL Server toexecute “optimal” plansexecute “optimal” plans
““KISS” PrincipleKISS” Principle
Slides are up on www.sqlserver.org.auSlides are up on www.sqlserver.org.au