SQL SERVER 2005/2008

Performance tuning for

the developer
Michelle Gutzait

[email protected]
[email protected]
Blog: http://michelle-gutzait.spaces.live.com/default.aspx
 Whoami?
SQL Server Team Lead @ www.pythian.com
24/7 Remote DBA services

I live in Montreal

[email protected]
[email protected]
Blog: http://michelle-gutzait.spaces.live.com/default.aspx
2
 Agenda Part I
General concepts of performance and
tuning
Performance bottlenecks
Optimization tools
Table and index
The data page
the optimizer
Execution plans

3
 Agenda Part II
Development performance Tips
T-SQL commands
Views
Cursors
User-defined functions
Working with temporary tables and table variables
Stored Procedures and functions
Data Manipulation
Transactions
Dynamic SQL
Triggers
Locks
Table and database design issues

4
The fact that I can
does not mean that I
should !
Kimberly Tripp (?)

5
Always treat your
code as if its
running:
Frequently
On large amount of data
In a very busy environment
6
 The goal

Minresponse time and Max

throughput
Reduce network traffic, disk I/O
and CPU time

Start optimizing as early as

possible as it will be harder
later.

7
Design and Tuning Tradeoffs

8
Client/Server Tuning Levels
Presentation Layer

Application Logic Client

Side
Client OS

Network
Operating
System and Network Communication
Hardware
Network

OS/IO Subsystem Server

SQL Server Side

Database Applications
9
The Typical Performance
Pyramid
Application / Query / Database
Design

Operating Environment

Beware: In Hardware
certain
environments
this pyramid
may be upside
down!
10
Application & performance

11
 The result

Ugly code may perform

much better

12
Performance bottlenecks - tools

Windows Performance Monitor

SQL Server Profiler

SQL Server Management Studio

13
Performance bottlenecks tools
(Cont)

Database Engine Tuning Advisor

DMVs and statistics

SQL Server 2008 Activity Monitor

14
Performance bottlenecks - tools
3-rd party tool

15
Lets remember few basic
concepts

16
 Tables and Indexes

Possible
bottleneck

Possible bottleneck

Possible
bottleneck Disk
17
 Rows On A Page

96 Page Header
bytes

Row A

Row C
8,096 Data rows
bytes
Row B

C B A Row Offset Table

2 bytes each
18
 The Data Row

Data

Header Fixed data NB VB Variable data

4 bytes
Null
Block
Variable
Block

19
Data access methods

20
 Index
Helps locate data more rapidly

21
Index Structure: Nonclustered Index

22
Structure of Clustered Index

23
Covering Index

24
Index

25
 Heap table
A table with no clustered index

RID is built from file:page:row

26
Table Scan

Will usually
be faster
using a
clustered
index

27
 Parsing

Sequence Tree

Normalization

Execution Is SQL? SQL

Plan cost Trivial Plan

Optimization

based Syntatic
Transformation

optimization T-SQL
Optimization

SARG Selection
NO
Index Selection

Optimizer hints JOIN Selection

View optimizer info

Execution Plan Is Cheap
Yes
Enough?

Caching

Memory Allocation

Execution 28
Few concepts in the Execution
Plan algorithm

29
 Search ARGuments
SARG
Always isolate Columns

SARG NOT SARG

where MonthlySalary > 600000/12 where MonthlySalary * 12 > 600000

where ID in (select ID from vw_Person) where dbo.fu_IsPerson(ID) = 1234

where firstname like 'm%' where SUBSTRING(firstname,1,1) = 'm

SARG:
= BETWEEN, >, <, LIKE x%, EXISTS
Not SARGABLE:
LIKE %x, NOT LIKE, NOT EXISTS, FUNCTION(column)
AND creates a single SARG
OR creates multiple SARGs
30
Table, column and index statistics
state Step Step #
Sales
AL AL 0
AK
CA
CA CA 1
CA
CT
IL IL 2
IL
IL sys.sysobjvalues (internal)
IL IL 3
statblob
IL
MT AL
OR 4 CA
OR
OR IL
PA TX 5 IL
TX OR
TX TX
WA WA 6 WA
WA WY
WA
WI WY 7
31
WY
Update statistics - Rules of thumb
Use auto create and auto update statistics
5% of the table changes
Still bad query:
Create statistics
Update statistics with FULLSCAN
Use multi-column statistics when queries have multi-
column conditions

Use AUTO_UPDATE_STATISTICS_ASYNC
database option
No stats for temporary objects and functions
32
 Join selection
JOIN Types
NESTED LOOP

MERGE

HASH

Factors:
JOIN strategies
JOIN order
SARG
Indexes
33
 Joins - Optimization tip

HASH Joins are used when no useful index

exists on one or both of the JOIN inputs.

These can be converted to MERGE or LOOP

joins through effective indexing.

34
 Index intersection
SELECT *
FROM authors
WHERE au_fname = Fox' AND au_lname
= Mulder'

35
Tuning with indexes

Index

36
 Index tips
MORE indexes for queries, LESS indexes for updates
More indexes more possibilities for optimizer
Having a CLUSTERED INDEX is almost always a good
idea

Sort operations: TOP, DISTINCT, GROUP BY, ORDER BY

and JOIN; WHERE
As narrow as possible to avoid excessive I/O
Use integer values rather than character values
Values with low selectivity
covering index - faster than a clustered index
37
 Index tips 2
CLUSTERED index key in all non-clustered indexes
(otherwise RID is used)
Frequently updated column and clustered index

Drop costly UNUSED indexes

High volume inserts incremental Clustered index
Surrogate integer primary key (identity ?)
Clustered index for random modifications and index
bottleneck
CLUSTERED index on non-unique columns 16 bytes
added (uniqueidentifier)

38
 Index tips 3
Creating index before rare heavy operations

When Changing/dropping CLUSTERED index, drop all

NON-CLUSTERED indexes first.
Dont forget to recreate them later

Indexes are almost always in cache, therefore are faster

Column referenced by OR and no index on the column
table scan.
PRIMARY KEY and UNIQUE CONSTRAINTS create
indexes
Foreign Keys do NOT create indexes

39
 Index tips 4
Wide and fewer indexes are sometimes better
than many and narrower indexes
INCLUDE columns for covering index
Indexes are used to reduce the number of rows
fetched, otherwise they are not necessary

If TEMPDB resides on different physical disk,

you may use SORT_IN_TEMPDB

40
Analyze execution plans and
statistics
Demo - Indexes

41
Fill Factor and PAD_INDEX

Default Fillfactor 0 data pages 100% full 42

Data modifications

43
 Data modifications
In-place
direct
96 Page Header
bytes
Row D
Row A Row A Ver 2

Row E
Row C
8,096 Data rows
bytes
Row B

C B A Row Offset Table

2 bytes each
44
 Data modifications
In-place
indirect
96 Page Header
bytes
Row D
Row A Row A

Row E
Row C
8,096 Data rows
bytes
Row B

C B A Row Offset Table

2 bytes each
45
 Data modifications
Differed
update
96 Page Header forwarded
bytes
Row D
Row A

Row E
Row C
8,096 Data rows
bytes
Row B In a heap rows are
forwarded leaving old
address in place

C B A Row Offset Table

2 bytes each
46
Index fragmentation

47
 INDEXES - fragmentation
DBCC SHOWCONTIG ('Orders)

DBCC SHOWCONTIG scanning 'Orders' table...

Table: 'Orders' (21575115); index ID: 1, database ID: 6
TABLE level scan performed.
- Pages Scanned................................: 20
- Extents Scanned..............................: 5
- Extent Switches..............................: 4
- Avg. Pages per Extent........................: 4.0
- Scan Density [Best Count:Actual Count].......: 60.00%
[3:5]
- Logical Scan Fragmentation ..................: 0.00%
- Extent Scan Fragmentation ...................: 40.00%
- Avg. Bytes Free per Page.....................: 146.5
- Avg. Page Density (full).....................: 98.19

SELECT *
FROM sys.dm_db_index_physical_stats
(DatabaseID, TableId, IndexId, NULL, Mode)
48
Indexed Views
Possible bottleneck

SELECT t1.Col2, t2.Col3,

count(*) as Cnt
FROM Table_1 t1
INNER JOIN Table_2 t2
ON t1.Col1 = t2.Col1
GROUP BY t1.Col2, t2.Col3

49
 Performance tuning SQL Statements
involves doing things to allow the optimizer
make better decisions

Your options for performance

tuning are indexing or rewriting

50
 End of Part I

Questions

51
 Agenda Part II
Development performance Tips
T-SQL commands
Views
Cursors
User-defined functions
Working with temporary tables and table variables
Stored Procedures and functions
Data Manipulation
Transactions
Dynamic SQL
Triggers
Locks
Table and database design issues
52
Returning/processing too much
data

53
 Presentation Layer

Application Logic

Client OS

Network

Network
Disk
OS/IO Subsystem

SQL Server

Database Applications

54
 What could possibly be wrong
with this query ?
SELECT * FROM MyTable WHERE Col1 = x

SELECT Col1 FROM MyTable1, MyTable2

SELECT TOP 2000000 Col1 FROM MyTable1

Looping on the Client side:

WHILE @i < 10000
Update tb1 WHERE Col = @i
@i = @i + 1

55
 What could possibly be wrong
with this query (cont) ?
SELECT *
FROM MyTable t1
INNER JOIN MyTable_2 t2 on t1.Col1 = t2.Col1
INNER JOIN MyTable_3 t3 on t1.Col1 = t3.Col1
LEFT JOIN MyTable_4 t4 on t1.Col1 = t4.Col1
LEFT JOIN MyTable_5 t5 on t1.Col1 = t5.Col1
LEFT JOIN MyTable_6 t6 on t1.Col1 = t6.Col1
LEFT JOIN MyTable_7 t7 on t1.Col1 = t7.Col1
LEFT JOIN MyTable_8 t8 on t1.Col1 = t8.Col1
LEFT JOIN MyTable_9 t9 on t1.Col1 = t8.Col1
LEFT JOIN MyTable_10 t10 on t1.Col1 = t8.Col1

56
 What is the difference?
Short Long(er) ?

IF EXISTS SELECT @rc=COUNT(*)

(SELECT 1 FROM MyTable) FROM MyTable
IF @rc > 0
IF EXISTS IF EXISTS
(SELECT 1 FROM MyTable) (SELECT * FROM MyTable)
IF EXISTS IF NOT EXISTS
(SELECT 1 FROM MyTable) (SELECT 1 FROM MyTable)

SELECT MyTable1.Col1, SELECT MyTable1.Col1,

MyTable1.Col2 MyTable1.Col2
FROM MyTable1 FROM MyTable1
INNER JOIN MyTable2 WHERE MyTable1.Col1 IN
ON MyTable1.Col1 = MyTable2.Col1 (SELECT MyTable2.Col1
FROM MyTable2)

57
 What is the difference?
Short Long(er) ?
SELECT MyTable1.Col1, SELECT MyTable1.Col1,
MyTable1.Col2 MyTable1.Col2
FROM MyTable1 FROM MyTable1
WHERE MyTable1.Col1 IN WHERE EXISTS
(SELECT MyTable2.Col1 (SELECT 1
FROM MyTable2) FROM MyTable2.Col1
WHERE MyTable2.Col1 =
MyTable1.Col1)

58
Sorting the data

59
 What is the difference?
Sort No sort

SELECT Col1 SELECT Col1

FROM Table1 FROM Table1
UNION UNION ALL
SELECT Col2 SELECT Col2
FROM Table2 FROM Table2

SELECT DISTINCT Col1 SELECT Col1

FROM Table1 FROM Table1
SELECT Col1 SELECT Col1
FROM Table1 FROM Table1
WHERE col2 IN (SELECT DISTINCT Col3 WHERE col2 IN (SELECT Col3
FROM Table2) FROM Table2)
CREATE VIEW VW1 CREATE VIEW VW1
SELECT * FROM DB2..Table1 SELECT * FROM
ORDER BY Col1 DB2..Table1

60
 Which one is BETTER ?
Sort No sort

SELECT Col1 SELECT Col1

FROM Table1 FROM Table1
WHERE ModifiedDate WHERE ModifiedDate =
IN (SELECT TOP 1 (SELECT MAX(ModifiedDate )
FROM Table1 FROM Table1)
ORDER BY ModifiedDate
DESC)

61
The OR operator

62
 What is the difference?
OR No OR

SELECT Col1 SELECT Col1

FROM Table1 FROM Table1
WHERE Col1 = x WHERE Col1 = x
OR Col2 = y UNION
SELECT Col1
FROM Table1
WHERE Col2 = y
SELECT Col1 SELECT Col1
FROM Table1 FROM Table1
WHERE Col1 IN WHERE EXISTS (SELECT 1 FROM Table2
(SELECT C1 FROM Table2) WHERE Col1 = C1)
OR Col1 IN UNION ALL
(SELECT C2 FROM Table2) SELECT 1 FROM Table2
WHERE Col1 = C2)

SELECT * SELECT *
FROM Table1 FROM Table1
WHERE Col1 IN
(SELECT C1 FROM Table2) ????
OR Col2 IN
(SELECT C2 FROM Table2)

63
Locks

64
 Lock granularity

Row Locks
Page Locks
Table Locks

65
 Lock granularity

Row Locks
Table Locks
Page Locks > 5000
locks

66
Principal lock types

S
U
X
Dirty Read

WITH (NOLOCK)
SET TRANSACTION ISOLATION LEVEL READ
UNCOMMITTED

68
Nonrepeatable Read

Default

69
Phantom Read

70
ANSI Isolation Level

Dirty Reads Nonrepeatable Phantom

Reads Reads
ReadLevel 0
uncommitted

ReadLevel
committed
1
(DEFAULT)

Repeatable
Level 2 reads
Serializable
Level 3

SNAPSHOT

71
Programming with isolation
level locks

Database
Transaction
Statement/table

72
Isolation levels - example
USE pubs
GO
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE
GO
BEGIN TRANSACTION
SELECT au_lname FROM authors WITH (NOLOCK)
GO
The locks generated are:
EXEC sp_lock
GO

73
 EXEC Sp_lock
spid dbid ObjId IndId Type Resource Mode Status
51 5 0 DB S GRANT

51 10 85575343 2 KEY (a802b526c101) RangeS-S GRANT

51 10 85575343 2 KEY (54013f7c6be5) RangeS-S GRANT
51 10 85575343 2 KEY (b200dbb63a8d) RangeS-S GRANT
51 10 85575343 2 KEY (49014dc93755) RangeS-S GRANT
51 10 85575343 2 KEY (170130366f3d) RangeS-S GRANT
51 10 85575343 2 PAG 1:1482 IS GRANT
51 10 85575343 2 KEY (c300d27116cf) RangeS-S GRANT
51 10 85575343 0 TAB IS GRANT

51 10 85575343 2 KEY (1101ed75c8f8) RangeS-S GRANT

51 10 85575343 2 KEY (2802f6d3696b) RangeS-S GRANT
51 10 85575343 2 KEY (0701fdd03550) RangeS-S GRANT
51 10 85575343 2 KEY (7f00d0d5506b) RangeS-S GRANT

SELECT object_name(85575343)
GO
-----------------------------
authors 74
Temporary Objects

75
 Temporary objects
#tmp
##GlobalTmp
Tempdb..StaticTmp
@TableVariable
Table-valued functions
Common Table Extention (CTE)
View ?
FROM (SELECT )

76
Stored Procedures

77
 What are the benefits of
Stored Procedures?
Reduce network traffic
Reusable execution plans
Efficient Client execution requests
Code reuse
Encapsulation of logic
Client independence
Security implementation
As a general rule of thumb, all Transact-SQL
code should be called from stored procedures.

78
 Stored Procedures tips
SET NOCOUNT ON
No sp_
Owned by DBO
Exec databaseowner.objectname
Select from databaseowner.objectname
Break down large SPs

79
 SP Recompilations
#temp instead of @Temp table variables
DDL statements
Some set commands

Use SQL Server Profiler to check recompilations

80
 Which one is better and why?
IF @P = 0
SQL Statement Block1
ELSE
SQL Statement Block2

IF @P = 0
Exec sp_Block1
ELSE
Exec sp_Block2

81
What could be problematic
MyTable
here? PK
CREATE PROC MySP 0
@p_FROM INT, @p_TO INT 5
AS 10
SELECT count(*) FROM MyTable
34
WHERE PK
between @p_FROM and @p_TO 87

CREATE WITH RECOMPILE 198,739
EXECUTE WITH RECOMPILE .
sp_recompile objname 3,898,787

7 million rows

82
 Dynamic SQL
Sp_exectusql VS. execute

83
 Which one is better and why?

EXEC (SELECT Col1 FROM Table1 +

WHERE + @WhereClause)

Exec sp_executesql @SQLString

Exec sp_executesql @SQLString,

@ParmDefinition, @PK = @IntVariable

84
Reusable Execution (Query) Plan -
generated by sp_executesql

85
Cursors

86
 Cusrors - implications
Resources Required at Each Stage

87
 What could possibly replace
cursors?
Loops ?
Temp tables
Local variables (!)
CTEs
CASE statements
Multiple queries
AND

88
 Replacing cursor
Tip #1
Select Seq=identity(int,1,1),
Seq Fld1 Fld2 ..
Fld1,
Fld2, 1 Aaa 45.7

2 Absb 555.0
Into #TmpTable
From Table1 3 Adasd 12.8

Order by 4 oioiooi 0.0

.. .. .. ..

89
 Replacing cursor
Tip #2
declare @var int

set @var = 0

Update Table1
set @Var = Fld2 = Fld2 + @Var
From Table1 with (index=pk_MyExampleTable)
option (maxdop 1)
go

90
Cursor Example

91
TRY ME.

92
Optimizer Hints

93
 Optimizer Hints
Most common
WITH (ROWLOCK)
WITH (NOLOCK)
WITH (INDEX = IX_INDEX_NAME)
WITH (HOLDLOCK)

SET FORCEPLAN ON
OPTION (MAXDOP 1)
Join hints (MERGE/HASH/LOOP)
Isolation levels WITH (SERIALIZABLE, READ COMMITED)
Granularity level (UPDLOCK, TABLOCK, TABLOCKX)
94
What is possibly wrong here?
MyTable
Col1
BEGIN
BEGINTRAN
TRAN x

MyTableSET
SETCol1Col1==xx
UPDATE
UPDATEMyTable x
WHERE
WHERECol1
Col1IN
IN y
(SELECT
(SELECTCol1
Col1from
fromMyTable_2)
MyTable_2 y
COMMIT TRAN WITH (NOLOCK) ) y
COMMIT TRAN
m
.
z

95
 Tip

If your database is Read Only in

nature, change it to be as such!

96
The Transaction Log
T-LOG

97
 What is wrong here?
MyTable
Col1
BEGIN TRAN x
UPDATE MyTable SET Col1 = x x
WHERE Col1 = y y
IF @@ROWCOUNT <> 10
y
ROLLBACK TRAN
y
COMMIT TRAN

m
.
z

1000 rows with

Col1 = y
98
 What could be possibly
MyTable
wrong here? Col1
x
BEGIN TRAN
x
DELETE MyTable
COMMIT TRAN y
y

T-Log size y

Concurrency
m
How do we solve this ? .
What if we have a WHERE clause in the z
DELETE ?
7 million
rows
99
 Transaction Habits

As short as possible
Long transactions:
Reduce concurrency
Blocking and deadlocks more likely
Excess space in transaction log to not be
removed.
T-log IO
No logical ROLLBACKS!

100
Triggers

101
 What is wrong here?
MyTable
PK Insert
CREATE TRIGGER TRG_MyTable_UP Date
ON MyTable 1
AFTER INSERT 5
AS 13
UPDATE MyTable 67
SET InsertDate = getdate() 89
FROM MyTable

INNER JOIN inserted
1234
ON MyTable.PK = inserted.PK
.
345667

102
Typical Trigger Applications
Cascading modifications through related tables
Rolling back changes that violate data integrity
Enforcing restrictions that are too complex for
rules or constraints
Maintaining duplicate data
Maintaining columns with derived data
Performing custom recording

Try to use constraints instead of triggers,

whenever possible.

103
Tables Design Issues

104
 Column name Type Property Key/index
Employees
table Employee ID Int NOT NULL Clustered
Identity (values are unique)
First Name Char(100) NOT NULL
Last Name Char(100) NOT NULL
Hire Date Datetime NULL
Description Varchar(8000) NULL

ContractEndDate Char(8) NOT NULL Index

SelfDescription Varchar(8000) NOT NULL default
Picture Image NULL
Comments Text NULL

Application rules:
All queries fetch EmployeeID , FirstName, LastName and HireDate WHERE EmployeeID
equals or BETWEEN two values, where ContractEndDate >= getdate()
All other column are fetched only when user drills down from application
FirstName, LastName, HireDate and ContractEndDate rarely change
Comments , Description And SelfDescription are rarely filled up and they never appear in
the WHERE clause
Picture column is ALWAYS updated after row already exists.
Once the contract ends, the data should be saved but will not be queried by application
105
Column name Type Property Key/index
Employee ID Int NOT NULL Clustered
Clustered
Identity (values are unique) UNIQUE

First Name Varchar(100)

Char(100) NOT NULL
Last Name Char(100)
Varchar(100) NOT NULL
Hire Date Datetime NULL
Description Varchar(8000) NULL

ContractEndDate Datetime
Char(8) NOT NULL Index
SelfDescription Varchar(8000) NULLNULL default
NOT
Picture Varbinary(MAX)
Image NULL
Comments Varchar(MAX)
Text NULL

First

106
 Employees (active
employees) OldEmployees (inactive
employees)
Column name Key/index
Column name Key/index
Employee ID Clustered PK
Employee ID Clustered PK
First Name
First Name
Last Name
Hire Date
This is vertical Last Name

ContractEndDate Index partitioning Hire Date

Description
ContractEndDate
Employees details 1:1 SelfDescription
Picture
Column name Key/index Comments

Employee ID Clustered PK

Description

SelfDescription
4 different tables?
Picture
Comments

107
Column name Type Horizontal partitioning
Employee ID INT

First Name Varchar(100)

Last Name Varchar(100)
Contract Date < 2008-01-01
Hire Date Datetime
ContractEndDate Datetime

Column name Type

Employee ID INT

First Name Varchar(100)

Last Name Varchar(100)
Contract Date >= 2008-01-01
Hire Date Datetime
ContractEndDate Datetime and < 2009-01-01

Column name Type

Employee ID INT

First Name Varchar(100)

Last Name Varchar(100)
Hire Date Datetime
ContractEndDate Datetime Contract Date >= 2009-01-01
108
Tips for the application side

109
 Beware of
Server-side cursors prior to .NET 2.0
Sorts and grouping on the client
End-user reporting
Default Transaction isolation levels
Intensive communication with database
Connection pooling
Long transactions
Ad-hoc T-SQL
SQL injection

110
 Performance Audit Checklist
Does the Transact-SQL code return more data than needed?
Is the interaction between the application and the Database Server too often.
Are cursors being used when they don't need to be? Does the application
uses server-side cursors?
Are UNION and UNION ALL properly used?
Is SELECT DISTINCT being used properly?
Is the WHERE clause SARGable?
Are temp tables being used when they don't need to be?
Are hints being properly used in queries?
Are views unnecessarily being used?
Are stored procedures and sp_executesql being used whenever possible?
Inside stored procedures, is SET NOCOUNT ON being used?
Do any of your stored procedures start with sp_?
Are all stored procedures owned by DBO, and referred to in the form of
databaseowner.objectname?
Are you using constraints or triggers for referential integrity?
Are transactions being kept as short as possible? Does the application keep
transactions open when the user is modifying data?
Is the application properly opening, reusing, and closing connections?
111
 End of Part II

Questions/
Autographs

112