–

Meir Dudai

SQL Server MVP

Content slide

When do I need to start

worrying?

100’s of queries/second

1000’s of DMLs/second

VLDB’s

When do I need to start

worrying?

Application was designed and developed

poorly…

When do I need to start

worrying?

You have just migrated from other platform

What can I do?

Throw in more hardware – not very useful

What can I really do?

Improve application design

Use database enhancements and features

Change system architecture

Who Am I?

Meir Dudai

MVP

SQL Server consultant

Editor of SQLServer.co.il

How to increase concurrency

without changing your application

How to adjust your architecture to

achieve better concurrency

Some basics

Magic settings

– RCSI

– Partition-level lock escalation

– Page Latches

Architectural design

– Bulk Load

– StreamInsight

– Caching

– Offloading Traffic

– Denali HADR

Common Concurrency Issues

with Applications

Wrong transaction isolation level

Incorrect locking granularity

Missing Indexes

Accessing large dataset leading to lock

escalation

Long running transactions

– User interactions in active transaction

Accessing objects in reverse order across

transactions

Some basics

Magic settings

– RCSI

– Partition-level lock escalation

– Page Latches

Architectural design

– Bulk Load

– StreamInsight

– Caching

– Offloading Traffic

– Denali HADR

GreenRoad Case Study:

RCSI

GreenRoad is an Israeli startup improving

driving behavior for fleets and consumers

Works as SAAS

More than 400 inserts/sec

Users running queries and reports

frequently

Method used to increase concurrency:

RCSI: Read Committed Snapshot Isolation

Row-1

Tran2 (Select)Tran1 (Update)

X-Lock S-Lock BlockedRow-1

Reader Writer Blocking

Read Committed Snapshot

Statement-Level ‘Snapshot Isolation’

New “flavor” of read committed

– Turn ON/OFF on a database

Readers see committed values as of beginning of

statement

• Writers do not block Readers

• Readers do not block Writers

• Writers do block writers

Can greatly reduce locking / deadlocking without

changing applications

Demo

Lock Escalation

HOBT

Page Page Page

Row Row Row

T1: IX

T1: IX

T1: XT1: X

T1: XT1: X

T1: XT1: X

T1: XT1: X

T1: X

Lock Escalation

Converting finer-grain locks to coarse grain locks.

– Row to Table

– Page to Table.

Benefits

– Reduced locking overhead

– Reduces Memory requirement

Triggered when

– Number of locks acquired on a rowset > 5000

– Memory pressure

Partitioned Tables and Indexes

SQL Server 2005 introduced partitioning, which some

customers use to scale a query workload

– Another common use is to streamline maintenance and enable

fast range inserts and removals from tables

FG1 FG2 FG3

PartitionedTable

Partition 1 Partition 2 Partition 3

Lock escalation on partitioned tables reduces concurrency as the table lock locks ALL partitions

Only way to solve this in SQL Server 2005 is to disable lock escalation

Lock Escalation: The Problem

IXX

FG1 FG2 FG3

PartitionedTable

Partition 1 Partition 2 Partition 3

Query 1 Query 2

update update

Lock Escalation: The Solution

SQL Server 2008 allows lock escalation to the partition level, allowing concurrent access to other partitions

Escalation to partition level does not block queries on other partitions

IX

X

FG1 FG2 FG3

PartitionedTable

Partition 1 Partition 2 Partition 3

Query 1 Query 2

update update

Demo

Some basics

Magic settings

– RCSI

– Partition-level lock escalation

– Page Latches

Architectural design

– Bulk Load

– StreamInsight

– Caching

– Offloading Traffic

– Denali HADR

Silent Blocker: Page Latches

Page Header

R1

R2

R3

What is a Pagelatch?

Access patterns leading to latch contention

– Clustering index has a increasing primary key with high

rate of inserts

– Relatively small tables with a lot of read/write/delete

– Indexes with low cardinality in the key with high rate of

inserts

– Allocation Structures (e.g. SGAM)

– Observed with 16+ concurrent threads

Silent Blocker: Page Latches

What can you do?– Cluster on some other column

– Partition the table on some other attribute (computed column (hash))

– Add another major key column

– Change the application

Demo

Some basics

Magic settings

– RCSI

– Partition-level lock escalation

– Page Latches

Architectural design

– Bulk Load

– StreamInsight

– Caching

– Offloading Traffic

– Denali HADR

Codename Lyngby Case Study:

Power of bulk inserts

Security/audit system

50GB of data inserted daily

Data is kept for one year and should be

available for queries and reports at any

time

Method used to increase concurrency:

Bulk inserts

Benefits of Bulk Inserts

Improved concurrency

Allow application-database decoupling

In Lyngby case – 50GB a day!

http://www.tapuz.co.il/blog/userBlog.asp?FolderName=Hygge

Bulk Inserts & RCSI:

SQLCAT Benchmark

What happens when you query a table

while bulk inserts are running?

Bulk Inserts & RCSI:

SQLCAT Benchmark

And with RCSI?

Bulk loading does not affect the size of the version store in tempdb under RCSI

Keep in mind: RCSI adds 14 noncompressible bytes into every row in every table

Bottom line: RCSI works extremely well with bulk inserts and achieves improved concurrency

More details about this benchmark:

http://sqlcat.com/technicalnotes/archive/2009/04/06/bulk-loading-data-into-a-table-with-concurrent-queries.aspx

StreamInsight:

Complex Event Processing

• SQL Server 2008 R2 feature

• Processing and querying of event data streams

• Data queried while “in flight”

• May involve multiple concurrent event sources

• Works with high data rates

• Aims for near-zero latency

Isn’t This Just a Database

Application?

Database CEP

Queries Ad hoc on stored data Continuous standing

queries

Latency Seconds Milliseconds

Data Rate Hundreds per Second Tens of thousands per

second

request

response

Eventoutput streaminput

stream

Caching

• Your database can get some rest...

Microsoft AppFabric

Other solutions

• Memcache− Requires application change

• 3rd party− Usually transparent

Tapuz Case Study:

Offloading Traffic

Tapuz is the leading user content based

website in Israel

– 1,200 forums

– 74,000 blogs

– 500,000 visitors each month

More than 4000 queries/sec

Method used to increase concurrency:

Offloading using Transactional Replication

Database Scale Out

Queries scaled out (often geographically) similar to reporting cases

Databases replicate reciprocally and are writable

Redundancy provides fault tolerance and lowers maintenance downtime

Online upgrades possible

Denali HADR

• V.Next, code named “Denali” offers a brand new HADR feature

• High-availability and disaster recovery feature, using up to 4 replicas

• Replicas can be synchronous or asynchronous − CTP supports only single

asynchronous replica

Denali HADR

• Coming up next!

Q&A

Related sessions

Monday

• 14:00-15:15 − New High Availability Solution in SQL

Server “Denali”

− Justin Erickson & Oren Bouni

Tuesday

• 09:30-10:45 − Mission Critical: Improving High

Availability and Reliability

Assaf Fraenkel

• 12:30-13:30 − Designing High Performance I/O System

with SQL Server

− Dubi Lebel

• 11:15-12:15 − Troubleshoot Like a Pro with the

Microsoft SQL Server 2008 Tools

− Guy Glantser

Open your mind to uncommon

features and technologies in SQL

Server and try them

Using these features, amazing

concurrency can be achieved by any

DBA using SQL Server!

-

Recommended Courses: SQL 2008•Database 2008 a Microsoft SQL Server Maintaining •Database 2008 Implementing a Microsoft SQL Server

SQL

•, Implementation and Maintenance2008TS: Microsoft SQL Server 432 -70

•Development, Database 2008TS: Microsoft SQL Server 433 -70

-TechNet

-