Deploying a 100,000 user SharePoint farm on FlexPod

Cisco Confidential© 2012 Cisco and/or its affiliates. All rights reserved. 1

Deploying a 100,000 user SharePoint farm on FlexPod

Vadiraja Bhatt

Performance Architect, DC Solutions Engineering and Ecosystem

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 2

Agenda• SharePoint Basic Concept

• SharePoint 2010 Characteristics

• Deploying a 100,000 user SharePoint farm on FlexPod

• Solution Overview• FlexPod for VMware Components• Solution Summary• References


SharePoint 2010 Basic Concept / Enterprise Server Roles

Application server

WFE Server

SQL - Database serverClustered or Mirrored


SharePoint 2010 Characteristics Web Front End -Tier

Application -Tier

Database Tier

Service Application CPU Load Memory Load

SharePoint Foundation Service High High

Timer Service Very High Very High

Logging Service Very High Very High

Workflow Capabilities High High

Service Application CPU Load Memory Load

SharePoint Foundation Service None None

Central Administration Service Medium Medium

Timer Service Very High Very High

Logging Service None None

User Profile Service Very High Very High

Word Viewing Service High High

Service Application CPU Load I/O Load Storage

SharePoint Foundation Service Very High High High

Timer Service None None None

Logging Service Very High High High

User Profile Service High High Very High

Word Viewing Service None None None

Workflow Capabilities None None None


Deploying a 100,000 user SharePoint farm on FlexPod

• Published March 2012• 100,000 seats environment with 10% concurrency • Design considerations – compute/network/storage• Performance Study


A Single, Cohesive Management System

• Cisco provides all the components needed for the Microsoft SharePoint farm

• Controls the server, storage, and network infrastructure through a single, cohesive management system

Lower Risk

• Making a move to SharePoint 2010 on Cisco UCS is easier and less risky.

• Provides confidence via a tested and validated reference architecture for Microsoft SharePoint 2010

SimpleEliminate Unnecessary Complexity

Cisco Validated Designs Unified Platform


Introducing Cisco UCS on FlexPod

7

Benefits

• Low-risk standardized shared infrastructure supporting a wide range of environments: scale out or scale out based on needs

• Highest possible DC efficiency

Features

• Performance-matched stack

• Step-by-step deployment guides

• Solutions guide for multiple environments

• Multiple classes of compute and storage supported in a single FlexPod

• Centralized management: NetApp OnCommand and Cisco UCS™ Manager

• Ideal foundation for virtualized applications…including MS SharePoint 2010


Data Center Efficiency and Flexibility

8

Each vendor brings Powerful technologies that jointly drive data center efficiency and flexibility.

• Industry’s Leading Server Virtualization Technology

• VMware vMotion and Storage vMotion

• VMware Distributed Resource Scheduler

• Resource Pooling

• High-Density Virtualization and Computing

• Consolidate Wiring

• 10 GE Unified and Virtualized Fabric


• Storage Efficiency Guaranteed

• RAID-DP and Deduplication

• Thin Provisioning

• Space-Efficient Clones

• Thin Replication


VMware vSphere Cisco ® UCS Platform and Unified Fabric NetApp FAS


100K User Seat Deployment on Cisco UCS

FlexPod Deployment

FC

Ethernet

IOM Links IOM Links

vPCvPC

Chassis 5108

NetAPP 3270 A

Nexus 5548up

NetAPP 3270 B

FI 6248 FI 6248

Nexus 5548up

vPCvPC

• 1 x Cisco UCS 5108 Chassis

• 2 x Cisco UCS

• 2248 Fabric Extenders

• 2 x Cisco UCS 6248 up Fabric interconnects

• 2 x Nexus 5548 up

• FAS 3270 HA Pair

• 3 x DS4243 disk Shelf's


100K User Seat Deployment on Cisco UCS

ESX Deployment

Vmware vCenter

B200 M2 – ESX 2

B200 M2 – ESX 4

B200 M2 – ESX 4

B200 M2 – ESX 3

B200 M2 – ESX 1

HA DRS

vSphere

NetApp FAS 3270VMFSDataStore

LUN’s

ESXSANBOOTLUN’s

CIFSShare

RDMLUN’s

NFS

vol vol vol vol vol

Aggregate64-bit

FAS-3270 A FAS-3270 B

NetApp FAS 3270

10GE

10GEESX 2

ESX 4

ESX 4

ESX 3

ESX 1

SharePoint 2010 SP1Web Front End/App/DB Tier

VM VM VM

VM VM VM

VM VM VM

VM VM

VM VM

Windows 2008 R2 Web Front End

Windows 2008 R2 Web Application Server

Windows 2008 R2 SQL 2008 R2

Windows 2008 R2 SQL Mirror Witness Server

VM

VM

VM

VM

FlexPod constitutes various technologies mainly, Cisco Unified Computing System, VMware vSphere 5.0 and NetApp Storage Technologies together to form a highly reliable, robust and a virtualized solution for SharePoint 2010.

Highly Available SharePoint farm • VMware Datacenter

provisioned with Virtual servers -HA / DRS configured

SharePoint environment:• WFE - configured with NLB

• Application server –Multiple Application Servers

• SQL servers - Mirrored witness server


Single Large 64 bit Aggregate configured with RiadDP. Single large aggregate may maximize performance for the SharePoint data.

Storage Infrastructure

VMFS LUN(1500 GB)

RDM-LUNDB MDF

(2500 GB)

RDM-LUNDB TLog

(1000 GB)

RDM-LUNDB MDF

(2500 GB)

RDM-LUNDB TLog

(1000 GB)OS

(10 GB)OS

(10 GB)OS

(10 GB)OS

(10 GB)OS

(10 GB)

FlexVol

NFS(2 TB)

FlexVolDatabas

e(4 TB)

FlexVolMirroredDatabas

e(4 TB)

FlexVolCIFS(400 GB)

FlexVolOS Boot

(100 GB) FlexVol

Vol 0(480 GB)

SAS Drives 15K RPM

Aggregate (64 bit)

• NetApp FAS3270A

• 3 x DS4243 disk shelf

• 600GB 15k rpm SAS disks

• Single large 64 Aggregate

• Boot LUNs for the 5 ESX hosts

• NFS datastore hosts the ESX vmdk files .

• FC/VMFS LUNs with RDMs attached to database VM’s to store MDF and Tlogs Files


HypervisorHypervisor

Cisco VM-FEX with vMotion(A.K.A Universal pass through switching /Direct path i/o)

UCS 6200

VM VM

UC

S V

IC

UC

S V

IC

UCS Server UCS Server

Port ProfilesDefined in UCSM

WEB Apps

Application

DB

Compliance

VC Deploys VMs with port groups

UCSM exports Port Profiles to VC

VM

VMVNIC

VETH

vCenter

In test vMotion took 27 sec to move a VM of 200 Gb between the ESX Hosts

Enhance performance by VM-FEX


• Collaboration & Workflow both client & Web-based

• Enterprise Social Networking

• Enterprise Content Management (ECM)

• Web Content Management (WCM)

• Discovery (Search)

Server Utilized in the Test1. Load Test Controller -VSTS 2010 SP1 2. VSTS Test Agents x 12

SharePoint 2010 Workloads / VSTS Test


100K User Seat Deployment on Cisco UCS CPU utilization

Users Vs Request Per Second

Each user targeted with 60 RPH

• Avg Req/sec – 560 req/sec

• Max Req/sec – 800 req/sec

Web Front End

• 8 x WFE deployed

• Each WFE configured with 4vCPU & 16 Gb memory

• Linear growth of CPU utilization with increased

concurrent users

• Max CPU utilization – 60% for 10k users

Users1200 3200 4400 5400 6600 7800 9400

0

50

100

WFE1, Processor, % Processor Time, _Total WFE2, Processor, % Processor Time, _Total

WFE3, Processor, % Processor Time, _Total

%P

roce

sso

r T

ime

WFE Server

Average RPS achieved – 560 request/second, for multiple user load with an average for 60 request/hour/userMaximum RPS was around 800 request/second

• Linear growth of CPU utilization with increased concurrent users.

Graph shows the possibility to further scale up the user load with an

ensured stable server performance.

• Smooth performance of the Virtual servers on Cisco UCS servers

B200 M2 as the request per second scale ascending linearly with the

user load without causing much stress on the server


100K User Seat Deployment on Cisco UCS CPU utilization

Users 800 1200 2000 3200 3600 4400 4600 5400 6200 6600 7400 7800 8600 9400 98000

20406080

100

Application Server

APPLICATION1, Processor, % Processor Time, _Total

User Load

%P

roce

sso

r T

ime

The Application server hosted the Central Administration and search services. The spikes shown on the graph are due to the search crawl at

the time of the performance test.

Application Server • 2 x Application server deployed

• Each Application Server configured with 4vCPU & 32 Gb Memory

• 100% CPU utilization

Database server• Configured with 4vCPUs & 64Gb Memory

• Mirrored SQL Server with Witness Server

• Average CPU utilization – 50-60%Users 800 1200 2000 3200 3600 4400 4600 5400 6200 6600 7400 7800 8600 9400 98000

40

80

Data Base Server

SQL-DB-1, Processor, % Processor Time, _Total

User Load

%P

roce

sso

r T

ime

On an average, the overall CPU utilization at the database tier with virtual servers remained at around 60 percent. This is when the server is hosted on the Cisco UCS B250 M2


• Unified Fabric simplifies SharePoint infrastructure while delivering scalability.

• SharePoint servers, especially SQL Clusters, benefit from multiple Ethernet adapters for cluster communication, replication, and management.

• Multiple HBA provide storage access path redundancy in conjunction with Microsoft MPIO.

• Create server profiles to consistently manage server attributes for each SharePoint Server role in the farm

Service Profiles and stateless blades allow consistent management for all server attributes for each SharePoint Server role. Ease of failover. Add Capacity.

All SharePoint server roles can run in a virtual machine. Templates based Service profile for Dev/Staging /QA environment

SQL DB

App

Web

SharePoint on UCSIncreased Agility, Reduced Costs:

Microsoft hyper-v (R2 recommended) or VMware ESX supported (KB 897615)


Solution Summary

Each user was targeted with a utilization of 60 requests per hour.

Request Per Second -The number of received requests per second for several user loads (60 RPH) at the end of testing on average was ~560.

Pages Per second - SharePoint 2010 farm served on an average 332 pages per second.

Average Page Time - SharePoint 2010 average page time well below 1 sec.

Web Front End servers utilization was around ~60%.

Application server reached a bottleneck of 100% utilization.

At this ratio, the SQL database CPU was around 60% utilized.


References Cisco UCS:http://www.cisco.com/en/US/netsol/ns944/index.html

Cisco CVD FlexPod for VMware -100K user seathttp://www.cisco.com/en/US/docs/unified_computing/ucs/UCS_CVDs/ucs_sharepoint2010_flexpod_vmware.html

Cisco CVD FlexPod for VMware -50K user Seat http://www.cisco.com/en/US/docs/solutions/Enterprise/Data_Center/Virtualization/flexpod_vmware.html

Microsoft SharePoint Server 2010 on Cisco Unified Computing System Medium Farm Solution-Performance and Scalability Study http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns944/cisco_ucs_scalability_performance.pdf

Cisco Nexushttp://www.cisco.com/en/US/products/ps9441/Products_Sub_Category_Home.html

Cisco Nexus 5000http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/configuration/guide/cli/CLIConfigurationGuide.html

http://www.cisco.com/en/US/netsol/ns944/index.html

http://www.cisco.com/en/US/docs/unified_computing/ucs/UCS_CVDs/ucs_sharepoint2010_flexpod_vmware.html

http://www.cisco.com/en/US/docs/unified_computing/ucs/UCS_CVDs/ucs_sharepoint2010_flexpod_vmware.html

http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns944/cisco_ucs_scalability_performance.pdf

http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns944/cisco_ucs_scalability_performance.pdf

http://www.cisco.com/en/US/products/ps9441/Products_Sub_Category_Home.html

http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/configuration/guide/cli/CLIConfigurationGuide.html

http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/configuration/guide/cli/CLIConfigurationGuide.html

Thank you.