前進 64 位元的 Exchange 2007

前進前進 6464 位元的 位元的 Exchange 2007Exchange 2007

謝合宜謝合宜微軟特約技術顧問微軟特約技術顧問MCSE : Security/Messaging MVP/MCTMCSE : Security/Messaging MVP/MCTBS7799/ISO27001 Lead AuditorBS7799/ISO27001 Lead Auditor

2

Exchange 2007 WebCast 系列課程課程名稱 時間Exchange 2007 --- 新一代的訊息系統 11/22Getting Started with Microsoft Exchange Server 2007 安裝建置的基礎 12/19

Exchange 2007 的 OWA 12/20前進 64 位元的 Exchange 2007 12/27Exchange 2007 的用戶端存取 1/10Exchange 2007 的系統部署與轉移 1/17Exchange 2007 的系統部署準備 --- 儲存預測與測試 1/24Exchange 2007 的 Management Shell 、 Scripting 與原則使用 1/31保護 Exchange 2007 免於垃圾郵件與病毒 2/7Exchange 2007 的系統備份、高可使用性與災難還原 2/14

3

Level 200

預備知識預備知識• 熟悉熟悉 Exchange 2000/2003Exchange 2000/2003 或 或 5.55.5 的系的系統管理與使用統管理與使用

4

Agenda

• 為什麼要 64 位元 ?• 64 位元的好處：證明的提出• Exchange 2007 的主要考量 ?• 準備迎接 64 位元的 Exchange 2007

5

Exchange Server 2003 的苦處• 擴展性的限制

– 資料庫大小 – 資料庫復原的限制– 4GB 記憶空間

• 資料庫暫存的資料命中率低 = 增加 I/O 負擔• 規畫時的考慮

– I/O 的獨特性 – 小量、隨機、爆發性的磁碟資料使用– 大量的磁碟空間被浪費來滿足 I/O 的需求– 多種不同類型的使用者 (Outlook 2002/2003, 黑莓 ,

OWA…)• 資源浪費在基本功能，進階功能只能用較少的資源

– 資訊的生命週期管理 – 內容的索引– 災難的還原

6

記憶體使用的限制

KernelKernel

UserUser• 32-bit 的架構只能使用 4GB 的記憶空間• 大型的 Exchange 伺服器需要設定

/3GB 參數– 減少虛擬記憶體的斷離情形– 較大的資料庫暫存

• /USERVA:3030 以 42MB 給核心來處理 System PTEs• 其他核心模式的需求

– 連線狀態的安全權杖 (Token) 需要消耗分頁區• 250MB on a server with /3GB• 8K token size = ~3,700 users (online or

cached mode) max• 請參考 hotfix 912480, KB912376

7

影響 I/O 的因素

• 影響 I/O Per Second per user (IOPS/u) 比率的因素– 同時連線的 Outlook Online 模式使用者數– 資料夾大小 ( 根據項目數量，目標 <5,000 per folder) – 資料庫大小、數量

• 傳統對 I/O 的影響率– 500 MB 1 GB mailbox size = +25% (online mode)– 1 GB 2 GB mailbox size = +40% (online mode)– 1,000 2,000 mailboxes = 25% increase– 1 20 DBs = up to 38% increase

User Type Database Volume IOPs/user Send/Receive per day Mailbox size

Light .5 20 / 50 50 MB

Average .75 30 / 75 100 MB

Heavy 1.0 40 / 100 200 MB

Large 1.5 60 / 150 500 MB

8

影響 I/O 的聚合效應• 對於 I/O 的影響因素幾乎不會單獨出現

– 信箱大小的增加經常佔有了更多的資料庫– 信箱愈多，影響愈大

I/O Impact

0

2000

4000

6000

8000

10000

12000

14000

500MB 1000MB 2000MB

Average Mailbox Size / user

Dat

abas

e IO

Ps

1,000 users, 4 DBs, 1 IOPs/u

2,000 users, 8 DBs, 1 IOPs/u

4,000 users, 20 DBs, 1 IOPs/u

1.06 1.331.86

1.43 1.782.49

1.732.16

3.02

9

為什麼 x64 很重要 ?• Exchange 的架構持續增加對 I/O 的需求

– 更多的網路流量– 更大的訊息、附件與信箱使用– 更多特殊的用戶端使用– 更多特殊的安全權限設定

• 降低 I/O 需求成為 Exchange 開發的重要目標• 減輕核心記憶體的壓力• 相關於 RCP over HTTP 的 TCP 連線限制

– 32 位元限制了 20K TCP 連線 ( 大約是 2,500 RPC over HTTP 使用者 )

10

Memory and CPU LimitsGeneral Memory Limits 32-bit 64-bit

Total Virtual Address Space 4 GB 16 TB

Virtual Address Space per 32-bit process

2GB (3 GB if system is booted with /3gb

switch)

4GB if compiled with /LARGEADDRESSAWA

RE 2GB otherwise

Virtual Address Space per 64-bit process

Not applicable 8 TB

Paged Pool 470 MB 128 GB

Non-Paged Pool 256 MB 128 GB

System Cache 1 GB 1 TB

Physical Memory and CPU Limits 32-bit 64-bit Windows XP Professional 4 GB / 1-2 CPUs 32 GB / 1-2 CPUs

Windows Server 2003 Standard Edition 4 GB / 1-4 CPUs 32 GB / 1-4 CPUs

Windows Server 2003 Enterprise Edition 64 GB / 1-8 CPUs 1 TB / 1-8 CPUs

Windows Server 2003 Datacenter Edition 64 GB / 1-32 CPUs 1 TB / 1-64 CPUs

11

Exchange 2007 概要• x64 everywhere

– 支援更大實體記憶體與定址空間 (user and kernel mode)– 提供更大的 CPU 擴展性– 對於 I/O 的關鍵性改變

• I/O 的使用變化– 資料庫暫存 1.2GB 32GB 以上– 不再有 STM, ExIFS– 更多的 SGs ( 大到 50 個 ) = 更深入的 checkpoint 使用

• 更新的搜尋引擎• 新的伺服器角色分類，關鍵任務的達成• LCR / CCR

12

STORE.exe 程序的記憶體使用

0

500

1000

1500

2000

2500

3000

Exchange 2003Exchange 2003

4 GB test4 GB testExchange 2007Exchange 2007

Elapsed time

0

2000

4000

6000

8000

10000

12000

12 GB test12 GB test

Exchange 2003Exchange 2003

Exchange 2007Exchange 2007

Elapsed time

Mem

ory

(MB)

Mem

ory

(MB)

13

資料庫暫存比較• 32GB

Ex2007 測試– 成長大概到

30GB• 8GB

Ex2003 測試– 停留在

1.2GB (max)

14

運作架構的改變• 4K 8K 分頁大小• I/O 處理 from 64KB 1MB ，較大但較少的寫入• 更多的資料預測載入• ESE 暫存使用 48-bit 定址，可使用更多實體記憶體

ProLiant DL385 2 Dual-Core CPU (2.2GHz), 4GB RAM, 1500MMB3 users, U320 SCSI 24 DB disks, 4 Logs. Search/Indexing=OFF. Exchange 2007 beta – results subject to change

Workload Profile changes

5,521

13,815

7,493

16,331

1060

2800

0

2,000

4,000

6,000

8,000

10,000

12,000

14,000

16,000

18,000

Exchange 2003 Exchange 2007

Exchange Version

Byt

es p

er tr

ansf

er

0

500

1000

1500

2000

2500

3000

ES

E C

ache

Siz

e (M

B)

Bytes per Read

Bytes per Write

ESE Cache

15

Read Write RatioExchange 2003 69% 31% ~2:1Exchange 2007 51% 49% ~1:1

Read/Write mix comparison

504

231175 168

0

100

200

300

400

500

600

Read IOPS Write IOPS

IOPs

Exchange 2003

Exchange 2007

ProLiant DL385 2 Dual-Core CPU (2.2GHz), 4GB RAM, 1500MMB3 users, U320 SCSI 24 DB disks, 4 Logs.ProLiant DL385 2 Dual-Core CPU (2.2GHz), 4GB RAM, 1500MMB3 users, U320 SCSI 24 DB disks, 4 Logs. Search/Indexing=OFF. Exchange 2007 beta – Search/Indexing=OFF. Exchange 2007 beta – results subject to changeresults subject to change

讀 / 寫的比率改變

16

• x64 與在暫存、 I/O運作的改變，在同樣 4GB 記憶體的伺服器上的效能改善• 從 4GB 8GB 的整體改善有 78% in IOPs/user(250MB

MB)64-bit I/O reduction impact

0.6

0.32

0.13

0

100

200

300

400

500

600

700

Exchange 2003 4GB Exchange 2007 4GB Exchange 2007 8GB

Version and Memory

IOP

s

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

IOP

S/u

ser

Reads/sec

Writes/sec

IOPs/u

ProLiant DL385 2 Dual-Core CPU (2.2GHz), 4GB RAM, 1500MMB3 users, U320 SCSI 24 DB disks, 4 Logs.ProLiant DL385 2 Dual-Core CPU (2.2GHz), 4GB RAM, 1500MMB3 users, U320 SCSI 24 DB disks, 4 Logs. Search/Indexing=OFF. Exchange 2007 beta – Search/Indexing=OFF. Exchange 2007 beta – results subject to changeresults subject to change

讀取的整體 IOPs/u 減少

17

Exchange 2003 RolesEnterprise networkEnterprise network

MailboMailboxx

Back-endBack-endServerServer

OtherOtherSMTPSMTP

ServersServers

Bridgehead / Bridgehead / GatewayGateway

RoutingRouting PolicyPolicy

Applications:Applications:OWAOWA

Protocols:Protocols:ActiveSync, POP, ActiveSync, POP,

IMAP, RPC / HTTP …IMAP, RPC / HTTP …Programmability:Programmability:Web services, Web Web services, Web

partsparts

Front-end Front-end ServerServer

IINNTTEERRNNEETT PublicPublic

FoldersFolders

18

Exchange 2007 RolesEnterprise networkEnterprise network

MailboMailboxx

MailboxMailboxServicesServices

EdgeEdgeServicesServicesOtherOther

SMTPSMTPServersServers

Hub TransportHub TransportServicesServices

RoutinRoutingg

HygienHygienee RoutingRouting PolicyPolicy

Voice Voice MessaginMessagin

gg

Unified Unified Messaging Messaging

ServicesServicesApplications:Applications:

OWAOWA

Protocols:Protocols:ActiveSync, POP, ActiveSync, POP,

IMAP, RPC / HTTP …IMAP, RPC / HTTP …Programmability:Programmability:Web services, Web Web services, Web

partsparts

Client AccessClient AccessServicesServices

PBX or PBX or VoIPVoIP

IINNTTEERRNNEETT PublicPublic

FoldersFolders

FaxFax

19

Exchange 2007 角色的硬體建議• 根據產品開發小組的設計目標建議• 請參考 blogs.msexchange.org 取得更多訊息

RoleTypical

Processor Config

Typical Memory Config

RoleExpected Processor

Config

Expected Memory Config

Gateway / Bridgehead

2 x Dual Core 2GB Edge/Hub

Transport2 x Dual Core 8GB

Front-end Server (FE)

2 x Dual Core 2GB CAS/UM

Server2 x Dual Core 8GB

BE Server/ PF Server

4 x Dual Core 4GB Mailbox

Server4 x Dual Core 32GB

Exchange 2003 Exchange 2007

20

64-bit 處理器• 目前絕大多數的伺服器組合都已經包含支援 64-bit 的 CPU• Exchange 2007 支援 x64 ，不支援 IA-64• x64 是 32-bit x86 的架構延伸

– Intel: EM64T (Extended Memory 64 Technology)– AMD: AMD64

x86 x64General Purpose

Registers 8 x 32-bit 16 x 64-bit

Physical Memory 36-bit (64GB) 40-bit (1TB)

Virtual Memory

Protected mode: 32-bit (4GB)2/2 or 3/1 GB for Windows user/kernel modesPAE: 36-bit (64GB)

48-bit (256TB)8TB / 8TB for Windows user/kernel modes

Instruction setIA-32Out-of-order execution

IA-32 compatibleOut-of-order execution

21

效能效能Dual-coreDual-core更快的記憶體存取更快的記憶體存取

更快的 更快的 FSBFSB更大、更快的 更大、更快的 L2 cacheL2 cache

更低的耗能更低的耗能Robust FeaturesRobust Features硬體支援的虛擬化技術硬體支援的虛擬化技術DEP - Data Execution Prevention /DEP - Data Execution Prevention /資料執行防止資料執行防止

比 GHz 更多的重點

221985

Clock speedClock speed

CacheCache

PowerPower(Voltage)(Voltage)

CoresCores

1993 1996 2002 2005 2007+

Intel386 DXIntel386 DX1 Freq/year1 Freq/year(16, 20, 25 & (16, 20, 25 & 33 MHz)33 MHz)

Intel Intel PentiumPentium2 2 Freqs/yearFreqs/year(66MHz – (66MHz – 233MHz)233MHz)

Cust

omer

Cho

ice P

oint

sCu

stom

er C

hoice

Poi

nts

Intel Intel Pentium ProPentium Pro2 cache2 cache256 & 512256 & 512

4 Freqs4 Freqs(150MHz – (150MHz – 200MHz200MHz))

Intel XeonIntel Xeon2xMP 2xMP cachescaches1xDP cache1xDP cache(1 – 4MB)(1 – 4MB)

3xMP Freqs3xMP Freqs6xDP Freqs6xDP Freqs(1.8GHz -2.8GHz)(1.8GHz -2.8GHz)

Intel XeonIntel Xeon5xMP Freqs5xMP Freqs9xDP Freqs9xDP Freqs(2.6 – 3.6GHz)(2.6 – 3.6GHz)

LV XeonLV Xeon

AMD OpteronAMD Opteron1 & 2 cores1 & 2 cores

LotsLots

Intel XeonIntel Xeon2xMP 2xMP cachescaches1xDP cache1xDP cache(1 – 8MB)(1 – 8MB)

MultipleMultiple

ManyMany

Lots of Lots of Skinny CoresSkinny Cores

Vs.Vs.FewerFewer

Fat CoresFat Cores

1 Choice1 Choice

2 Choices2 Choices

4 Choices4 Choices

n Choicesn Choices

4 & More 4 & More CoresCores

Hyper-ThreadingHyper-Threading25% improvement25% improvement

x64x64

ExchangeExchange4.04.0

5.0 – 5.55.0 – 5.5

20002000

20032003

20072007

Dual coreDual coreNearly 2x scaleNearly 2x scale

Microprocessor 的演進

23

Mailbox Server 的記憶體規畫• 大致需求 : 5MB per user + 2GB*

– 1,000 users: 8GB– 2,000 users: 12GB 16GB– 3,000 users: 17GB 24GB– 4,000 users: 22GB 24 – 32GB

• 新的 FB-DIMMs (完全緩衝記憶體模組 ) 提供更好擴展、更快速度與改善的 ECC

• 請參考將使用的伺服器規格資訊

* * Based on beta 1 software; subject to changeBased on beta 1 software; subject to change

24

Exchange 的 I/O 規劃The basics• 平衡三個要素

– 在一定延遲下的交易 I/O• Database: fast reads (<20ms still is the threshold)• Logs: fast writes (<10ms still is the threshold)

– 備份與還原的速度– 資料儲存與備份媒體的容量

• 資料擺放的大致規則– 分開交易紀錄檔與資料庫檔案

• 效能與災難復原的考量– 透過 SAN 來進行使用– 建議 RAID10； RAID5 適合容量小而快的磁碟

• 爆發性與隨機的 I/O 特徵– Monday morning peak

25

Disk IOPs

RAID 1+0 RAID 515K SCSI 150 100

10K SFF SAS 130 907.2K SATA 50 33

• 資料儲存不只是從兩個面向來考慮的問題• IOPs 的假設

– Latency: 20ms read / 10ms write

– 100% random I/O– 1:1 R/W

26

ProLiant DL385 2 Dual-Core CPU (2.2GHz), 4GB RAM, 1500MMB3 users, SAS 16 DB ProLiant DL385 2 Dual-Core CPU (2.2GHz), 4GB RAM, 1500MMB3 users, SAS 16 DB disks, 4 Logs. Search/Indexing=OFF. Exchange 2007 beta – results subject to change.disks, 4 Logs. Search/Indexing=OFF. Exchange 2007 beta – results subject to change.

RAID level comparison

0

100

200

300

400

500

600

700

800

RAID 10 RAID 5 RAID 6

RAID level

IOP

s / R

espo

nse

Tim

e

0

5

10

15

20

25

30

35

ms

Loadsim response (ms)

IOPs

Read Latency

RAID 的比較

27Rebuild priority = Low

Array Rebuild Times

0 1 2 3 4 5 6 7 8 9 10

U320 RAID 5

SAS RAID 5

U320 RAID 10

SAS RAID 10

Tech

nolo

gy

Time (hours)

RAID 重建• RAID 10 的重建影響單顆磁碟• RAID 5 的重建是磁碟組所有磁碟重建 parity

28

Logs DBs

Logs DBs

Q

Q Q

DB

DB

Logs

LogsLCRLCR CCRCCR

Continuous Replication • 資料的兩份複寫• 網路的影響規劃 (CCR)

– Database seeding– Baseline replication– Failover conditions

• I/O and CPU impact to-be-determined

29

NetworkStorage

ClusterSoftware FunctionsPhysical Cable

儲存建構的考量• I/O 的減少意指企業儲存建置的考量有更多選項

– iSCSI• Multi-function NICs provide performance enhancements,

CPU relief, iSCSI booting• Microsoft’s initiator integrated with MPIO, allowing for dual path

for high availability and load balancing

• Single fabric serves multiplefunctions

– Fibre Channel and storage virtualization– Local array controllers with SAS / SATA connectivity

30

網路的考量• Gigabit Ethernet 即將降臨• CCR Mailbox role

– Recommend multi-function Gigabit Ethernet with Jumbo Frames

– NIC Teaming recommended, particularly for database seeding, backup/restore and other heavy replication events

• Mailbox servers with network-based backup– Recommend NIC Teaming, multi-function NICs

• CAS and Hub Transport roles also require Gigabit Ethernet

31

準備 Exchange 2007 的環境• 記得 x64 架構• 很多的變數會改變

– 打破 CPU 瓶頸： Dual-core/Quad-core– 更仔細了解記憶體選項：平衡價位與效能

You *will* deploy more memory.– 儲存容量的計畫，尤其使用 LCR/CCR磁碟架構與技術的挑選更形重要– Gigabit Ethernet (multi-function NICs) and NIC teaming

• 當然有些重要的原則：– 切割伺服器角色來取得最大擴展性與還原能力– 分開擺放資料庫與交易紀錄檔– 分割 databases / storage groups 的使用

32

可以協助你的工具• System Center Capacity Planner 2006

– Provides CPU, Disk, Network, Server capacity planning based on a given user profile

– Exchange 2003/Outlook 2003 environments.

• Microsoft Exchange Server Profile Analyzer– Can be downloaded with an MSDN subscription

• Server / storage validation– DiY: Loadsim, ESP, JetStress– Microsoft + Vendor community: ESRP

33

講題總結講題總結• 轉用 x64 是必須的

– 記憶體延展性的突破– 外在環境的變動與使用者使用習慣的改變大幅增加 I/O 系統的壓力

• 更多的記憶體利用 = 減輕的 I/O 壓力– 53% in 4GB to 4GB ； 78% in 4GB to 8GB– 在磁碟陣列重建所需時間的更少影響– 資料讀取的 I/O 大量減輕 (larger ESE cache)

• 採用更大的信箱與更少的 I/O = 更多的儲存選擇與選項的前期計畫– 磁碟系統與架構的選擇依然是根基在平衡 ”容量“、” I/O” 、災難還原的需求上

34

Exchange ResourcesSystem Center 2006 Capacity Plannerhttp://www.microsoft.com/windowsserversystem/systemcenter/sccp/evaluation/overview/default.mspx

Microsoft Exchange Server Profile Analyzerhttp://www.microsoft.com/downloads/details.aspx?FamilyID=8f575f60-bd80-44aa-858b-a1f721108fad&DisplayLang=en

LoadSim http://www.microsoft.com/downloads/details.aspx?FamilyID=92EB2EDC-3433-47CA-A5F8-0483C7DDEA85&displaylang=en

JetStresshttp://www.microsoft.com/downloads/details.aspx?FamilyID=94b9810b-670e-433a-b5ef-b47054595e9c&DisplayLang=en

Exchange Stress and Performance Tool (ESP)http://www.microsoft.com/downloads/details.aspx?FamilyID=6e725211-cde4-49a8-bd3d-89c9ff1cc57a&DisplayLang=en

Exchange Solution Reviewed Program (ESRP)http://www.microsoft.com/technet/prodtechnol/exchange/2003/esrp.mspx

35

For More Information…For More Information…• TechNetTechNet

– www.microsoft.com/taiwan/technet• Exchange Server 2007Exchange Server 2007

– www.microsoft.com/exchangeexchange• TechNet TechNet 技術論壇技術論壇

– www.microsoft.com/taiwan/technet/forum• MVP CommunityMVP Community 社群網站社群網站

– www.microsoft.com/taiwan/community• MS Exchange Team BlogMS Exchange Team Blog

– http://msexchangeteam.com/• Exchange 2007 online HelpExchange 2007 online Help

– http://www.microsoft.com/technet/prodtechnol/http://www.microsoft.com/technet/prodtechnol/exchange/E2k7/ZH-TW/Help/ExchHelp/cb24ddb7-exchange/E2k7/ZH-TW/Help/ExchHelp/cb24ddb7-0659-4d9d-9057-52843f861ba8.mspx?mfr=true0659-4d9d-9057-52843f861ba8.mspx?mfr=true