Storage System Lab SKKU 0 SAN Management and Application 성균관대학교 저장시스템 연구실 윤 희용 2001. 9. 11 21 세기 유망핵심부품 기술 세미나

1Storage System Lab SKKU

SAN Management andApplication

성균관대학교저장시스템 연구실

윤 희용

2001. 9. 11

21 세기 유망핵심부품 기술 세미나


Outline

• SAN Architecture and Model• SAN Technology Trends• SAN Management Architecture• SAN Applications• SAN File System• SAN Standardization• SAN Hot Issues• Conclusion


SAN Architecture

SANManagement

SANInterface

SAN Hardware BaseStorage

Extract the value of data• Backup / Recovery• Outboard data movement• Clustering, data sharing

Zero latency, one-touch management• Centralized local point• Reduced cost• Flexibility and automation

Migration paths• Investment protection• Fibre channel• Distance, speed, reliability• Hubs, switches• Any storage to any system

SAN-enabled• Native attachments• SAN-aware

SAN from initiative to implementation• Reduced time & cost to implement• Skilled resources• SAN integration, testing, & verification

Serv

ices

and

Sup

port

Solu

tions

SAN Application


Storage Model

Sto

rage d

om

ain

File/record layerFile/record layer

Block layerBlock layer

Storage devices (disks, …)Storage devices (disks, …)

Serv

ices

Serv

ices

Dis

cove

ry, m

onit

orin

gD

isco

very

, mon

itor

ing

Res

ourc

e m

gmt,

conf

igur

atio

nR

esou

rce

mgm

t, co

nfig

urat

ion

Secu

rity

, bil

ling

Secu

rity

, bil

ling

Red

unda

ncy

mgm

t (ba

ckup

, …)

Red

unda

ncy

mgm

t (ba

ckup

, …)

Hig

h av

aila

bili

ty (

fail

-ove

r, …

)H

igh

avai

labi

lity

(fa

il-o

ver,

…)

Cap

acit

y pl

anni

ngC

apac

ity

plan

ning

Database(dbms)

File system(fs)

Network

Host

DeviceBlock aggregation

Application


Storage Model – Full function View

Hos

t. w

ith L

VMan

d so

ftwar

e R

AID

1. Direct-attachF

ile/r

ecor

dla

yer

Blo

ckla

yer

Device block-aggregation

SN block-aggregation

Host block-aggregation

NAS head

4. NAS head

2. SN-attach

Hos

t. w

ith L

VM

Disk array

SN

5. SN aggregation

Aggregationappliance

Blockmetadata

server

File metadata

server

6. Metadata aggr’n

Application3. NAS server

LAN

HostHost

NASserver


Storage Model – Multi-site Block Storage

Disk array

SNAggregation

appliance

Hos

t. w

ith L

VMApplication

Disk array

Aggregationappliance

Hos

t. w

ith L

VM

Application

SN appliancesSN appliances

WAN

WAN

Host-to-hostHost-to-host

Functions: point-in-timecopy, caching, local & remote mirroring, …

Functions: point-in-timecopy, caching, local & remote mirroring, …

WAN

Device-to-deviceDevice-to-device

Device block-aggregation

SN block-aggregation

Host block-aggregation


Why Networked Storage?

Doubling every 8-9 months

opticalnetworks

speed Doubling every 12 months

storagecapacity

Doubling every 18 months

computingpower Moore’s Law


Storage and Network Convergence

IP Networking

IP Networking

External StorageExternal Storage

Optical Networking

Optical Networking

StorageStorageNetworkingNetworking


Technology Trend – Fibre Channel

• Fibre Channel provides new levels of Bandwidth, Connectivity, and Flexibility

–Switches provide non-blocking 100 MB/sec (1 Gbit/sec) per port with any-to-any connectivity

–Long-wave ports provide up to 10 km distance

• Future Developments:–Large port-count switches (~ 256)

–Fibre Channel has grown to 2 Gbit/sec

10 Gbit/sec in 2-3 years


SAN Connectivity Evolution

DATA CENTER

ESCON / BUS & TAG

1990 1998

FCAL

FC-AL

HUB

SWITCH

FC-SW

FC-SW

1999

1-11 KM

CAMPUS

REMOTE

1994

SCSI25M

1997

FC-AL (Point-

to-Point) 500M

2001

FC-SW

SWITCHMA

N 100K

mDWDM

SWITCH

FC -SW

Global IP-SAN

1,000+ Km

FC-SW

FC / ISCS

I

2002+

SWITCH

FC -SW


Conventional IT Organization

Network used for exchange of data between clients and servers and between servers themselves.

Difficult to manage, wasted resources, often unprotected: Impossible to scaleDifficult to manage, wasted resources, often unprotected: Impossible to scale

Heterogeneous Servers

Client Network

Clients

LAN

UNIX NTMVS


IT Organization with Enterprise Storage

Network used for exchange of data between clients and servers and between servers themselves.Heterogeneous

Servers

Client Network

Clients

LAN

UNIX NTMVS

Enterprise StorageEnterprise Storage

Enterprise Storage provides storage and exchange of data by servers

Simpler to manage, resources easily re-deployed, superior protectionSimpler to manage, resources easily re-deployed, superior protection.

The next stage of manageability and scalabilityThe next stage of manageability and scalability


IT Organization SAN

Data Logistics

Information MoveBackup/RestoreInformation MgmtCluster Support

Data Presentation

Information visualization using HTML, Java, etc.

Storage Area Storage Area NetworkNetwork

Heterogeneous Servers

Client Network

Clients

LAN

UNIX NTMVS

NAS Appliance


Why Networked Storage?

• Consolidation - Application Server and Storage– Storage Consolidation

• Management - Total Cost of Ownership (TCO)• Disaster recovery - Backup / Restore

• Availability – 7 x 24 x ForeverForever

• Flexibility – Rapid adaptation of changes in business

requirements• Capacity Expansion• Performance• Configuration


Shift in Customer Perceptions

1) IT saves dollars IT contributes to revenueIT contributes to revenue growthgrowth

2) Choose technology first Put information needs firstPut information needs first

3) Managing technology Managing informationManaging information

4) Acquisition costs Total cost of ownershipTotal cost of ownership

5) Rigid infrastructure Change-tolerant infrastructureChange-tolerant infrastructure Technology / Vendor lock-in Flexibility, compatibilityFlexibility, compatibility


SNIA Management Standards - 1999

– Common Information Model (CIM)

– Simple Network Management Protocol- Management Information Bases (SNMP-MIB)

• Reviewed and assisted in the standardization of two SNMP MIBS

– Fibre Alliance MIB in process within IETF

– Creation of new MIB for Storage Libraries (First MIB to use CIM schema)

• SNIA CIM schema definitions submitted for CIM V2.2

– Storage Media Library (SML WG)

– Disk Resource Management (DRM WG)

– Fibre Channel (FC WG)• October 1999 Storage Networking WorldFirst demonstration of V2.2 CIM web based technology


SNIA Management Standards – 1st half 2000

• Education– Release of the first Storage Networking Dictionary to

the industry– Successful completion of three major industry events:

• The SNIA workgroup conference - Feb 2000 • Spring Storage Networking World - April 2000• CIFS 2000 Conference and Interoperability Lab - May

2000

• Management – Delivery of Industry first Open Source - CIM-OM (CIM-

Object Manager)

• Disaster Recovery - Backup – SCSI-3 extended copy command

• Submitted to ANSI T10

– Driving the NDMP version 4 development


SNIA Management Standards – 2nd half 2000 • Education

– The SNIA workgroup Symposium - Sept. 2000 – Vendor and IT Consumer Surveys – Fall Storage Networking World 2000– Architectural white papers

• Interoperability– Continued Standardization of CIFS (Common Internet File System)– Test plans and reference implementations for new and existing standar

ds• Management

– Adoption of the SNIA developed HBA-API by T11 – Management models for extended copy sessions– MIBs and extensions to the CIM Schema for storage management

• Disaster Recovery– APIs for snapshot and checkpoint


SNIA Management Standards – 2001

• Grand Opening of the SNIA Technology Center Jan ‘01

• SNIA-Japan - SNW Japan Jan. ‘01• SNIA Shared Storage Model March ‘01 • Spring Storage Networking World April

‘01• Spring SNIA Work Group Symposium

April ‘01• Summer SNIA Work Group Symposium

June ‘01• SNIA CIFS Conference Aug ‘01 • European Storage Networking World

Sept. ‘01 • Fall Storage Networking World USA

Oct. ‘01


SAN Management Architecture


SAN Management Architecture – EMS & CMS

• EMS☞ SNMP based service☞ DMI based service☞ FC service☞ SES service☞ Native CIM service

• CMS☞ Entity mapping☞ Common data model☞ Access services


SAN Management Open Standard Platform

• SCSI Instruction Set• SCSI Enclosure Services(SES)• SCSI Self-monitoring Analysis and Reporting Technology (SMART)• SCSI Accessed Fault-Tolerant Enclosure(SAF-TE)• SNMP• Web-based Enterprise Management(WBEM)


SAN Management

Storage Storage ManagementManagement

ProblemProblemSAN SolutionSAN Solution

Time for data backup

Improvement of backup speedthrough bandwidth and protocol efficiency

Impossibility of remote backup, mirror, or recovery

Support remote operation through a cable of about 10km

Insufficient solution for local backup and mirroring

Ideal platform for hierarchical storage management(HSM)

Exhaustion of network bandwidth by LAN connection for server backup

Decrease LAN traffic by separating server-storage from LAN connection


SAN Application

ClusterinClusteringg

Data Data ProtectioProtectio

nn

DataDataVaultingVaulting

DataDataExchangeExchange

ReplicateReplicateRecoveryRecovery

NetworkNetworkArchitecturArchitectur

eeSANSAN


SAN Application - Remote Backup

SA N SA NL A N SA N L A N

S ite A

S ite C

S ite B

10km 10km


SAN Application - Server-Storage Clustering

SA N

C lus ter B

SA N

C lus ter A

SA N

C lus ter C

L A N


SAN Serverless Backup

Client ClientClient

Diskstorage

Tape

LAN

Net serverBackup server

SAN

Net server


SAN Storage Integration

Client Client

Diskstorage

Tape

LAN

Net server

SAN

Net serverNet server

Diskstorage

Diskstorage


SAN Storage Expansion FC-AL


SAN Storage Expansion Fabric

server

FC host 어댑터


SAN Storage Expansion Multiple Loops

Loop #1

Loop #2

Loop #3

Loop #4

Loop #5

Individual loopSerially connected


SAN FC Workgroup

workstation

FC-AL w/IP

FC-AL

w/SCSI

NT file server

Shared storage


SAN Recovery – Server Clustering

server

FC switch/hub

JBOD or RAID


SAN Recovery – HA clustering


SAN Recovery – Data Mirroring


SAN File System

• SAN Support • Customizability

☞ According to specific I/O requirement such as web, multimedia• User-Controllability

☞ Users decide logical storage space, data format, fault tolerance, and recovery

• High Performance and Scalability☞ Dynamic load balancing across all disks☞ Hot system mode change☞ Storage environment optimization according to I/O requirement

• High Availability☞ Software RAID, Chained-Declustering, Fast Recovery

• Compatibility withexisting file system in API and access method


SAN File System Requirement

• Mass Storageability☞ Unlimited

• High Scalability☞ Capacity, performance, connectivity, bandwidth, cost

• High Availability☞ Duplex☞ Online data backup

• Easy Manageability☞ Effective central management and remote control through Internet

• High Utilizability☞ Logical disk pool

• Load Balanceability☞ Data migration through SAN


Application Software (Oracle, Word, Web Browser, etc.)Application Software (Oracle, Word, Web Browser, etc.)

Hardware (Disk, Memory, CPU, Network Cards, etc.) Hardware (Disk, Memory, CPU, Network Cards, etc.)

Kernel

System Call Interface

Networking

Process Management

Memory Management

Device Driver

IPCTimer Function Ext2 I/FNew I/F

Buffer Cache

Virtual File System

MP

I-IO

NT

FS Ext

FS

HFS

SA

N

FS

New Driver

SAN File System Structure


SAN File System Products

YesYesYesYesHigh Availability

YesNoNoNoMPI-IO

YesNoNoNoLinux version

FS ShareSAN Support

I/O Perf

SAN Support

FS ShareSAN

Support

SAN Support

Key Functionality

YesYesYesYesDisk share

YesYesYesYesSAN Support

PossiblePossibleNoPossibleParallel I/O

HighLowNoLowTuning

NewNativeNTFS/UFSNativeFile System

MultiplatformUnix/NT/VMSUnix/NTUnixO S

TargetSoftWorksIBM TivoliVeritasItem


Standard Collaboration

• DMTFDMTF – WBEM (Web-Based Enterprise Mgmt), CIM

• ANSIANSI – T10 (SCSI) and T11 (Fibre Channel)• IEEEIEEE – Media management and tape• IETFIETF – SNMP, IP SANs, IP storage,directories/L

DAP and policy • And others

– FCIAFCIA – Fibre Channel Industry Association– Fibre AllianceFibre Alliance - SNMP MIB


SAN Standardization - FC

• 1994 ANSI X3T11

IPI-3

Inst mapping

(IPI-3 STD)

SCSI-3

Inst mapping

FC link

Encapsulation

FC-LE

FC-ATM

SCSI-3 IP ATM

Common service

FC

Physical/signaling

Interface

(FC-PH,FC-PH2, FC-PH3)

FC-AL FC-AL-2

ULP(Upper Level Protocol)

FC-4

FC-3

FC-2

FC-1

FC-0

Frame protocolEn/decoding

Physical

8B/10B Coding

Copper/Fibre


SAN – Hot Issues

• Storage over Wide Area Networks• SANs for Everyone• Storage Services Providers• Storage Management Applications• Extended Copy for server-free data movement• Host Adapter API for fibre channel management• IP Storage technologies• Fibre channel switch interoperability • Infiniband


SAN over WAN


SAN for Everyone


Storage Services Providers


Storage Management


IP Storage


Switch Interoperability


InfiniBand


PCI vs IB

Characteristics PCI 2.2 PCI-X 1.0 IB

Speed 66MHz 133MHz/slot 2.5Gbps

Data width 32/64 bit 32/64 bit 1x, 4x, 12x

Max bandwidth 533Mbyte/sec 1,064Mbyte/sec 2.5Gbps/port

Connection Shared bus Shared bus Point-to-point

Max nodes Less than 10 1/133MHz 216/subnet

Communication Shared memory Shared memoryChannel-based

Message passing


Conclusion - Trends

The Next Big ThingThe Next Big Thing

20032003

20012001

20042004

20022002

• Focus on the problem new technologies may solve/displace– I/O bus vs. IP network vs. SCSI-based channel

network• Most of these new technology specifications

have not been defined, much less adopted as “standards” 10Gb FC

10Gb FCDAFSDAFS2Gb FC

2Gb FC

10Gb Ethernet

10Gb Ethernet

InfiniBand

InfiniBand

SCSI over IP

SCSI over IP

EtherStorage

EtherStorage

iSCSIiSCSI

Storage over IP

Storage over IP


SNIA Vision

SecuritySecurityOBSDOBSD

PolicyPolicyNASNAS

DiscoveryDiscovery

StorageStorageMediaMedia

LibraryLibrary

FileFileSystemsSystems

DRMDRM

BackupBackup

FibreFibre ChannelChannel

WebWebDevelop-Develop-

mentment

MarketingMarketing

EducationEducation

FinanceFinance

InteroperabilityInteroperability

MembershipMembershipDevelopmentDevelopment

ConferenceConferenceDevelopmentDevelopment

StrategicStrategicPlanningPlanning Technology Technology

CouncilCouncil11 members11 members


SAN Milestone

Block I/O NAS Object

As technology marches forward… …SNIA will address practical storage networking concerns

2001 2002 2003 2004 2005

Access Models…

Changing!

RequiredStorage

Capacity…Doubling!

IT Concerns UsageModels

Goals SNIATechnicalActivities

FC T

unne

ling

Ove

r LAN

/WAN

Fibr

e Ch

anne

l

iSCS

I

Infini

Band

InfrastructureComplexity…Increasing!

SCSI

STANDARDS MANAGEMENT SECURITY AVAILABILITY

PERFORMANCE

DATA PROTECTION

INTEROPERABILITY

Documents

Storage System Lab SKKU 0 SAN Management and Application 성균관대학교 저장시스템 연구실 윤 희용 2001. 9. 11 21 세기 유망핵심부품 기술 세미나