02/24/09 2
Agenda
• The Data Center Status Quo• Future State Goals• Our NYSERDA Advanced Concepts Datacenter proposal
02/24/09 3
The Data Center Status Quo
• Architectural– Built in 1963, updated somewhat in the 1980's.– 5000 sf (465 m^2) raised floor machine room space.– 1750 sf (163 m^2) raised floor space, now offices.– 12” raised floor– Adequate support spaces nearby
• Staff• Staging• Storage• Mechanical & fire suppression
02/24/09 4
The Data Center Status Quo (cont'd)
• Electrical– Supply: 3-phase 208V from automatic
transfer switch.– Distribution: 208V to wall-mounted panels;
120V to most servers.– No central UPS; lots of rack-mounted units.– Generator: 1750 kW shared with other
users.• At capacity.
– No metering. (Spot readings every decade or so:-)
– IT demand load tripled from 2001-2008• 2001: 107 kVA• 2008: 335 kVA
02/24/09 5
The Data Center Status Quo (cont'd)
• Mechanical– On floor CRAC units served by campus
chilled water.– Also served by backup glycol dry coolers.– Supplements a central overhead chilled air
system.– Heat load is shared between the overhead
and CRAC.– No hot/cold aisles– Rows are in various orientations– Due to tripling of demand load, the backup
(generator-powered) CRAC units lack sufficient capacity.
02/24/09 6
The Data Center Status Quo (cont'd)
• IT systems– A mix of mostly administrative (non-
research) systems.– Most servers dual-corded 120V
power input.– Many ancient servers.– Due to lack of room UPS, each rack
has UPSes taking up 30-40% of the space.
– Lots of spaghetti in the racks and under the floor.
02/24/09 7
The Data Center Status Quo (cont'd)
• Other data centers– Many small school, departmental &
research server rooms all over the place.– Most lack electrical or HVAC backup.– Many could be better used as academic
space (labs, offices, classrooms).– Growth in research HPC putting
increasing pressure on these server rooms.
– Lots of wasted money building new server rooms for HPC clusters that are part of faculty startup packages, etc.
02/24/09 8
Future State Goals – Next 5 years
• Begin phased upgrades of the Data Center to Improve power and space efficiency. Overall cost ~ $20M.
• Consolidate and replace pizza box servers with blades (& virtualization).
• Consolidate and simplify storage systems.• Accommodate growing demand for HPC research clusters• Accommodate server needs of new Interdisciplinary
Science Building.• Develop internal cloud services.• Explore external cloud services.
02/24/09 9
Future State Goals – Next 10 years
• Build a new data center of 10,000-15,000 sf• Consolidate many small server rooms.• Significant use of green-energy cloud
computing resources.
From www.jiminypeak.com
02/24/09 10
Our NYSERDA Proposal
• New York State Energy Research & Development Authority Program Opportunity Notice 1206
• ~$1M ($447K from NYSERDA awarded pending contract)• Improve space & power efficiency of primarily
administrative servers.• Contribute to Columbia's PlaNYC carbon footprint
reduction goal.• Make room for shared research computing in the existing
data center.• Measure and test vendor claims of energy efficiency
improvements.
02/24/09 11
Our NYSERDA Proposal – Specific Tasks
• Identify 30 old servers to replace.• Instrument server power consumption and data center
heat load. “Real time” with SNMP.• Establish PUE profile (use DoE DC Pro survey tool)• Implement 9 racks of high-density cooling (in-row/rack).• Implement proper UPS and high-voltage distribution.• Compare old & new research clusters' power consumption
for the same workload.• Implement advanced server power management and
measure improvements.• Review with internal, external and research faculty
advisory groups.• Communicate results.