36
CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team, Steve Traylen (IT-PES), Matthias Schröder (IT-OIS), Michał Kwiatek (IT- OIS)

CERN IT Department CH-1211 Genève 23 Switzerland IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

Embed Size (px)

Citation preview

Page 1: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 2011

Software and Hardware Inventory Initiatives

Computer Security Team, Steve Traylen (IT-PES),Matthias Schröder (IT-OIS), Michał Kwiatek (IT-OIS)

Page 2: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 20112

Agenda:– Goals and motivation– Computer Security background

– Linux desktops– Quattor-managed Linux Clusters– Mac desktops– Windows computers

– Feedback

Software and Hardware Inventory Initiatives

Page 3: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 20113

Software and Hardware Inventory Initiatives

Goals:• Monitor the state and evolution

of computers on the CERN site– Software and Hardware– Mac, Linux and Windows– Computer Centre and Personal Computers

Page 4: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 20114

Motivation:• Efficient Service Management– Ease software deployment– Precondition for Change Management

• Ease User Support– Provide tools to Service Desk

• Protect computers from security risks– Improve (automate) our insight in software

vulnerabilities across CERN– Keep computers up to date– Promptly respond to new threats

Software and Hardware Inventory Initiatives

Page 5: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 2011

Timely updating and patching is our 1st line of protection!

Computer Security Team

Software and Hardware Inventory Initiatives

Page 6: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 20116

Background• Any unprotected/unpatched/outdated computer connected to the

Internet is likely to be infected within minutes!• From OC5: “The user shall take the necessary precautions to protect

his personal computer or work station against unauthorized access.”• Timely updating and patching is the 1st line of protection!• This applies for MS Windows, but also to Linux and Macs. • Worse: Attacks are moving away from the O/S and targeting now

the application level.

• A central patch monitoring portal allows every user and service manager (as well as the Security Team ) to understand the security posture of their computer and servers.

• Areas for improvement and vulnerable computer/servers can be spotted in real-time and the corresponding user/manager can be quickly informed and asked for mitigation.

Computer Security Background

Page 7: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 2011

Linux Desktops

Matthias Schröder (IT-OIS)

OS Patch Deployment Monitoring

Page 8: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 20118

Background

• About active 4k nodes on site• Automatic updates enabled by default– But easy to disable…– Kernel updates require reboot– Conflicts can block updates

• Basic configuration done via lcm– Ncm-components and local profiles– Relies on SW updates for changes

• No further central management• No central backups

Scientific Linux Desktops

Page 9: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 20119

Current situation

• OCS-inventory– Open source inventory software– Available for Mac, Linux, Windows and more– Data collectors running on clients

• Little load on client• Available for many OS• Configured via ncm-component

– Reporting to central server• Hardware of nodes• Installed software• Running kernel• Keeps only snapshot• User activity is not reported

– Installed on all updating nodes

Scientific Linux Desktops

Page 10: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 201110

OCS host listing

Scientific Linux Desktops

Page 11: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 201111

OCS Summary Example

Scientific Linux Desktops

Page 12: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 201112

OCS Node Info Example

Scientific Linux Desktops

Page 13: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 201113

Future steps

• Deployment started spring 2011• Next:– Develop queries for data mining– Extend CERN specific info

Scientific Linux Desktops

Page 14: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 2011

Quattor-managed Linux Clusters

Steve Traylen (IT-PES)

Software and Hardware Inventory Initiatives

Page 15: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 201115

Quattor Managed Background

• CERN CC contains quattor configured hosts:– SLC4 : SLC5 : SLC6 = 301 : 7375 : 32– RHEL4 : RHEL5 : RHEL6 = 242 : 283 : 3

• Managed as 117unique clusters.– Each cluster is pinned to an SLC snapshot date.

• e.g OSDATE=20110523.

– Each cluster has it’s own package update policy. – Today time range of OSes are > 1 year.

• Quattor configuration only prescriptive.– It does what you ask, no matter what.

Quattor-managed Linux Clusters

Page 16: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 201116

Quattor Current situation• OSDATE Monitoring of CDB Clusters

– Monthly email sent per cluster to each IT-Contact.– e.g lxplus:

Cluster: lxplusMinimum OSDATE within lxplus is 201106XXMost frequently occurring OSDATE within lxplus is 201106XX

Of a total 117 clusters lxplus is calculated asnumber 13 in the ordered list of most upto date clusters.

• This monitors configuration only not reality.– This monitoring is very imprecise, reality may be worse.

• General details on the OSDATE mechanism: http://twiki.cern.ch/twiki/bin/view/ELFms/OsUpdates

Quattor-managed Linux Clusters

Page 17: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 201117

Quattor Managed Future steps

• Package Level Inventory– We need to know what is installed.

• For both security and operational reasons.

– Results to be cluster neutral and correlated with RedHat CVE guidelines.• Traditionally Pakiti has been solution.• Pakiti produces a list of outstanding CVEs per node.

• OCSagents are being deployed across CC.– OCSagents collect everything Pakiti needs.

• An OCS collector can be added to report limited CDB data.– e.g cluster name, clustersub name.– Allow joins of OCS to existing DBs: CDB, SDB, ….

Quattor-managed Linux Clusters

Page 18: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 201118

Quattor Managed Future steps

• Run Pakiti engine on extracted results of OCS database.– Pakiti client itself dropped, a duplication of collection.

• Web Interface for Pakiti results:– Views needed for security team and cluster managers.– Evaluate if Pakiti web-interface can be used or adapted.

• Early attempts were unusable, batch deluge results.

– Evaluate if an existing CERN aware web-interface can be adapted to pakiti results.• e.g. cluman, desktop DB (see later).

– Create a new web-interface which is e-group, cdb cluster aware.• Monthly Report

– A monthly report of CVEs per cluster can be generated.• Quattor and non-managed will be treated equally.

– Pakiti results for SLC desktops will also be available.

Quattor-managed Linux Clusters

Page 19: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 2011

Mac Desktops

Matthias Schröder (IT-OIS)

OS Patch Deployment Monitoring

Page 20: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 201120

Background

• About 2k active clients on site• System and main apps check for updates– But users can de-activate this– Users only reminded that updates available

• No central management• No central configuration• No central back-ups

Mac Desktops

Page 21: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 201121

Current situation

• K2 to monitor usage of licensed SW– Only on nodes using licensed SW– Rather complete monitoring• Hardware• Software• Can monitor usage of selected SW

– Requires license per node

Mac Desktops

Page 22: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 201122

K2 Node List

Mac Desktops

Page 23: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 201123

K2 Licence Information

Mac Desktops

Page 24: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 201124

K2 Software List

Mac Desktops

Page 25: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 201125

Future steps

• Plan to install OCS Inventory on all nodes– Gradual process

• Share OCS Server with Linux• Need to keep K2 for licensed SW

Mac Desktops

Page 26: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 2011

Windows Computers

Michal Kwiatek (IT-OIS)

Software and Hardware Inventory Initiatives

Page 27: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 201127

Windows Background

• Windows computers at CERN:– 6000 Centrally Managed– 1500 Locally Managed– 1500 not in the CERN domain

Windows Computers

Not in the

CERN DomainIn the

CERN Domain

Managed

Centrally

Loca

lly

Page 28: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 201128

Windows Background

• Windows computers that belong to the CERN domain are managed with CMF

• CMF enables:– Deployment of the desired software

configuration, incuding patches– When necessary, delegation of software

deployment tasks to Local Administrators (ex. Experiments, Controls)

– Reporting of the actual configuration of Windows Computers• Requires manual configuration for unsupported apps

Windows Computers

Page 29: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 201129

Windows Background

• Every day, we actively assess the risk of security exploits of CERN computers

Windows Computers

History of computers reinstalled because of detected security problems (per week)

Page 30: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 201130

Windows Background

• To manage software lifecycle, we must understand configurations across CERN

Windows Computers

Page 31: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 201131

Windows Current Situation

• 6000 Centrally Managed PCs and Servers– Monthly deployment of patches

for OS and supported applications– Email alerts for owners of computers running unsupported

applications with known security vulnerabilities

• 1500 Locally Managed computers– Monthly recomendation to Local Admins concerning patch

deployment– Email alerts for Local Admins when their computers run a

configuration with a known security flaw (ex. unsupported OS, no Antivirus)

• 1500 computers which are not in the CERN domain– Computers belonging to short-term visitors, managed by

their respective owners (IT has no control)Windows Computers

Page 32: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 201132

Windows Current Situation

• Microsoft patch deployment follow-up

Windows Computers

Page 33: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 201133

Windows Current Situation

• Follow-up for unsupported applications

Windows Computers

Page 34: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 2011

Future Steps• DesktopDB– Initially designed to keep history

of desktop configurations across all OS– Now extended to quattor-managed clusters

in the Computer Centre

34Windows Computers

CMF OCSDesktopDB

Page 35: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 201135

Future Steps

• DesktopDB– Evolution of SW and HW configurations– Across all OS: Windows, Mac and Linux• Including Quattor-managed Linux Clusters

– Prototype for• ITIL CMDB data source• Service Desk tool

Page 36: CERN IT Department CH-1211 Genève 23 Switzerland  IT Forum, June 2011 Software and Hardware Inventory Initiatives Computer Security Team,

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/it

IT Forum, June 2011

Feedback?

Software and Hardware Inventory Initiatives