TF-NOC flash presentation NIIF/Hungarnet NOC - TERENA · PDF filemultimedia ([email protected]) [email protected] 28 May 2013 . ... NIIF/HungarnetNOC-ﬂash-talk-Outline • NIIF Network

TF-NOC flash presentation NIIF/Hungarnet NOC

János Mohácsi – director of network and multimedia ([email protected]) [email protected] 28 May 2013

Nemze% Információs Infrastruktúra Fejlesztési Intézet

NIIF/Hungarnet NOC flash talk

Outline

• NIIF Network • NIIF/Hungarnet organisation • NOC structure and services • NOC tools and integration • Frontend • Communication tools • Documentation • Conclusion



NIIF Networks – member institutions •  ~5670 member institutions locations ~

1.5 million users •  ~3000 member institutions

  Universities, academic/research institutions   Primary and secondary schools   Public collections: libraries and museums   Some hospitals

•  Access speed from 512Kbps to 100 Gbps   601 monitored core devices   10799 monitored access devices

(data from 5/2013)

•  Hungarnet – bottom up association



HBONE+ backbone + Access

•  Infrastructure   DF (IRU)   Leased lambdas

4.

  Licensed ptp microwaves

  ADSL   DOCSIS Cable   MetroEthernet   Leased lines

•  Leased Services from 8 telcos



HBONE+ Network elements •  Optical backbone – lots of effort

  69 locations, ~3200km   Shared usage of fibre with HEC   Local loops

•  Countrywide DWDM system - Alcatel-Lucent PSS-1830   Optical services – from user to user   Most advanced 80 channel system with ROADMs – efficient

operation   10G, 40G and 100G services   Operated by NIIF NOC

•  Countrywide IP/MPLS system – Cisco ASR9000   MEF Carrier Ethernet/MPLS SP Platform   92 (184) Gbps slot capacity   „Green” system   Operated by NIIF NOC

•  IP devices   Mostly Cisco platform

5.



Hybrid Network Services

• Lightpath Service  Dedicated Network capacity for projects

  several demands   Point-to-point circuits – λoD

• L2 VPN, managed Ethernet - Fine-grained capacity provisioning – with routers

• Traditional IP: IPv4/6, unicast/multicast, L3 VPN, with QoS and SLA

6.



NIIF Organisation and NOC

• 8 departments (~60)   4 technical + financial + project +

partner relations + international relations

• ~26 services • NOC is part of Department of

Network Infrastructure   15 full-time member  ~ 15 supporting member at regional

centers (remote-hand) – 1-2 hours support/months



NOC organisation and services

•  Tier0: Call center – 24x7 – basic question – dispatching •  Tier1: Working Days 8-18h

  2T1 in the morning & 2 T1 in the afternoon •  Tier2: 24x7 – weekly shift – on duty engineer

  8-18 in case of emergency, 18-8+weekend - major maintenance, major failure

•  NOC staff (15+1)~ 5 T1, 2 T2, 5 T1&T2, 3 field, 1 boss •  Services:

  Helpdesk for institutions (Network, Multimedia services, application services)

  Connectivity management (NIIF Network, GEANT, BIX, private peerings)

  Monitoring and network maintenance   Network configuration (QoS, filtering, routing, IP allocation)   T2 members also planning and development



NOC tools •  Strategy

  Everything should use eduID (Shibboleth)   Use/develop opensource components

•  Tools:   [email protected] - heavily relying on e-mail communication + phone   Drupal based ticketing with federated AAI- http://drupal.org/project/ticketing   Integrated Information system – IIR – own development   DWDM PhM   Weathermap – modified http://www.network-weathermap.com/   Cricket – modified http://cricket.sourceforge.net/ - also for optical monitoring   Nagios with plenty of own developed plugins– modified http://www.nagios.org -migrating

to Icinga   Syslog-ng + Netcool for log analysis   Rancid http://www.shrubbery.net/rancid/ + routerCVS with cvsweb   Nfdump and NfSen - http://sourceforge.net/projects/nfsen/   Small scripts (perl, python, bash, php ): Integration, Reporting & Automation   FreeRadius and Cisco AR (phasing out)   Performance: NDT – network diagnostics, iperf   OOB   OTDR, power meter, cleaning tools, Ethernet traffic generators

•  Integration:   Web portal for NOC and for users with CRM portal



Ticketing system

Self made: drupal, shibboleth authen?ca?on, role based authorisa?on Communica?on: web and mail Used for tracking troubles, tasks, leased line crea?on/dismantling



Front end

•  What types of users are using your network and services?   end-users – individual users   NIIF member institutions – system administrators and

senior representatives   Outside partners   Other departments

•  How does the SLAs or agreements you have with your customers look like?   SLA in the agreements – 99.5%, but achieved 99.97%   Membership type organisation – discussion/collective

work

•  Which tools are used to communicate with and keep track of users?



Frontend /2



Frontend /2



Frontend /2



Communication tools

•  To communicate with and keep track of users   E-mail, phone, videoconference, monthly meeting   Trouble ticketing system – user can submit and keep track of

tickets   IIR

•  Other groups   e-mail, phone   Issue tracking systems (redmine)   IIR   Trouble ticketing system   Instant messaging

•  Outside NIIF   e-mail, phone   IIR   Trouble ticketing system + external TT



IIR informations

• Data about the end-point   Location  Responsible persons  Devices - contracts   Integrated with existing systems -

Cricket, nagios, ticketing

16. oldal



IIR informations



17. oldal



IIR informations



18. oldal



Documentation

•  Documented information – mostly hungarian   Service descriptions   Procedures   Advices, hints   Modifications   Config procedures   Contacts

•  Tools   Moinmoin wiki   IIR   Word document

•  Do you have any Best Practice Documents available to share?   ?



THANKS

?