Embed Size (px)
Citation preview
eCensus 2011: A case studyIBM Smarter Planet – eCensus
Rod Hennegan Lead Architect eCensus 2011 & 2006Global Business ServicesIBM Australia
eCensus 2006 awards
2
Mass Scale Internet Event
• Event– eCensus 5 weeks– Australian open 2 weeks
• Mass scale – 2011 eCensus
• 4.8 million users (50% of dwellings in Australia)• Peak (Census night)
– 2011 Australian Open• 10 Million users• Peak (final)
% of total submissions per day
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
27/0
7/20
06
28/0
7/20
06
29/0
7/20
06
30/0
7/20
06
31/0
7/20
06
1/08
/200
6
2/08
/200
6
3/08
/200
6
4/08
/200
6
5/08
/200
6
6/08
/200
6
7/08
/200
6
8/08
/200
6
9/08
/200
6
10/0
8/20
06
11/0
8/20
06
12/0
8/20
06
13/0
8/20
06
14/0
8/20
06
15/0
8/20
06
16/0
8/20
06
17/0
8/20
06
18/0
8/20
06
19/0
8/20
06
20/0
8/20
06
21/0
8/20
06
22/0
8/20
06
23/0
8/20
06
24/0
8/20
06
25/0
8/20
06
26/0
8/20
06
27/0
8/20
06
28/0
8/20
06
3
Australian Open/Olympics
eCensus
Internet
eCensus
Open/Olympics
Fan, Viewer
eCensus Respondent
Internet
Servers xN
Servers xN
Caching layer
eCensus and Open/Olympics• eCensus
– Collection system– Private data
• Tennis Open/Olympics– Dissemination– Public data (Broadcast)
Public data
SSL encrypted tunnel
Private data
15 1
4
Census night - very public event
• Census night– High visibility (Census night)– Peak load period
• System outage on Census night– Shhhhh, maybe no-one will notice – ABS and IBM reputations
• Key areas– Hardware/Software/System failures– Malicious attack– Human error– Impact on other systems/clients
“Call me cynical but I was just waiting for eCensus to fall over in a pathetic heap….” Personal BLOG of
Computer World JournalistRod Gedda 9 Aug 2006
5
The ABS – IBM relationshipABS
(Data analysis & dissemination)• Contract
– Requirements & Questions– User Acceptance testing– 3rd Party testing
• Usability • Security (application & hosting)• Performance• Accessibility (Vision Aust.)
– Manage event• Field force management• Supervision of IBM access
IBM(Data collection & delivery)
• Contract– Application development– Create hosting environment– Testing
• System testing• Usability• Web security testing• Performance & Availability
– Hosting the event• Operations management• Security operations
6
Security/Privacy of Census data
• Aust. Government (PSM/ISM)• Census and Statistics Act 1905
– Encryption on all network links– Employed under the act to access raw census data
• Encrypted when written to persistent storage (database)– Custom security solution (Hardware security modules)– Decrypted (visible) only to
» ABS after delivery» Respondent (application processing a respondent request)
– ABS legal review (post 2006)• Access under the act = plain text, encrypted, or physical• No IBM Staff access to privileged accounts
– Except under direct ABS supervision– Supervision process remote supervision of remote staff
7
Security/Privacy of Census data
• Enforcement– Application source reviewed 3rd party
• Ensure implementation of these mechanisms• Prove production version = code reviewed
– Extensive cutover process• All privileged accounts passed to ABS• Secure racks locked by ABS
– Post production data deletion process• IBM still responsible operations
– Extensive monitoring and automated failover– Automation of required functions (stop/start etc)– Supervised access
8
eCensus application
• Web Browser based – why ?– Multi-platform
• Run on browser on virtually any operating system– Removes install process support– Zero footprint solution
• No plug-ins, no requirement to install Java, Active X controls
• eCensus implementation– Two distinct user interfaces optimised for
• User experience• Accessibility – (W3C Accessibility AA)
9
eCensus application
• Implementation approaches– Use a Web Forms product
• XForms standards based (Others like Adobe forms)• Pixel perfect replication of a paper form
– Custom Web application• Build and present web pages containing questions• 2006 eCensus solution• Stateless application (0 MB per session at server)
– Generic Questionnaire engine• Questionnaire definition in XML• Application interprets definition• Requires 1/3 server hardware of 2006 solution
10
Generic Questionnaire engine
• XML definition (Abstraction of a questionnaire)– Questionnaire –> Forms –> Questions -> Responses– Question response types (Text, Date, Address, Selection)– Question sequencing (Conditions based on responses)– Use of response content– Define business rules for responses/across questions– Tabular presentation for Text and Select questions– Consistent presentation through use of CSS
• Support virtually any internet questionnaire– Population Census, Agricultural Census– ABS strategic surveying solution
11
Using the eCensus intellectual property
• How can we utilise the eCensus technology – ABS provides surveying services– eCensus application and technology
• eCensus is not a product• ABS - rights to the IP for the application
– Contact Paul Lowe ([email protected])• IBM - eCensus application & all related services
– Commercial: Robert Howdin ([email protected])– Technical: Rod Hennegan ([email protected])
• IBM - Smarter planet initiative– Innovative solutions to business opportunities– Services, COTS/custom software, middleware, hardware
• Innovating for 100 years – its what we do and we do it well
12
eCensus 2006 Awards
• 2007 Computer World Honours Laureate– Honouring Those Who Use Information Technology to
Benefit Society http://www.cwhonors.org/viewCaseStudy.asp?NominationID=312
Winner 2007 Australian Excellence in e-Government Awardshttp://www.finance.gov.au/publications/excellence-in-e-government-awards-2007-finalist-case-studies
