Upload
philippa-barrett
View
217
Download
0
Embed Size (px)
Citation preview
Management System of Event Processing and Data Files Based on XML Software Tools at Belle
Ichiro Adachi, Nobu Katayama, Masahiko YokoyamaIPNS, KEK, Tsukuba, Japan
Yutaka Ueshima, Yukimasa Matsuda, Takashi OkamotoQuatre-i Science Ltd., Kyoto, Japan
Belle Experiment at KEKB Accelerator• B-factory experiment at KEK, where 8 GeV electrons collide with 3.5 GeV positrons, to explore frontier in flavor physics with millions of B-meson pairs
• KEKB accelerator: world’s highest e+e- collider providing 1.6x1034 /cm2/sec luminosity
• More than 700/fb of data has been accumulated since 1999
Luminosity Recordspeak luminosity: 1.71x1034 /cm2/secIntegrated luminosities
daily: 1.23/fb8 hour: 0.43/fb
further improvement expected due to installation of crab cavities
Belle Event Processing and File Management
Rawdata
Physics event skims
mu-pair
Bhabha
Real-time processing
DST data
hadronic
Reprocessing
Calibration constant making
Physics analysis files
file registration into user analysis list
archive for backup
appropriate distribution for user analysis
detector experts’ analysis
data quality monitoring
• Most of steps in event processing have been automated, however human intervention is still necessary in some parts.
• More than 1 million data files are difficult to manage in a consistent way.
need a systematic procedure for checking/monitoring in each step as well as stable management for data files
Introduce RCM system tools
RCM ( R&D Chain Management ) System Software
Automated repetition of event processing procedureCapability of automatic retrieval of errorHigh traceability of each stepKeep history in databaseClear visualization of processing status using Web
access controllog-database
Web accessXML viewerCLI template
1-st layer
2-nd layer
3-rd layer
REST SSH SCP
XML-USER INTERFACE
User PC
Internet、 LAN,etc
Web server
File server
Analyze server
Sensor devices
Visualize server
DB sever
XML DATABASE
Super computer
Control server
XML-CONTROL
Concept of RCM system
server configurationwork-flow configuration
automation of repetitive work
login window in a browser
Layer 2 Layer 3
File nameÅF I_250mA.datDateÅF2004-04-27File NameÅFÅ@I_250Time_VÅDÇêÇéÇáDate; 2004-04-27SizeÅF100k
File NameÅFÅ@I_250DistÅDÇêÇéÇáDate; 2004-04-27SizeÅF180k
èâä í̇l ÅF 220mADate ÅF 2004-04-26Code VerÅF 1.8
Flexible shared level
Public
Private
Group
workflow
log
DB request
RCM-DB
Design free database
annotation
Analysis flow is defined as a set of “workflow templates” written in XMLIn each workflow template, check point can be provided as users’ requestEach check point ensures correct stream in the flow
XML user interfaceMulti-functional viewer provides automatic display of text, thumbnail and download link interpreting query result written in XML
List of templates
Edit window of template GUI window of template
XML workflow templates automate complex data search, analysis, registration in XML-DB. which are shared and easily edited
DST processing including output data management was described as 27 stepsand each step was defined in one XML template
data file servers
database server
HSM archive server
Data file quality
If fail, copy back from HSM archive server
Data database update
MD5 checksum
RAID disk server
issue copy back from HSM
database server
data location stored in DBupdated in a consistent way
Processed data files are also managed by RCMMD5 checksum is always done and result is kept when output files is producedIn case of disk trouble, files are automatically examined, and if corrupted, automatic recovery is issued from RCM
Intelligent Belle Processing & Data File Management System Based on RCM
Produced data consistency check
DST job error analysis
DST job completion
set of 27 templates
test
answer
query
answer
In each template, one can communicate with a database server as well as any scripts to perform as one step in a RCM templateNecessary checking as well as monitoring can be made automaticallyProcessing error, if happens, can be notified via email to managers Recovery process can be also made by RCM if defined beforehand
Event processing File management
Abstract number : 282
2006-2007 Quatre-i Science. Ltd. All rights reserved ref: http://www.i4s.co.jp/