Click here to load reader
View
214
Download
0
Embed Size (px)
NG/AMS - User's Manual
ESO
NG/AMS - User's Manual
Doc.
Issue
Date
Page
VLT-MAN-ESO-19400-2739
2
08/01/2003
126 of 126
EUROPEAN SOUTHERN OBSERVATORY
Organisation Europenne pour des Recherches Astronomiques dans l'Hmisphre Austral
Europische Organisation fr astronomische Forschung in der sdlichen Hemisphre
Data Management Division
DFS Software
NG/AMS
Next Generation Archive Management System
Users Manual
Doc. No.: VLT-MAN-ESO-19400-2739
Issue: 2
Date: 08/01/2003
DRAFT - FOR INTERNAL USE ONLY
Name
Date
Signature
Prepared:J.Knudstrup
08/01/2003 (
Name
Date
Signature
Approved:A.Wicenec
/ /2003 (
Name
Date
Signature
Released:M.Peron/P.Quinn
/ /2003 (
ESO * TELEPHONE: (089) 3 20 06-0 * http://www.eso.org
CHANGE RECORD
ISSUE
DATE
SECTION/PAGE
AFFECTED
REASON/INITIATION
DOCUMENTS/REMARKS
1/Preperation 1
29.01.2002
All
First issue.
1/Preparation 2
05.03.2002
All
Added comment after internal review.
1/Preparation 1
08.01.2003
All
Major update with new features.
TABLE OF CONTENTS
111About this Guide
1.1Purpose & Scope11
1.2How to Read this Manual11
1.3How to Get Help or Report Problems with NG/AMS or this Manual11
1.4Disclaimer12
1.5Reference Documents12
1.6Acronyms12
1.7Glossary13
2Overview15
2.1The Concept of NGAS & NG/AMS15
2.2Services & Features16
2.3Starting & Stopping the NG/AMS Server17
2.4The NG/AMS Server States & Sub-States18
2.5The NG/AMS Storage Media Infrastructure18
2.6Data Classification & Handling19
2.7Disk Handling/Life Cycle of an NGAS Storage Media21
3Basic Features22
3.1Data File Archiving22
3.2Data File Retrieval & Processing22
3.3Logging23
3.4Email Notification24
3.5Disk Space Monitoring25
3.6Simulation Mode25
3.7Back-Log Buffering26
3.8The NG/AMS Server Command Interface27
3.9Data Consistency Checking27
3.10Label Printing28
3.11Security28
4EXPERT: Advanced Features30
4.1EXPERT: Operation in Cluster Mode30
4.2EXPERT: Data Subscription Service32
4.3EXPERT: Server Suspension/Wake-Up Service33
5The NG/AMS Server and Utilities34
5.1NG/AMS Server Command Line Interface34
5.2Python and C Command Utilities35
6EXPERT: Configuring NG/AMS38
6.1EXPERT: NG/AMS Configuration DTD - "ngamsCfg.dtd"38
6.2EXPERT: NG/AMS Base DTD - "ngamsInternal.dtd"38
6.3EXPERT: NG/AMS Configuration - Example45
7EXPERT: NG/AMS Server Communication Protocol48
7.1EXPERT: Format of NG/AMS HTTP Command Messages48
7.2EXPERT: Format of the NG/AMS HTTP Reply49
7.3EXPERT: Format of the NG/AMS Redirection HTTP Response49
8EXPERT: The NGAS DB51
8.1EXPERT: Table - "ngas_disks"51
8.2EXPERT: Table - "ngas_disks_hist"52
8.3EXPERT: Table - "ngas_files"52
8.4EXPERT: Table - "ngas_hosts"53
8.5EXPERT: Table - " ngas_subscr_back_log "54
8.6EXPERT: Table - " ngas_subscribers "54
8.7EXPERT: Synchronizing Distributed NGAS DBs55
9EXPERT: The C-API57
9.1EXPERT: NG/AMS C-API - Header File: ngams.h57
9.2EXPERT: NG/AMS C-API - Man Page57
10EXPERT: The Python API65
11EXPERT: The NG/AMS Plug-In API67
12EXPERT: The System Online Plug-In68
12.1EXPERT: Interface of a System Online Plug-In68
12.2EXPERT: Example System Online Plug-In69
13EXPERT: The System Offline Plug-In71
13.1EXPERT: Interface of a System Offline Plug-In71
13.2EXPERT: Example System Offline Plug-In71
14EXPERT: The Label Printer Plug-In73
14.1EXPERT: Interface of a Label Printer Plug-In73
14.2EXPERT: Example of a Label Printer Plug-In73
15EXPERT: The Data Archiving Plug-In - DAPI76
15.1EXPERT: Interface of a DAPI78
15.2EXPERT: Overall Structure & Algorithm of a DAPI79
15.3EXPERT: Example DAPI - WFI/FITS File DAPI80
16EXPERT: The Register Plug-In83
16.1EXPERT: Interface of a Register Plug-In83
16.2EXPERT: Example Register Plug-In83
17EXPERT: The Data Processing Plug-In - DPPI85
17.1EXPERT: Interface of a DPPI85
17.2EXPERT: Example DPPIs86
18EXPERT: The Data Checksum Plug-In89
18.1EXPERT: Interface of a Data Checksum Plug-In89
18.2EXPERT: Example Data Checksum Plug-In89
19EXPERT: The Suspension Plug-In89
19.1EXPERT: Interface of a Suspension Plug-In89
19.2EXPERT: Example Data Checksum Plug-In89
20EXPERT: The Wake-Up Plug-In89
20.1EXPERT: Interface of a Wake-Up Plug-In89
20.2EXPERT: Example Wake-Up Plug-In89
21EXPERT: The Filter Plug-in89
21.1EXPERT: Interface of a Filter Plug-In89
21.2EXPERT: Example Filter Plug-In89
22The NG/AMS Status XML Document89
22.1EXPERT: NG/AMS Status DTD (ngamsStatus.dtd)89
22.2NGAS Disk Info Status - Example89
22.3NGAS File Info Status - Example89
23EXPERT: The NG/AMS Python Modules89
23.1EXPERT: NG/AMS Module Structure89
23.2EXPERT: Online Browsing of NG/AMS Inline Python Documentation89
24EXPERT: Installation89
25NG/AMS Log and Error Messages Definition89
26NG/AMS License Conditions89
27NG/AMS Commands89
27.1ARCHIVE Command - Archive Data Files89
27.2CLONE Command Copy Files89
27.3EXIT Command - Terminate Server89
27.4INIT Command - Re-Initialize the System89
27.5LABEL Command - Generating Disk Labels89
27.6OFFLINE Command - Bring System to Offline State89
27.7ONLINE Command - Bring System to Online State89
27.8REGISTER Command - Register Existing Files on a Disk89
27.9REMDISK Command Remove Information about Disks89
27.10REMFILE Command Remove Files from the System89
27.11RETRIEVE Command - Retrieve & Process Files89
27.12STATUS Command - Query System Status & Other Information89
27.13SUBSCRIBE Command Subscribe to Data from NGAS Host89
27.14UNSUBSCRIBE Command Unsubscribe a Previous Data Subscription89
28Index89
LIST OF FIGURES
15Figure 1: Example operational environment of the NG/AMS Server.
18Figure 2: Response to STATUS command.
18Figure 3 The NG/AMS Storage Media Infrastructure.
20Figure 4: Data channeling.
21Figure 5: Life cycle of an NGAS disk.
25Figure 6: Disk Change Email Notification Message.
26Figure 7: Example reply when Back-Log Buffering is applied.
27Figure 8: Interaction with an NG/AMS Server from a WEB browser.
27Figure 9: Example of interaction with NG/AMS using telnet.
28Figure 10: Example of a Data Consistency Checking Report.
28Figure 11: Example of a Data Consistency Checking Status Log.
28Figure 12: Example Disk Label as generated by NG/AMS.
30Figure 13: Example of hierarchical NGAS Cluster.
31Figure 14: Example of a simple NGAS Cluster.
34Figure 15: The NG/AMS Server online help output (as written on stdout).
37Figure 16: The NG/AMS C-Client and Python-Client online help output (on stdout).
38Figure 17: NG/AMS Configuration DTD (FILE: ngams/ngamsData/ngamsCfg.dtd).
44Figure 18: NG/AMS Configuration generic DTD (FILE: ngams/ngamsData/ngamsInternal.dtd).
47Figure 19: Example NG/AMS Configuration file (FILE: ngams/ngamsData/ngamsServer.xml).
48Figure 20: Format of an Archive Push HTTP request.
48Figure 21: Example of Archive Push HTTP request.
48Figure 22: Structure of NG/AMS GET method HTTP request.
48Figure 23: Example of NG/AMS GET method HTTP request (Archive Pull Request).
49Figure 24: Format of NG/AMS HTTP response
49Figure 25: Example of NG/AMS HTTP response (Archive Request).
49Figure 26: Example of NG/AMS HTTP response, Retrieve Request.
50Figure 27: Structure of NG/AMS HTTP Redirection Response.
50Figure 28: Example of NG/AMS HTTP Redirection Response.
55Figure 29: Example of a Distributed NGAS installation using unidirectional, conditional DB replication.
64Figure 30: The functions and macros of the NG/AMS C-API.
65Figure 31: Using the NG/AMS Python-API.
66Figure 32: Small example program using the Python-API (FILE: ngams/ngamsPClient/ngamsPClientEx).
66Figure 33: Output on stdout from example program using the Python-API.
68Figure 34: Function interface of a System Online Plug-In.
69Figure 35: The NG/AMS Physical Disk Dictionary.
70Figure 36: Example System Online Plug-In (FILE: ngams/ngamsPlugIns/ngamsLinuxOnlinePlugIn.py.
72Figure 37: Example System Offline Plug-In (FILE: ngams/ngamsPlugIns/ngamsLinuxOfflinePlugIn.py).
73Figure 38: Function interface of a Label Printer Plug-In.
75Figure 39: Example Label Printer Plug-In (FILE: ngams/ngamsPlugIns/ngamsBrotherPT9200DxPlugIn.py).
77Figure 40: Handling of an Archive Request.
78Figure 41: Function interface of a DAPI.
78Figure 42: DAPI return statement.
79Figure