Click here to load reader

NG/AMS - User's Manual - European Southern jknudstr/NGAS/docs/VLT-MAN-ESO-19400-2739-V... · Web viewTable 1: Conventions/styles used in the NG/AMS User’s Manual. 11 Table 2: Reference

  • View
    214

  • Download
    0

Embed Size (px)

Text of NG/AMS - User's Manual - European Southern...

NG/AMS - User's Manual

ESO

NG/AMS - User's Manual

Doc.

Issue

Date

Page

VLT-MAN-ESO-19400-2739

2

08/01/2003

126 of 126

EUROPEAN SOUTHERN OBSERVATORY

Organisation Europenne pour des Recherches Astronomiques dans l'Hmisphre Austral

Europische Organisation fr astronomische Forschung in der sdlichen Hemisphre

Data Management Division

DFS Software

NG/AMS

Next Generation Archive Management System

Users Manual

Doc. No.: VLT-MAN-ESO-19400-2739

Issue: 2

Date: 08/01/2003

DRAFT - FOR INTERNAL USE ONLY

Name

Date

Signature

Prepared:J.Knudstrup

08/01/2003 (

Name

Date

Signature

Approved:A.Wicenec

/ /2003 (

Name

Date

Signature

Released:M.Peron/P.Quinn

/ /2003 (

ESO * TELEPHONE: (089) 3 20 06-0 * http://www.eso.org

CHANGE RECORD

ISSUE

DATE

SECTION/PAGE

AFFECTED

REASON/INITIATION

DOCUMENTS/REMARKS

1/Preperation 1

29.01.2002

All

First issue.

1/Preparation 2

05.03.2002

All

Added comment after internal review.

1/Preparation 1

08.01.2003

All

Major update with new features.

TABLE OF CONTENTS

111About this Guide

1.1Purpose & Scope11

1.2How to Read this Manual11

1.3How to Get Help or Report Problems with NG/AMS or this Manual11

1.4Disclaimer12

1.5Reference Documents12

1.6Acronyms12

1.7Glossary13

2Overview15

2.1The Concept of NGAS & NG/AMS15

2.2Services & Features16

2.3Starting & Stopping the NG/AMS Server17

2.4The NG/AMS Server States & Sub-States18

2.5The NG/AMS Storage Media Infrastructure18

2.6Data Classification & Handling19

2.7Disk Handling/Life Cycle of an NGAS Storage Media21

3Basic Features22

3.1Data File Archiving22

3.2Data File Retrieval & Processing22

3.3Logging23

3.4Email Notification24

3.5Disk Space Monitoring25

3.6Simulation Mode25

3.7Back-Log Buffering26

3.8The NG/AMS Server Command Interface27

3.9Data Consistency Checking27

3.10Label Printing28

3.11Security28

4EXPERT: Advanced Features30

4.1EXPERT: Operation in Cluster Mode30

4.2EXPERT: Data Subscription Service32

4.3EXPERT: Server Suspension/Wake-Up Service33

5The NG/AMS Server and Utilities34

5.1NG/AMS Server Command Line Interface34

5.2Python and C Command Utilities35

6EXPERT: Configuring NG/AMS38

6.1EXPERT: NG/AMS Configuration DTD - "ngamsCfg.dtd"38

6.2EXPERT: NG/AMS Base DTD - "ngamsInternal.dtd"38

6.3EXPERT: NG/AMS Configuration - Example45

7EXPERT: NG/AMS Server Communication Protocol48

7.1EXPERT: Format of NG/AMS HTTP Command Messages48

7.2EXPERT: Format of the NG/AMS HTTP Reply49

7.3EXPERT: Format of the NG/AMS Redirection HTTP Response49

8EXPERT: The NGAS DB51

8.1EXPERT: Table - "ngas_disks"51

8.2EXPERT: Table - "ngas_disks_hist"52

8.3EXPERT: Table - "ngas_files"52

8.4EXPERT: Table - "ngas_hosts"53

8.5EXPERT: Table - " ngas_subscr_back_log "54

8.6EXPERT: Table - " ngas_subscribers "54

8.7EXPERT: Synchronizing Distributed NGAS DBs55

9EXPERT: The C-API57

9.1EXPERT: NG/AMS C-API - Header File: ngams.h57

9.2EXPERT: NG/AMS C-API - Man Page57

10EXPERT: The Python API65

11EXPERT: The NG/AMS Plug-In API67

12EXPERT: The System Online Plug-In68

12.1EXPERT: Interface of a System Online Plug-In68

12.2EXPERT: Example System Online Plug-In69

13EXPERT: The System Offline Plug-In71

13.1EXPERT: Interface of a System Offline Plug-In71

13.2EXPERT: Example System Offline Plug-In71

14EXPERT: The Label Printer Plug-In73

14.1EXPERT: Interface of a Label Printer Plug-In73

14.2EXPERT: Example of a Label Printer Plug-In73

15EXPERT: The Data Archiving Plug-In - DAPI76

15.1EXPERT: Interface of a DAPI78

15.2EXPERT: Overall Structure & Algorithm of a DAPI79

15.3EXPERT: Example DAPI - WFI/FITS File DAPI80

16EXPERT: The Register Plug-In83

16.1EXPERT: Interface of a Register Plug-In83

16.2EXPERT: Example Register Plug-In83

17EXPERT: The Data Processing Plug-In - DPPI85

17.1EXPERT: Interface of a DPPI85

17.2EXPERT: Example DPPIs86

18EXPERT: The Data Checksum Plug-In89

18.1EXPERT: Interface of a Data Checksum Plug-In89

18.2EXPERT: Example Data Checksum Plug-In89

19EXPERT: The Suspension Plug-In89

19.1EXPERT: Interface of a Suspension Plug-In89

19.2EXPERT: Example Data Checksum Plug-In89

20EXPERT: The Wake-Up Plug-In89

20.1EXPERT: Interface of a Wake-Up Plug-In89

20.2EXPERT: Example Wake-Up Plug-In89

21EXPERT: The Filter Plug-in89

21.1EXPERT: Interface of a Filter Plug-In89

21.2EXPERT: Example Filter Plug-In89

22The NG/AMS Status XML Document89

22.1EXPERT: NG/AMS Status DTD (ngamsStatus.dtd)89

22.2NGAS Disk Info Status - Example89

22.3NGAS File Info Status - Example89

23EXPERT: The NG/AMS Python Modules89

23.1EXPERT: NG/AMS Module Structure89

23.2EXPERT: Online Browsing of NG/AMS Inline Python Documentation89

24EXPERT: Installation89

25NG/AMS Log and Error Messages Definition89

26NG/AMS License Conditions89

27NG/AMS Commands89

27.1ARCHIVE Command - Archive Data Files89

27.2CLONE Command Copy Files89

27.3EXIT Command - Terminate Server89

27.4INIT Command - Re-Initialize the System89

27.5LABEL Command - Generating Disk Labels89

27.6OFFLINE Command - Bring System to Offline State89

27.7ONLINE Command - Bring System to Online State89

27.8REGISTER Command - Register Existing Files on a Disk89

27.9REMDISK Command Remove Information about Disks89

27.10REMFILE Command Remove Files from the System89

27.11RETRIEVE Command - Retrieve & Process Files89

27.12STATUS Command - Query System Status & Other Information89

27.13SUBSCRIBE Command Subscribe to Data from NGAS Host89

27.14UNSUBSCRIBE Command Unsubscribe a Previous Data Subscription89

28Index89

LIST OF FIGURES

15Figure 1: Example operational environment of the NG/AMS Server.

18Figure 2: Response to STATUS command.

18Figure 3 The NG/AMS Storage Media Infrastructure.

20Figure 4: Data channeling.

21Figure 5: Life cycle of an NGAS disk.

25Figure 6: Disk Change Email Notification Message.

26Figure 7: Example reply when Back-Log Buffering is applied.

27Figure 8: Interaction with an NG/AMS Server from a WEB browser.

27Figure 9: Example of interaction with NG/AMS using telnet.

28Figure 10: Example of a Data Consistency Checking Report.

28Figure 11: Example of a Data Consistency Checking Status Log.

28Figure 12: Example Disk Label as generated by NG/AMS.

30Figure 13: Example of hierarchical NGAS Cluster.

31Figure 14: Example of a simple NGAS Cluster.

34Figure 15: The NG/AMS Server online help output (as written on stdout).

37Figure 16: The NG/AMS C-Client and Python-Client online help output (on stdout).

38Figure 17: NG/AMS Configuration DTD (FILE: ngams/ngamsData/ngamsCfg.dtd).

44Figure 18: NG/AMS Configuration generic DTD (FILE: ngams/ngamsData/ngamsInternal.dtd).

47Figure 19: Example NG/AMS Configuration file (FILE: ngams/ngamsData/ngamsServer.xml).

48Figure 20: Format of an Archive Push HTTP request.

48Figure 21: Example of Archive Push HTTP request.

48Figure 22: Structure of NG/AMS GET method HTTP request.

48Figure 23: Example of NG/AMS GET method HTTP request (Archive Pull Request).

49Figure 24: Format of NG/AMS HTTP response

49Figure 25: Example of NG/AMS HTTP response (Archive Request).

49Figure 26: Example of NG/AMS HTTP response, Retrieve Request.

50Figure 27: Structure of NG/AMS HTTP Redirection Response.

50Figure 28: Example of NG/AMS HTTP Redirection Response.

55Figure 29: Example of a Distributed NGAS installation using unidirectional, conditional DB replication.

64Figure 30: The functions and macros of the NG/AMS C-API.

65Figure 31: Using the NG/AMS Python-API.

66Figure 32: Small example program using the Python-API (FILE: ngams/ngamsPClient/ngamsPClientEx).

66Figure 33: Output on stdout from example program using the Python-API.

68Figure 34: Function interface of a System Online Plug-In.

69Figure 35: The NG/AMS Physical Disk Dictionary.

70Figure 36: Example System Online Plug-In (FILE: ngams/ngamsPlugIns/ngamsLinuxOnlinePlugIn.py.

72Figure 37: Example System Offline Plug-In (FILE: ngams/ngamsPlugIns/ngamsLinuxOfflinePlugIn.py).

73Figure 38: Function interface of a Label Printer Plug-In.

75Figure 39: Example Label Printer Plug-In (FILE: ngams/ngamsPlugIns/ngamsBrotherPT9200DxPlugIn.py).

77Figure 40: Handling of an Archive Request.

78Figure 41: Function interface of a DAPI.

78Figure 42: DAPI return statement.

79Figure

Search related