Upload
dustin-casey
View
27
Download
1
Embed Size (px)
DESCRIPTION
an International Virtual Observatory data exchange format. Roy Williams François Ochsenbein Clive Davenhall Daniel Durand Pierre Fernique David Giaretta Robert Hanisch Tom McGlynn Alex Szalay Andreas Wicenec. VOTable. XML: Structured Information. - PowerPoint PPT Presentation
Citation preview
an International Virtual Observatoryan International Virtual Observatory
data exchange format data exchange format
VOTable
Roy WilliamsFrançois Ochsenbein
Clive DavenhallDaniel Durand
Pierre FerniqueDavid Giaretta
Robert HanischTom McGlynn
Alex Szalay Andreas Wicenec
XML: Structured Information
<From>Antonio Stadivarius</From><To>Domenico Scarlatti</To><Date>
<Day>13</Day><Month>4</Month><Year>1723</Year>
</Date><Body>Io bisogno una appartamento acoglienti a Cremona …</Body>
4/13/23
April 13, 1723
17.iv.1723
Separation of structure from presentation
The computer can read the document:“Find all memos from April 1723”
VOTable
• Full metadata representation• Hierarchy of RESOURCEs• containing PARAMs and TABLEs• UCD (unified content descriptor)
– a has unit meter
– a has UCD ORBIT_SIZE_SMAJ (Semi-major axis of the orbit )
• Can reference remote and/or binary streams• Table can be
– Pure XML
– "Simple Binary"
– FITS Binary Table
VOTable Parentage
• Astrores• XML format for tables• Developed at CDS Strasbourg• Presented at ADASS 1999• Vizier implementation
• XSIL• XML format for Tables and Arrays• Developed at LIGO Caltech 2000• Extensible through Type-Class dynamic loading• Java parsing, browsing, editing• Matlab interface
Sample VOTable
<?xml version="1.0"?><!DOCTYPE VOTABLE SYSTEM "http://us-vo.org/xml/VOTable.dtd"><VOTABLE version="1.0"> <DEFINITIONS> <COOSYS ID="myJ2000" equinox="2000." epoch="2000." system="eq_FK5"/> </DEFINITIONS> <RESOURCE> <PARAM name="Observer" datatype="char" arraysize="*" value="William Herschel"> <DESCRIPTION>This parameter is designed to store the observer's name </DESCRIPTION> </PARAM> <TABLE name="Stars"> <DESCRIPTION>Some bright stars</DESCRIPTION> <FIELD name="Star-Name" ucd="ID_MAIN" datatype="char" arraysize="10"/> <FIELD name="RA" ucd="POS_EQ_RA" ref="myJ2000" unit="deg" datatype="float" precision="F3" width="7"/> <FIELD name="Dec" ucd="POS_EQ_DEC" ref="myJ2000" unit="deg" datatype="float" precision="F3" width="7"/> <FIELD name="Counts" ucd="NUMBER" datatype="int" arraysize="2x3x*"/> <DATA> <TABLEDATA> <TR> <TD>Procyon</TD><TD>114.827</TD><TD>5.227</TD> <TD>4 5 3 4 3 2 1 2 3 3 5 6</TD> </TR> <TR> <TD>Vega</TD><TD>279.234</TD> <TD>38.782</TD><TD>8 7 8 6 8 6</TD> </TR> </TABLEDATA> </DATA> </TABLE> </RESOURCE></VOTABLE>
<DATA> <FITS> <STREAM href="ftp://server.com/mydata.fits" expires="2002-02-22" actuate="onRequest"/> </FITS></DATA>
Table Cell
scalar
arrays
variable length arrays
etc
etc
boolean
bit
unsignedByte
short
int
long
char
unicodeChar
float
double
floatComplex
doubleComplex
Primitives
VOTable is Flexy
• eg Table of images• UCD="JPEG_IMAGE"
datatype="unsignedByte" arraysize="*"
• eg Table of URL links• UCD=“DATA_LINK"
datatype="char" arraysize="*"
VOTable Schema (xsd)
Table Data Model
• Metadata• Class definition for Row• FIELD
– data type
– semantic type
• Data• Each Row is a list of Cells• Each Cell is an array of Primitives
– may be variable length
Table Data Layout
• All metadata first – small, complex, XML
• Class definition for table record• + params, description, etc etc
• Then data– (may be) large, remote– XML | binary | FITS
• Instantiations of table record• All records MUST have same format
Param Data Model
• Param is “Table with one cell”• Like a FIELD value• But with a “value” attribute
Primitives
• All have fixed binary length
• Same as FITS primitives
• Except Unicode
datatype Meaning FIT
S Bytes
"boolean" Logical "L" 1
"bit" Bit "X" *
"unsignedByte" Byte (0 to 255) "B" 1
"short" Short Integer "I" 2
"int" Integer "J" 4
"long" Long integer "K" 8
"char" ASCII Character "A" 1
"unicodeChar" Unicode Character 2
"float" Floating point "E" 4
"double" Double "D" 8
"floatComplex" Float Complex "C" 8
"doubleComplex" Double Complex "M" 16
Multidimensional Array Cell
• A table cell can have lots of Primitives• Example: WCS parameters are arrays
– <FIELD name=“CRVAL”datatype=“double”arraysize=“2”/>
• Example: up to 10 images, each 64x64– <FIELD name="thumbs"
datatype="unsignedByte" arraysize="64x64x10*"/>
Hierarchy
• A VOTable contains RESOURCES– RESOURCE can contain:
• TABLE• RESOURCE• etc etc
• Usage example• Many observations in the file,
– each is a RESOURCE
• Each observation is– Parameters
– Calibration table
– Raw data table
Hierarchy
• New feature: GROUP
<TABLE name=“Nutation and Aberration”> <GROUP name=“Nutation”> <FIELD name=“Longitude”/> <FIELD name=“Obliquity”/> </GROUP> <GROUP name=“Aberration”> <GROUP name=“Equinox 1950.0”> <FIELD name=“C”/> <FIELD name=“D”/> </GROUP> <GROUP name=“Equinox 1955.0”> <FIELD name=“C”/> <FIELD name=“D”/> </GROUP> </GROUP></TABLE>
Unified Content Descriptors
• UCD is a “semantic type”
PHOT.INT-MAG.B Integrated total blue magnitude ORBIT.ECCENTRICITY Orbital eccentricity STAT.MEDIAN Statistics Median Value INST.QE Detector's Quantum Efficiency
• Can be resolved by web service– to description, examples, etc
• Base + Specifiers• eg error in default right ascension• POS.EQ.RA, MAIN, ERROR
VOTable Friends
XML BinaryStreaming
Table Datacube Semantics
VOTable√ √ √ √
BinX√ √ √ √
MS Dataset √ √
HDF√ √
XDF√ √
Some “self-describing” file formats
XML Parsing
SAX: Event-BasedHandlers for StartElement, Text, EndElement, etc.
Found element BookCatalogueFound element BookFound Element TitleFound Text The Cambridge Star AtlasFound End Element Title….
Parsing
DOM: Document Object ModelReturns a tree-like Document object with data attached
BookCatalogue
BookBook
Title
ISBN Parallel Computing Works!
Cambridge Star Atlas
Title
Author
Wil Tirion
Binding to make a Parser
for(int i=0; i<table.getFieldCount(); i++){ Field field = (Field)table.getFieldAt(i); String u = field.getUcd(); if(u != null && u.equals("POS_EQ_RA_MAIN")) System.out.println("Field " + i + " is for RA"); }
From the Schema an API and library is generatedJAXBBreezeCastor
This is JAVOT (Caltech)
VOTable Software
Treeview from UK-VO
VOTable Software
VOPlot from India-VO
VOTable Software
VOTool from US-VO
VOTable software
Mirage from Bell Labs