Virtual Observatory Topics at ADASS XII

Baltimore, MD 13-17 October 2002

Vince McIntyre (ATNF, CSIRO)

ANITA meeting, 2003 Jan 28/29

About the Conference

● Astronomical Data Analysis Systems & Software● Computer system developments for astronomy

● telescope scheduling● telescope control● data taking formats● data processing software● data archiving● archive access

● Hosted by STScI● (CDS Strasbourg in 2003)

● a whole lot less virtual than it was● prototype web services running now,

●e.g. http://ads.harvard.edu/pubs/ws/● Mon: eSTAR initiative ● Tue: NRAO e2e● Wed: Theory data● Thu: Interop

Virtual Observatory at ADASS

Monday: eSTAR

● Prototype robotic telescope network●test computing infrastructure, for possible larger scale projects (e.g. Large Synoptic Survey Telescope)●network of telescopes with:

●rapid reduction pipelines●intelligent agents (IAs)

●Intelligent Agents ●examine the data ●request follow-up observations, as needed●wake up astronomer

Monday: eSTAR

●IA architecture

Monday: eSTAR

●IA prototype

Monday: eSTAR

● Requesting observations is important -●discovery nodes (telescopes) regulate workload by rejecting requests they have no time to fill (or because the astronomer has no allocated time)●applies in any distributed processing system – ie computational grids. No central control bottleneck means the system can scale up hugely

● Uniform interface is important - ●From the astronomers perspective, there is no difference between a telescope and a group of federated astronomical databases – apart from the query processing time...

Tuesday: e2e

● e2e stands for <End to End> its a data pipeline● Exploration project

●set up VLA archive based on AIPS++ toolkit. ●Data processed in AIPS++ from visibilities to images

● Capturing observers intentions in the logfiles is

critical● Relatively simple to capture intentions

●e.g. boxes on proposal form● Processing starts at proposal submission. Having visibilities marks the half-way point

Wednesday: Theory-VO

● <We need a place to put extinction curves!>●typical observers task - correct an image for extinction by interstellar dust●many versions, theoretical and observed, for different wavelength regimes●revised over the years - one wants to be able to redo corrections on old photometry, with modern curves.

●VO should help us discover the latest (best? most widely used?)

Wednesday: Theory-VO

●Other examples -●luminosity functions (stars, HII regions, planetary nebulae, galaxies)●stellar evolution tracks & isochrones

●Small datasets, small models are important too

●A structured approach is needed, if software is going to make use of it

Thursday: VO interop

● Data Models important for interop & seamless scaling● Lots of prototyping going on now● Working demos by 3rd quarter 2003● Coordination site: www.ivoa.net

Aside: VOTable

● VOTable v1.0 published June 2002● VOTable: hierarchy of Metadata + Tables

● Metadata: Parameters + Infos + Descriptions + Links + Fields

● Table: list of Fields + Data● Data: stream of Rows● Row: list of Cells● Cell: Primitive, or

variable-length list of Primitives, or

multidimensional array of Primitives

Aside: VOTable

<!DOCTYPE VOTABLE SYSTEM "http://us-vo.org/xml/VOTable.dtd"> <VOTABLE version="1.0" xmlns="http://vizier.u-strasbg.fr/VOTable">

<DESCRIPTION>This is an example VOTable document</DESCRIPTION>

<DEFINITIONS>

<COOSYS ID="myJ2000" equinox="2000." epoch="2000" system="eq_FK5"/>

</DEFINITIONS>

<RESOURCE name="GSC1.2">

<DESCRIPTION>

This is an excerpt of the HST Guide Star Catalog, Version 1.2 (Lasker+ 1996). This version was re-reduced with PPM catalogue.

</DESCRIPTION>

<TABLE>

<DESCRIPTION>Default result of GSC1.2 Server around a target</DESCRIPTION>

<FIELD ID="_r" name="_r" ucd="POS_ANG_DIST" unit="arcmin"

datatype="float" width="7" precision="4">

<DESCRIPTION>Distance from target NGC40</DESCRIPTION>

<VALUES type="actual">

<MIN value="0.0"/>

<MAX value="10.0"/>

</VALUES>

</FIELD>

Thursday: VO Web Services

● Web Services Acronyms●SOAP - Simple Object Access Protocol

●application of XML for sending commands to a remote service (RPC)

●WSDL - Web Services Description Language

●describe what your site can do●GLU - Generateur des Liens Uniformes

●helps maintain links between sites●symbolic names (UCDs) generate URLs●system of GLU servers work together to maintain links to resources

Thursday: VO Web Services

● GLU servers distribute WSDL files, so each node knows what resources are available [done]● Each GLU site has a SOAP responder to process requests [progressing]● GLU APIs in Java (C & Perl exist) [testing]

Thursday: VO Web Services

● requester sends this -<?xml version="1.0" encoding="UTF-8"?><SOAP-ENV:Envelope xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance" xmlns:xsd="http://www.w3.org/1999/XMLSchema"> <SOAP-ENV:Body> <namesp1:query xmlns:namesp1="http://ads.harvard.edu/AbsQuery"> <c-gensym3 xsi:type="xsd:string">jou_pick</c-gensym3> <c-gensym5 xsi:type="xsd:string">yes</c-gensym5> <c-gensym7 xsi:type="xsd:string">author</c-gensym7> <c-gensym9 xsi:type="xsd:string">Accomazzi</c-gensym9> <c-gensym11 xsi:type="xsd:string">ref_stems</c-gensym11> <c-gensym13 xsi:type="xsd:string">2001adass</c-gensym13> </namesp1:query> </SOAP-ENV:Body></SOAP-ENV:Envelope>

Thursday: VO Web Services

● this is returned (excerpt)-<opt> <anon> <score>1</score> <url>http://adsabs.harvard.edu/cgibin/nphbib_query?bibcode=2002DPS....34.3210E&amp;db_key=AST</url> <title>Formatting and Retrieval Options in the Astrophysics Data System (ADS)</title> <bibcode>2002DPS....34.3210E</bibcode> <author>Eichhorn, G.</author> <author>Accomazzi, A.</author> <author>Grant, C. S.</author> <author>Kurtz, M. J.</author> <author>Murray, S. S.</author> <pubdate>9/2002</pubdate> <link> <url>http://adsabs.harvard.edu/cgibin/nphdata_query?bibcode=2002DPS....34.3210E&amp;link_type=ABSTRACT</url> <name>Abstract</name> </link> </anon></opt>

Thursday: VO Web Services

● requester code

#usage: #perl adsq.pl author=Accomazzi jou_pick=yes ref_stems=2001adass##use strict;use SOAP::Lite +trace;use XML::Simple;

my %query = map { split(/=/,$_,2) } @ARGV;

my $response = SOAP::Lite ->uri('http://ads.harvard.edu/AbsQuery') ->proxy('http://ads.harvard.edu/ws/bibserver') ->query(%query);

print XMLout($response->result, noattr=>1);

Thursday: VO interop - VOTable

● Image Access Protocol discussion● Proposal to use VOTable for image access service storage and transfer format

● VOTable is application of XML.● FITS image can be part of VOTable● details at http://www.us-vo.org/VOTable/

● Trying to define standard●How to get it endorsed (rapidly)

●W3C? Global Grid Forum?

Thursday: Refactoring UCDs

●a coordinate system for astrophysics conceptsUCDs - Universal Content Descriptors

●developed by SIMBAD http://vizier.u-strasbg.fr/doc/UCD.htx●Hierarchy of concepts

pos_galpos

pos_eq

pos_gal_longpos_gal_lat

pos_eq_ra

pos_eq_dec●examples

"PHOT_INT-MAG_B" Integrated total blue magnitude "ORBIT_ECCENTRICITY" Orbital eccentricity "STAT_MEDIAN" Statistics Median Value "INST_QE" Detector's Quantum Efficiency

Thursday: Refactoring UCDs

●current set of UCDs have too few classes for precise classification(e.g. error_* UCD exists, but no way to distinguish error in R.A. vs error in Dec.)

●A tree that is too 'bushy' is also unwieldly

●Some quantity names have many independent aspects, mean different things in different contexts

Thursday: Refactoring UCDs

●Proposal to refactor - build up UCDs from 'atoms' in approved combinations ●e.g. POS_EQ_RA_MAIN

(main RA column in a table)

becomes POS_EQ_RA/MAIN, where POS_EQ_RA and MAIN are atoms that can be recognised separately in parsing

●Many 'corner cases', under discussion

Thursday: Data Models

● Describe (in machine-encodable terms) what you really have in the data set

● UCDs are part of this; UCDs let you know if quantities can be intercompared

● Data models let you know how you could transform a quantity for comparison

● More generally: DM for a resource tells me what questions I can ask of the resource

Thursday: VO interop - Data Models

● Filter Bandpass example●Archiving a Near-IR image. ●Passband has:

●generic name: J Band●specific name: AAO J Band

● The full name implies effective wavelength, wavelength range; ie transmission curve

● How best to express this in the metadata?

Thursday: VO interop - Data Models

● Do you enter filter_AAO_J-Band and have a name resolver (GLU) find and enter the transmission curve details for you?● Or just enter the UCD and leave resolution to be done when someone looks at the image?● Or enter filter FHWM, peak transmission etc?

● nasty cases need to be handled:●red leak●periodic filters●is atmospheric transmission included?

Thursday: VO interop - Data Models

● Another example - data from four sources:● AAT V-band image, FITS file● WFPC2 multi-element FITS file● Chandra x-ray (arrival time, position) table● ISOCAM FITS table

● Nothing in the UCD string captures the fact these are the same kind of data – images● Data model for each data source should allow you to infer this (or state it explicitly)

Recap

● UCDs describe the things you are seeking● WSDL file from each site gives full list of services ● your local resource locator (e.g. GLU server) uses collected WSDL files to find the same UCDs in several repositories around the globe ● SOAP enquiry to (e.g.) image retrieval services at several sites● VOTables returned as result of queries

The End

Thursday: VO interop - Data Models

● Image access protocol - scary UML diagrams