This topic aims to describe the specification from a technical point of view.
TODO: define the instrument mode (can we get same general concept across various instruments : resolution (LOW MEDIUM HIGH), Lambda min/max etc...? )
Data import
Parse/Read: submitted files are parsed with oitools to produce a structure of data from the file.
Analyze/extract data: given the data structure above, it splits the data into granules and outputs relevant metadata that are formatted as an XML fragment for each granule.
XML2SQL: transform the XML fragments describing a granule into SQL insertion for the database.
Target resolution: a service that try to identify the target of an observation given incomplete or approximate inputs from the file data.
Instrument mode resolution: a service that try to identify the instrumental mode used during an observation given a few elements from the file.
DB: relational database containing observations as support for the web portal and tap service with subtables for instruments and targets. A subset of the database (view) is exported through TAP
Backoffice: a panel for administrator to supervise the import process.
Data analysis
It add a few identifier to the OIFits structure as derived columns. These values can be queried to extract metadata granules.
OI_TARGET: add column(s) for target identifier - TARGET_UID - derived from OI_TARGET.TARGET, RAEP0, DECEP0, EQUINOX (converted to coords RA[deg], DEC[deg] J2000)
OI_WAVELENGTH: add keyword instrument mode - INSMODE (string) - derived from OI_WAVENLENGTH.INSNAME, OI_WAVELENGTH.EFF_WAVE_min/max, count(OI_WAVELENGTH.EFF_WAVE)
OI_DATA: add column for night identifier - NIGHT_ID - derived from OI_DATA.MJD or OI_DATA.DATE-OBS+TIME (converted to MJD) and OI_ARRAY(OI_DATA).ARRAYX, ARRAYY, ARRAYZ
Scenarios
- OIFits file without OI_ARRAY (optional)
Impact
- jmcs: convert coords to given epoch
- oitools: column definitions
- oiexplorer-core: analysis of target id, instrument mode, night identifiers / set column values
- API oiexplorer-core : TODO oifits.getOiTableOfInsMode(insmodeId), oidata.getInsModeId() oidata.getDistinctNightId(), oidata.getDistinctTargetUid()
Target resolution
Purpose: define identifier for target
Input: TARGET_NAME (string), coords (RA[deg], DEC[deg] J2000)
Output: TARGET_UID (pair of strings with ra as HMS and dec as DMS ?)
A service that builds an identifier based on previous resolutions and interaction with other services (like Simbad).
So doing it also help validating the input data and assigning validation flag to input data.
When a target can not be resolved, the identifier is built using input data and a flag is added specifying the reason of the failure (service unavailable, unknown target, no match between name and coords).
Scenarios
- name resolution does not return a result (unknown target) -> try with coords
- name resolution returns a single result
- resolved coords equals to passed coords
- resolved coords significantly different from passed coords
- name resolution returns more than one result
- coord reverse resolution returns no result
- coord reverse resolution returns a single result
- coord reverse resolution returns several results
Impact
- oiexplorer-core: resolutions, error reporting
Notes
- As new aliases are assigned to targets, new definition of names/aliases, insertions into Simbad, updates of coordinates happen, the service should also be routinely re-evaluated: identifying target in the database for which resolution return a result different from the one previously saved (main name, coords, identifier). A report would display the differences for validation/modification by an administrator.
Instrument mode resolution
Purpose: define identifier for instrument mode
Input: INSNAME (string), MIN_WL (double), MAX_WL (double), NB_CHANNEL (double), knowledge database for instrument modes
Output: INSMODE (string)
A service that build a fingerprint/hash to describe an instrument mode using a knowledge database of instrument modes.
This knowledge database may be the configuration file of ASPRO2 or an extract of
OiDB and build along data imports.
Scenarii
- fingerprints with slightly different INSNAME values (e.g. wavelengths min and max as in PIONIER_Pup(1.5994200/1.8035799) )
- instrument names with constant values (e.g. VEGA )
- unknown instrument mode
- unknown instrument
Impact
- oiexplorer-core: build a fingerprint/hash
- oidb: match fingerprint/hash with instrument mode
Notes
- footprint definition (single string, fields concatenation...)?
- how to solve instrument modes ambiguity (e.g. for MIDI and MATISSE how to choose between SCI_PHOT_LM_LOW or HIGH_SENS_LM_LOW )
Group by nights
Purpose: associate a night identifier to observations
Input: MJD (double[]), optionnal ARRAY_NAME (string)
Output: NIGHT_ID[], distinct_NIGHT_ID[]
Find which night corresponds to the given MJD at the array location.
Scenarios
- single night
- multiple nights (contiguous or not)
Impact
- jmal: night boundary at location,date
- oiexplorer-core: compute NIGHT_ID
General
- do we have to refresh/compare data retrieved by simbad