(r6) OiDbTechnicalSpecs < Jmmc/Software

Tags: view all tags
This topic aims to describe the specification from a technical point of view.

%TOC%

TODO: define the instrument mode (can we get same general concept accross various instruments : resolution (LOW MEDIUM HIGH), Lambda min/max etc...  )

---++ Data import

<img src="%ATTACHURLPATH%/import.png"/>

Parse/Read: submitted files are parsed with oitools to produce a structure of data from the file.

Analyze/extract data: given the data structure above, it splits the data into granules and outputs relevant metadata that are formatted as an XML fragment for each granule.

!XML2SQL: transform the XML fragments describing a granule into SQL insertion for the database.

Target resolution: try to identify the target of an observation given incomplete or approximate inputs from the file data.

Instrument mode resolution: try to identify the instrumental mode used during an observation given a few elements from the file.

DB: relational database containing observations as support for the web portal and tap service with subtables for instruments and targets. A subset of the database (view) is exported through TAP

Backoffice: a panel for administrator to supervise the import process.


---+++ Data analysis

It add a few identifier to the OIFits structure as derived columns. These values can be queried to extract metadata granules.

OI_TARGET: add column(s) for target identifier - TARGET_UID - derived from OI_TARGET.TARGET, RAEP0, DECEP0, EQUINOX (converted to coords RA[deg], DEC[deg] J2000)

OI_WAVELENGTH: add column instrument mode - INSMODE (string) - derived from OI_WAVELENGTH.EFF_WAVE_min/max, count(OI_WAVELENGTH.EFF_WAVE)

OI_DATA: add column for night identifier - NIGHT_ID - derived from OI_DATA.MJD or OI_DATA.DATE-OBS+TIME (converted to MJD) and OI_ARRAY(OI_DATA).ARRAYX, ARRAYY, ARRAYZ 

---++++ Scenarios

   * OIFits file without OI_ARRAY (optional)

---++++ Impact

   * jmcs: convert coords to given epoch
   * oitools: column definitions
   * oiexplorer-core: analysis of target id, instrument mode, night identifiers / set column values


---+++ Target resolution

Purpose: define identifier for target

Input: TARGET_NAME (string), coords (RA[deg], DEC[deg] J2000) %BR%
Output: TARGET_UID (pair of strings with ra as HMS and dec as DMS ?)

It builds an identifier by interacting with other services (like Simbad and previous resolution from OiDB itself).
So doing it also help validating the input data and assigning validation flag to input data.

When a target can not be resolved, the identifier is built using input data and a flag is added specifying the reason of the failure (service unavailable, unknown target, no match between name and coords).

---++++ Scenarios

   * name resolution does not return a result (unknown target) -> try with coords
   * name resolution returns a single result
      * resolved coords equals to passed coords
      * resolved coords significantly different from passed coords
   * name resolution returns more than one result
   
   * coord reverse resolution returns no result
   * coord reverse resolution returns a single result
   * coord reverse resolution returns several results

---++++ Impact

   * oiexplorer-core: resolutions, error reporting

---++++ Notes


---+++ Instrument mode resolution

Purpose: define identifier for instrument mode

Input: INSNAME (string), MIN_WL (double), MAX_WL (double), NB_CHANNEL (double), knowledge database for instrument modes %BR%
Output: INSMODE (string)

Build a fingerprint/hash to get an instrument mode using a knowledge database of instrument modes.
This knowledge database may be the configuration file of ASPRO2 or an extract of OiDB and build along data imports.

---++++ Scenarii

   * fingerprints with slightly different INSNAME values (e.g. wavelengths min and max as in PIONIER_Pup(1.5994200/1.8035799) )
   * instrument names with constant values (e.g. VEGA )
   * unknown instrument mode
   * unknown instrument

---++++ Impact

   * oiexplorer-core: build a fingerprint/hash
   * oidb: match fingerprint/hash with instrument mode

---++++ Notes

   * footprint definition (single string, fields concatenation...)?
   * how to solve instrument modes ambiguity (e.g. for MIDI and MATISSE how to choose between SCI_PHOT_LM_LOW or HIGH_SENS_LM_LOW  )

---+++ Group by nights

Purpose: associate a night identifier to observations

Input: MJD (double), ARRAY_NAME (string)
Output: NIGHT_ID

Find which night corresponds to the given MJD at the array location.

---++++ Scenarios

   * single night
   * multiple nights (contiguous or not)

---++++ Impact

   * jmal: night boundary at location,date
   * oiexplorer-core: compute NIGHT_ID


---++ General

   * do we have to refresh/compare data retrieved by simbad
Topic revision: r6 - 2014-04-09 - GuillaumeMella