OiDbChara < Jmmc/Software

Tags: view all tags
%TOC%

---+ Overview

Theo ten Brummelaar is working on the definition of a file for publishing the observation logs from the CLIMB and CLASSIC instruments at [[http://www.chara.gsu.edu/][CHARA]].

---+ Data submission

Observation logs are provided in a comma-separated-value (CSV) file, one row per observation.

An XQuery module is in charge of extracting the data and creating a OiDB granule for each observation.

---+ Data mapping

Each line (apart from header) is parsed as a CSV record and fields are imported as L0 data in OiDB.

The following rules are followed for mapping columns from the file row to the database schema of OiDB (based on the the [[http://www.jmmc.fr/twiki/bin/view/Jmmc/Software/OiDbColumnsL0][!ObsCore model]]):

   * the rows are currently imported as part of a collection named "CHARA Import" (=obs_collection=)
   * the names from the PI field are split and the first one is put in the =datapi=
   * the ID from the PROGRAM field is copied to the =progid=
   * the coordinates are resolved with Simbad from the name in the STAR column
   * the observation start and stop times (=t_min=, =t_max=) are set to the value of the MJD column 

   * the name of the facility used for the observation is set to 'MtW.CHARA' (from ever-growing list of facilities - ADS Web Services)
   * the name of the instrument (=instrument_name=) is taken from the COMBINER column 
   * the name of the instrument mode (=instrument_mode=) is taken from the FILTER column
   * the COMBINER and FILTER are used for searching the min (=em_min=) and max (=em_max=) wavelengths in ASPRO configuration files

   * the data URL (=access_url=) is left empty
   * the total exposure time (=t_exptime=) is not filled
   * the temporal resolution (=t_resolution=) is not filled
   * the spectral resolving power (=em_res_power=) is not filled
   * the channel count (=nb_channel=) is not filled
   * the counts for VIS (=nb_vis=), VIS2 (=nb_vis2=) and T3 (=nb_t3=) measurement is not filled 

   * the granule is marked as proprietary (=data_rights=) with no release date

The TYPE column (OBJ, LAB, CAL), SCOPES, B1, B2, B3 and T0 are currently not mapped.

If the target can not be resolved (unknown by Simbad) or the mode can not be found in ASPRO configuration files, no granule is saved and a warning is logged.

---+ Usage

The operation is triggered from a form on the !OiDB backoffice to upload a composite CSV file of all observations. The process is asynchronous.

The resulting granules are attached to the "CHARA Import" collection (=obs_collection= = chara_import). The full collection is rebuilt and old granules are dropped on a successful attempt at updating the data.

---+ Notes

Discussion for nightly or weekly updates on new observations is ongoing. The exchange format will likely be the same (CSV fragment) in files downloaded from CHARA. 2008-2014 observations are provided as one file for each year (only 2008-2013 currently available).

   * [[%ATTACHURL%/climbclassic.csv][climbclassic.csv]]: Observation logs example as of 2014-08-05
Attachments
Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
csv	climbclassic.csv	r1	manage	0.7 K	2014-08-05 - 09:12	PatrickBernaud	Observation logs example as of 2014-08-05
Topic revision: r6 - 2014-11-18 - PatrickBernaud