This page gathers the enhancements and current limitations on the JMMC OITools library focused on the OIFITS data handling.

Its main features are:

• read / write OIFITS files (V1 / V2 + IMAGE_OI extensions = OImaging)
• read / write FITS image(s) and cubes (primary HDU, extra Image HDU)
• data model description: meta-data for keywords, columns, tables
• in-memory data structures to deal with tables (columns, keywords) and derived columns (hard-coded & expression support)
• validation (command-line interface + web interface)
• serialization in TSV / XML formats (used by oidb)

Enhancements

The roadmap

• Version 2 of OIFTIS is set up, we are able to analyze fits files in version 2. We are also always able to analyze files fits in version 1. It is the program itself which analyzes the nature of the file and that adapts to it. We must continue to improve our API by making the necessary modifications, the current goal is to fix minor bugs and improve if possible part of the code to make it more robust and efficient.
• We have already defined our table of rules and DataModel, we must now prepare it for a presentation to the scientists.
• Personal presentation: prepare the presentation of my work on OITools and the explanations of my involvement.
• Review the project documentation to see if we are always in the right direction, what are still the projects to be carried out. We still have to make some big changes.
• It would be nice to have the first feedback for OIFits version 2 of the users pard
• Validation item

The rules for OIFits Files

• The table for all rules (and apply to) has been created for DataModel version 1 and version 2

FeedBack OIFits2 Standard:

• We have different synthax for keyword %s, %f, %d for header keyword.

Actual TODO:

• Functional lack of V2: Unit (Any_Units), Checksum and Datasum, management of dimensions (NWaves)
• TODO: Headercards type is string for Int/float/double too
• TODO: imageHDU should be returned even if no image to hold keywords -> could rely on the firstOnly flag ?
• Improve the OIFitsValidator test to compare the validation results
• Finish adding any missing rules
• Nom.tam.fits update
• Github
• take stock of all TODO, point by point
• sticky notes

Junit Tests

• TODO: describe the JUnit tests + file structure
• TODO: Test update ? (History order) / (Hierarch)
  • see strict compare Nom.tam.fits

Copy files

Objective:

• Load & write reference FITS (images) & OIFits (V1, V2, with IMAGE_OI extensions) files (OK)
• Compare the OIFitsFile structures (all tables, all keywords, all columns, header cards) (OK)
• Compare the nom.tam.Fits (raw HDUs):

TODO: FIX the lenient comparison mode (strict is too exhaustive) to ensure the copied files have not lost or modified original keyword / data

See README in the src/test/resource folder describing the reference files

Improve the OIFITS validator (V1 / V2)

• improve the validation results:
  • extract the severity, origin (table/keyword or column) from the validation message
  • sort & filter validation results

TEST UPDATE NOM.TAM.FITS

Problem 1: We have multi min/max methodes

• In several file we have methode for calculate min/max value
• for exemple : oifitsFile.getMinWavelengthBound() and oiWavelength.getEffWaveMin()
• We need just 1 method to compute all kind of bound on any kind of mesurements (see also xmlOutputVisitor.appendRecord() )

Problem 2: We need to change Tam Fits Test for scrict comparison

• Currently we ignore some error for the strict test or we need to fix that
• We need to chose the best chose fix or ignore error, depending on severity.

See: oitools/src/test/resources/ref/WriteOIFitsTest.log

Ignore case:

• ARRAYXYZ: 123.0 vs 123.
• NAXIS1 = row size different (taille des String)
• TFORM = Data format
  • different value of header card[TFORM2] '6A' <> '16A'
  • different value of header card[TFORM3] 'I' <> '1I'

Extra Tables

Binary Tables that are not in the OIFits standards (V1 or V2) are totally skipped (OIFitsLoader? / OIFitsWriter? )

• Consequence: Copying a FITS file may loose 'unknown' tables (load then write) !
• Solution: When loading such a table, it seems possible to parse its header and create keyword / column descriptors on the fly then load it as usual (GenericTable extends Table)

Extra Columns in table

• Try to detect the displaced columns ... => avoid false positives

1. file: PRODUCT_V838_Mon_1-copy.90-2.53micron_2013-04-15T01_49_24.8034--------0.00000------------inf--------1--------2--------3

• keywords:
  • OI_VIS Missing header card TTYPE15 was = VISCOVRI
  • OI_VIS Missing header card TFORM15 was = 16D
• Columns:
  • different number of columns 17 <> 14 in VISCOVRI

1. file: GRAVITY-copy.2016-01-09T05-37-06_singlesci_calibrated

• different number of columns 16 <> 14 in ASTROMETRIC_DELAY
• different number of columns 3 <> 2 in POLAR

• OI_VIS Missing header card TTYPE15 was = ASTROMETRIC_DELAY
• OI_VIS Missing header card TTYPE15 was = VISCOVRI

• OI_T3 different number of header card 50 <> 52

Dimensions (TDIM)

1. file: AMBER_070409-copy

• OI_ARRAY Missing header card TDIM5 was = (3)
• OI_VIS Missing header card TDIM7 was = (20)
• different values for column[VISDATA]
• different values for column[VISERR]

See OIFits V2 standard that mentions how to use the TDIM keyword

Problem 6: Setters API

To easily fill tables (complex keywords, columns), a simple setter API will be implemented:

/**
* Define the column value for the given row (index)
*/
(OITable).setColumn(int row, (Type) value)

/**
* Define all column values for the given row (index)
*/
(OITable).setRow(int row, values...)

Limitations

Cube Image support: Fixed

Last version can load FITS image cubes from one ImageHDU but also write back to a FITS or OIFits file into one ImageHDU (with the proper wavelength keywords - AXIS3)

This use case is not yet required but next ones certainly will require code enhancement:

• support of polychromatic data in OImaging

External resources

• https://github.com/jsy1001/oifitslib provides some V2 test files