This page gathers the enhancements and current limitations on the JMMC OITools library focused on the OIFITS data handling.

Its main features are:

  • read / write OIFITS files (V1 / V2 + IMAGE_OI extensions = OImaging)
  • read / write FITS image(s) and cubes (primary HDU, extra Image HDU)
  • data model description: meta-data for keywords, columns, tables
  • in-memory data structures to deal with tables (columns, keywords) and derived columns (hard-coded & expression support)
  • validation (command-line interface + web interface)
  • CLI interface: validation & serialization in TSV / XML formats (used by OIDB)

Useful links

Feedback on the OIFits2 standard

  • Rule: Columns that are not in the standard must start with "NS_". What would be the severity of an unknown table column name not starting with "NS_"
    • Answer: Yes I agree. Not using "NS_" is harmless except when designing a later version of the standard.
  • Question: The types for the keywords of the primaryHDU are defined by %s, %d, %f. We should therefore redefine these types in the same way for tables (currently in the form A, I, D, ...).
    • Answer: The correction has been applied to the standard and the new version is available.
  • Question: For the new table OI_INSPOL we have columns MJD_OBS and MJD_END, do we really need the keyword DATE_OBS in addition?
    • Answer: The correction has been applied to the standard and the new version is available. The keyword DATE_OBS is removed from the OI_INSPOL table.
  • Question: For some keywords (AMPTYP, PHITYPE) we have in description no information but only the possible values.
    • Answer: The usefulness of these keywords is explained in the text, as well as the explanation of possible values.

To restore

Improve the OIFITS validator (V1 / V2)

  • improve the validation results:
    • extract the severity, origin (table/keyword or column) from the validation message
    • sort & filter validation results


Problem 1: We have multi min/max methodes

  • In several file we have methode for calculate min/max value
  • for exemple : oifitsFile.getMinWavelengthBound() and oiWavelength.getEffWaveMin()
  • We need just 1 method to compute all kind of bound on any kind of mesurements (see also xmlOutputVisitor.appendRecord() )

Problem 2: We need to change Tam Fits Test for scrict comparison

  • Currently we ignore some error for the strict test or we need to fix that
  • We need to chose the best chose fix or ignore error, depending on severity.

See: oitools/src/test/resources/ref/WriteOIFitsTest.log

See an example of difference in header for raw comparison in the attached file , describing differences in header usually seen with raw comparison option (Set .level=INFO in file
These diff are displayed in log generated in RAW mode. Look for key string "list srcHeader", "list dstHeader", following lines give content of cards.

enhancement possible: put cards content in java collection and process automatically the comparison (show only differences), method TamFitsTest.compareHeader().

Ignore case:

  • ARRAYXYZ: 123.0 vs 123.
  • NAXIS1 = row size different (taille des String)
  • TFORM = Data format
    • different value of header card[TFORM2] '6A' <> '16A'
    • different value of header card[TFORM3] 'I' <> '1I'

Extra Tables

Binary Tables that are not in the OIFits standards (V1 or V2) are totally skipped (OIFitsLoader? / OIFitsWriter? )

  • Consequence: Copying a FITS file may loose 'unknown' tables (load then write) !
  • Solution: When loading such a table, it seems possible to parse its header and create keyword / column descriptors on the fly then load it as usual (GenericTable extends Table)

Extra Columns in table

  • Try to detect the displaced columns ... => avoid false positives

  1. file: PRODUCT_V838_Mon_1-copy.90-2.53micron_2013-04-15T01_49_24.8034--------0.00000------------inf--------1--------2--------3
  • keywords:
    • OI_VIS Missing header card TTYPE15 was = VISCOVRI
    • OI_VIS Missing header card TFORM15 was = 16D
  • Columns:
    • different number of columns 17 <> 14 in VISCOVRI

  1. file: GRAVITY-copy.2016-01-09T05-37-06_singlesci_calibrated

  • different number of columns 16 <> 14 in ASTROMETRIC_DELAY
  • different number of columns 3 <> 2 in POLAR

  • OI_VIS Missing header card TTYPE15 was = ASTROMETRIC_DELAY
  • OI_VIS Missing header card TTYPE15 was = VISCOVRI

  • OI_T3 different number of header card 50 <> 52

Dimensions (TDIM)

  1. file: AMBER_070409-copy

  • OI_ARRAY Missing header card TDIM5 was = (3)
  • OI_VIS Missing header card TDIM7 was = (20)
  • different values for column[VISDATA]
  • different values for column[VISERR]

See OIFits V2 standard that mentions how to use the TDIM keyword

Problem 6: Setters API

To easily fill tables (complex keywords, columns), a simple setter API will be implemented:

* Define the column value for the given row (index)
(OITable).setColumn(int row, (Type) value)

* Define all column values for the given row (index)
(OITable).setRow(int row, values...)

External OIFITS resources

Topic attachments
I Attachment History Action Size Date Who Comment
Unknown file formatres diffHeader.res r1 manage 40.4 K 2018-06-29 - 08:42 VincentJammet difference in header for raw comparison
Edit | Attach | Watch | Print version | History: r32 < r31 < r30 < r29 < r28 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r32 - 2018-06-29 - VincentJammet
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback