ODM Output Precision Control

Not seeing a way to control output precision for CCSDS ODM messages. Can that be done or could that control be added? Orbit Data Messages (ccsds.org) section 7.5.6 and 7.5.7 limit maximum precision, but not seeing a way to do that through OemWriter and related classes.

7.5.6 Non-integer numeric values expressed in fixed-point notation shall consist of a
sequence of decimal digits separated by a period as a decimal point indicator, with an
optional leading sign (‘+’ or ‘-’). If the sign is omitted, ‘+’ shall be assumed. Leading and
trailing zeroes may be used. At least one digit shall appear before and after a decimal point.
The number of digits shall be 16 or fewer.
7.5.7 Non-integer numeric values expressed in floating point notation shall consist of a
sign, a mantissa, an alphabetic character indicating the division between the mantissa and
exponent, and an exponent, constructed according to the following rules:
a) The sign may be ‘+’ or ‘-’. If the sign is omitted, ‘+’ shall be assumed.
b) The mantissa must be a string of no more than 16 decimal digits with a decimal point
(‘.’) in the second position of the ASCII string, separating the integer portion of the
mantissa from the fractional part of the mantissa.

Hi @bdkelly welcome!

You are right, there is a problem here, we missed this limitation of the CCSDS format and the workaround is not straightforward.
Internally, the output of double values is performed by the Generator passed to the writeMessage method (or the underlying writeHeader, writeSegment, writeFooter and similar methods). Orekit provides two implementations of Generator: KvnGenerator and XmlGenerator, both extending a general purpose AbstractGenerator class. It is AbstractGenerator that converts the double values in SI to strings taking the CCSDS units into account. It does it be delegating the conversion to strings to the Ryū algorithm. This design choice was deliberate as it ensures proper round-trip operations while still ensuring shortest string representation, i.e. when someone parses the value that has been written, one get the exact same double number. This means that despite we use a character-based intermediate message, we don’t lose any accuracy, which is something I have experienced a lot and that induces a lot of problems when doing accurate computation.

Ensuring round-trip safety means that for some values, a lot of digits may be required, and sometimes the shortest representation is not that short. In fact, with 64 bits numbers following IEEE754 binary64 representation (which is what Java primitive double do), it is known that round-trip safety is always ensured if one uses 17 decimal digits (for the record, this is called the p10 value and is equal to 9 digits for the binary32 representation, 17 digits for the binary64 representation and 36 digits for the binary128 representation, as explained in section 3.1.5 of Muller et al. Handbook of Floating-Point Arithmetic), so we know the output of Ryū algorithm will always have 17 digits at most… which is one digit more than allowed by CCSDS! This is a pity, and I would say it is an overlook of the CCSDS standard which probably did not know about p10 being 17 for double numbers (it is not widely known, many people think double numbers are 15 or 16 digits accurate and don’t know the subtlety of round-trip safety, also known as error-free write-read cycle in Muller’s book).

So, how can we handle that?

I first will notify the problem to CCSDS, telling them that the limitation to 16 digits hinders round-trip safety but I think they will not change the 16 into 17 as it is most probably used everywhere in many CCSDS recommendations.

You could open an issue in our issue tracker referring to this forum thread, so we implement a global fix.

You may try to set up a workaround by setting up your own implementation of Generator, perhaps by extending either KvnGenerator or XmlGenerator and overriding the doubleToString method they originally inherit from AbstractGenerator. The current implementation in AbstractGenerator reads:

    public String doubleToString(final double value) {
        return Double.isNaN(value) ? null : AccurateFormatter.format(value);
    }

Maybe you can try something like:

    public String doubleToString(final double value) {
        if (Double.isNaN(value)) {
             return null;
        } else {
             return String.format(Locale.US,
                                  (value < 1.0e-3 || value > 1.0e7) ?  "%.16e" : "%.16f",
                                  value);
       }
    }

Warning: I did not try this suggestion myself, it may be wrong, I just wrote this code directly in the forum without any test.

Thanks Luc. In your comment to CCSDS, the fixed-point section is particularly problematic, as the Ryu will leave 16 or 17 significant digits, which may get up to 20 digits to the right of the decimal or 21 total in output I’ve seen, which is quite a bit more than the stated limit of 16 digits, if taken literally. If they’ll go for 17, see if they would be willing to state it as up to 17 significant digits.

Captured Can’t truncate to CCSDS ODM-specified digits (#1297) · Issues · Orekit / Orekit · GitLab.

I am not sure the 16 digits limitation counts for the complete field, I would think it counts only for the significant digit, i.e. the mantissa. Otherwise, one can have to 7 more characters in the field than in the mantissa (mantissa sign, mantissa decimal separator, exponent marker, exponent sign, 3 digits for exponents), so considering 16 significant digits, one would expect up to 23 characters wide field, or if we limit to 16 characters for the complete field, the mantissa reduces to 9 digits, which is clearly not very accurate.

Agree completely, but the wording The number of digits shall be 16 or fewer leaves it open to a less accurate interpretation by adopters. If CCSDS is willing to change that section, clarification should be made.

FYI, we found the OemWriter is calling the AccurateFormatter directly, which seems a little different than the Opm and OmmFormatters, so we are attempting to extend that class as well as a workaround. At the moment, we are pursuing %.15e globally, due to the fixed point digit language in the CCSDS.