I was doing some testing with the 10.3 release recently comparing the reference frame stability with previous releases of Orekit. (Arguably I should have done this before 10.3 was released.) I was quite surprised when I found differences of 50 cm when comparing ephemerides generated with previous versions of Orekit. These differences seemed to be much larger than the assumptions in Orekit’s frames, even with simpleEop. Investigating further the discrepancies only seemed to appear when comparing ephemerides in OEM format, so perhaps not a frame issue at all. The 10.3 change log didn’t mention any backward incompatibilities, neither did the release vote. On further investigation it seems to be due to the changes for #686 (user defined formats in OEM) changing behavior by discarding significant digits by default, with no way to regain the previous behavior.
The point of this message is not the past. (I’m assuming the contributors had the best interest of the Orekit community in mind.) The point of this message is instead to see if we can improve the process for the future. I think Orekit’s current policy of not being too strict with compatibility leaves much undefined. Are changes in behavior with the same interface ok? Is it the developer’s responsibility to notify about incompatibilities or the user’s responsibility to figure out what changed?
I think it is good for Orekit that we are not too strict with compatibility. I also think it would be good to have some guidelines about communicating about incompatibilities in minor releases so that the community and contributor can know their responsibilities. Perhaps a good guideline would be to note incompatible changes as such in the change log, release vote, and release announcement. That way there will be better communication throughout the community about them and the community can discuss them before the release.
I would say it depends… Loosing accuracy as in the example you cite is bad (but was most certainly not foreseen, otherwise it would have been discussed). Changing behavior by increasing accuracy (which is expected to occur with the new Ryū algorithm) is probably acceptable even on minor releases.
Yes, but developers can make mistakes.
I don’t think many people read the change logs before release has been completed. I would say the forum is the place to discuss this and decide together on a case-by-case basis if an incompatibility is minor enough to be included in a minor release or if it should wait.
I was in charge of supervising the person who introduced the possibility of changing the OEM writing format and I didn’t pay enough attention to the repercussions. I apologize for that.
The problem is that the default formats do not allow for enough precision.
This problem will be fixed in 11.0 with the introduction of Ryu’s algorithm. However, if you think this is necessary, I can manage a 10.3.1 release.
Bryan, you’ve done so much of the work for 10.3 I don’t think there is anything to apologize for. (I was trying to avoid casting blame.) As Luc mentioned we’re all human.
I don’t think a 10.3.1 release is necessary with 11.0 coming out soon. Seems like it would be a big hassle for a relatively small problem.
So going forward it seems that bringing up incompatibilities on the forum is a consensus.
To catch more mistakes Orekit could implement a more formal peer review system, such as with GitLab merge requests, but I doubt the Orekit developers have time to devote to that. A while ago when I had more time I used to informally review many of the commits to Orekit, but that is rare now. So my sense is that we accept some mistakes in order to not slow down development.
