The Starting the build process section on the OpenIFS 48r1 Getting Started page shows how openifs-test can be used to test whether a suite of t21 case run successfully on a system. This is very useful to testing a successful build, basic simulations and the parallel set-up of the system. This testing does not, however, automatically test for bit identical results. This page presents some notes about how to perform such tests.
Bit comparison testing over multiple tests can be a very useful tool to ensure that any code changes only impact the results that they are expected to impact, while other results are unaffected. By default, the way we have run openifs-test so-far, only checks that an installation and/or code change builds and runs without any major runtime errors; it does not perform bit-comparison testing against a reference set of known-good output (KGO).
OpenIFS does not include a reference set of known-good output (KGO) because the KGO is system and compiler dependent. Hence, in order to perform a bit identical it is necessary to create the KGO from, ideally you a "known-good" source before making any changes to the source code. KGO can be created by running the test with the environment variable IFS_TEST_BITIDENTICAL=init:
...