cyclonedds/docs/dev/mpt_req.md

Eclipse Cyclone DDS Multi Process Testing Requirements

This document present some requirements and considerations regarding the Multi Process Test Framework.

Requirements

1.1) To test certain Cyclone DDS features, multiple processes running Cyclone DDS are needed to force communication through the whole stack.

1.2) Should be buildable and runnable on platforms that support multiprocess and filesystems including the ones used in the continues integration context.

1.3) Results should be easily analyzable within the continues integration context and when running locally. This can be done by reporting the results in a standard format like xunit or cunit.

1.4) No processes must be left behind (f.i. deadlock in child process) when the test finished (or timed out).

1.5) When running tests parallel, they should not interfere with each other.

1.6) Processes of the same test should be able to communicate (for settings, syncing, etc).

1.7) It should be possible to analyze output/messages/tracing of the parent and child processes to be able to draw a proper test conclusion.

Considerations

2.1) The files that actually contain the tests, should be focused on those tests. This means that the process management and setting up (and usage of) IPC between test processes should be handled by a test framework so that the test files can remain as clean as possible.

2.2) If possible, there shouldn't be a need for writing log files to a file system when running the tests normally. It could be helpful, however, that these log files are written when debugging related tests.

2.3) Preferably, the DDS communication between the processes should not leave localhost.

Intentions

There doesn't seem to be a 3rd party test framework that addresses our requirements in a satisfactory manner.

After some discussions with a few people (different people in different meetings), it was decided to create our own framework and to go in the following direction:

• Process creation/destruction/etc is (re)introduced in the ddsrt.
  [1.1/1.2]

• The files that contain the tests, should be easy to understand and focus on the tests themselves.
  [2.1]

• Other files (generated or in the framework) should take care of the intricacies of starting/monitoring the proper processes with the proper settings.
  [1.4/1.6/2.1]

• To do this, a similar approach of the current CUnit build will be used; CMake will scan the test files and create runners according to macros within the test files.
  [2.1]

• The tests should be executed by CTest. For now this means that a proper runner exit code for pass/fail is enough. We would like to add CUnit like output in the future.
  [1.2/1.3]

• The Cyclone DDS API contains the possibility to monitor generated log traces. This means we won't be needing to monitor actual log files. Just register a log callback and go from there.
  [1.7/2.2]

• The framework should be able to generate unique domain ids and unique topic names when necessary. That way, tests won't interfere with each other when running in parallel.
  [1.5]