87 lines
3.1 KiB
Markdown
87 lines
3.1 KiB
Markdown
|
# Eclipse Cyclone DDS Multi Process Testing Requirements
|
||
|
|
|
||
|
|
This document present some requirements and considerations regarding the
|
||
|
|
[Multi Process Test Framework](multi_process_testing.md).
|
||
|
|
|
||
|
|
## Requirements
|
||
|
|
1.1) To test certain Cyclone DDS features, multiple processes running Cyclone
|
||
|
|
DDS are needed to force communication through the whole stack.
|
||
|
|
|
||
|
|
1.2) Should be buildable and runnable on platforms that support multiprocess
|
||
|
|
and filesystems including the ones used in the continues integration
|
||
|
|
context.
|
||
|
|
|
||
|
|
1.3) Results should be easily analyzable within the continues integration
|
||
|
|
context and when running locally. This can be done by reporting the
|
||
|
|
results in a standard format like xunit or cunit.
|
||
|
|
|
||
|
|
1.4) No processes must be left behind (f.i. deadlock in child process) when the
|
||
|
|
test finished (or timed out).
|
||
|
|
|
||
|
|
1.5) When running tests parallel, they should not interfere with each other.
|
||
|
|
|
||
|
|
1.6) Processes of the same test should be able to communicate (for settings,
|
||
|
|
syncing, etc).
|
||
|
|
|
||
|
|
1.7) It should be possible to analyze output/messages/tracing of the parent
|
||
|
|
and child processes to be able to draw a proper test conclusion.
|
||
|
|
|
||
|
|
|
||
|
|
## Considerations
|
||
|
|
2.1)
|
||
|
|
The files that actually contain the tests, should be focused on those tests.
|
||
|
|
This means that the process management and setting up (and usage of) IPC
|
||
|
|
between test processes should be handled by a test framework so that the
|
||
|
|
test files can remain as clean as possible.
|
||
|
|
|
||
|
|
2.2)
|
||
|
|
If possible, there shouldn't be a need for writing log files to a file system
|
||
|
|
when running the tests normally. It could be helpful, however, that these log
|
||
|
|
files are written when debugging related tests.
|
||
|
|
|
||
|
|
2.3)
|
||
|
|
Preferably, the DDS communication between the processes should not leave
|
||
|
|
localhost.
|
||
|
|
|
||
|
|
|
||
|
|
## Intentions
|
||
|
|
There doesn't seem to be a 3rd party test framework that addresses our
|
||
|
|
requirements in a satisfactory manner.
|
||
|
|
|
||
|
|
After some discussions with a few people (different people in different
|
||
|
|
meetings), it was decided to create our own framework and to go in the
|
||
|
|
following direction:
|
||
|
|
|
||
|
|
- Process creation/destruction/etc is (re)introduced in the ddsrt.<br>
|
||
|
|
[1.1/1.2]
|
||
|
|
|
||
|
|
- The files that contain the tests, should be easy to understand and focus on
|
||
|
|
the tests themselves.<br>
|
||
|
|
[2.1]
|
||
|
|
|
||
|
|
- Other files (generated or in the framework) should take care of the
|
||
|
|
intricacies of starting/monitoring the proper processes with the proper
|
||
|
|
settings.<br>
|
||
|
|
[1.4/1.6/2.1]
|
||
|
|
|
||
|
|
- To do this, a similar approach of the current CUnit build will be used;
|
||
|
|
CMake will scan the test files and create runners according to macros within
|
||
|
|
the test files.<br>
|
||
|
|
[2.1]
|
||
|
|
|
||
|
|
- The tests should be executed by CTest. For now this means that a proper
|
||
|
|
runner exit code for pass/fail is enough. We would like to add CUnit like
|
||
|
|
output in the future.<br>
|
||
|
|
[1.2/1.3]
|
||
|
|
|
||
|
|
- The Cyclone DDS API contains the possibility to monitor generated log traces.
|
||
|
|
This means we won't be needing to monitor actual log files. Just register a
|
||
|
|
log callback and go from there.<br>
|
||
|
|
[1.7/2.2]
|
||
|
|
|
||
|
|
- The framework should be able to generate unique domain ids and unique topic
|
||
|
|
names when necessary. That way, tests won't interfere with each other when
|
||
|
|
running in parallel.<br>
|
||
|
|
[1.5]
|
||
|
|
|