* Add PropertyPolicy to QoS API for Security settings
This commit adds the public API for PropertyQosPolicy including
tests. This policy can be used to set the parameters for the DDS security
implementation, as an alternative for using the xml configuration.
Tests are also inlcuded for setting security properties and conflict
resolving when both security configuration and qos properties are present.
Finally, the pubsub tool is updated so that is handles this qos correctly.
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
* Init binary_value.props to fix failing qos merge and moved init code in qset_prop functions
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
* Add additional test and some validation improvements based on review comments
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
* Refactoring of qos property
Refactored the qos property handling based on review comments. Setting
and unsettings functions are simplified and now use helper functions
for lookup, property initialisation is simplified. Added an additional
check for required security properties when creating participant using
security settings from qos, and added a test-case for this code.
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
* Check for qos flag before getting property index from qos
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
* Participant creation should fail on inconsistent security qos properties, and some minor code improvements in property qos api functions
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
* Update log message in test security_config_qos
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
* Fixed unused label compiler error in q_entity.c when security is disabled
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
* Refactored qprop functions with macros to avoid code duplicate code
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
CMAKE_PROJECT_NAME refers to the top-level project name, not the most recent project. So any CMake project that pulls this in as a dependency was in for a nasty surprise.
https://cmake.org/cmake/help/latest/variable/CMAKE_PROJECT_NAME.html
Signed-off-by: Dan Rose <dan@digilabs.io>
The ``docs/makernc.pl`` perl script can extract the options from the
source code for the configuration processing and turn these into a
GitHub MarkDown document and a RELAX NG Compact form (RNC) description
of the options. Standard conversion tools can be used to turn the RNC
file into an XSD.
Although they are generated by a mechanical transformation, it is not
(yet) part of the build process, in no small part because there no
proper generation and publishing system has yet been set up for the
documentation.
Now that there is an XSD and a Markdown file for discovering the
options, there is no need to retain the configuration editing tool. Any
decent editor will do a better job.
Signed-off-by: Erik Boasson <eb@ilities.com>
Change the structure of the configuration file (in a backwards
compatible manner) to allow specifying configurations for multiple
domains in a file. (Listing multiple files in CYCLONEDDS_URI was
already supported.) A configuration specifies an id, with a default of
any, configurations for an incompatible id are ignored.
If the application specifies an id other than DDS_DOMAIN_DEFAULT in the
call to create_participant, then only configuration specifications for
Domain elements with that id or with id "any" will be used. If the
application does specify DDS_DOMAIN_DEFAULT, then the id will be taken
from the first Domain element that specifies an id. If none do, the
domain id defaults to 0. Each applicable domain specification is taken
as a separate source and may override settings made previously.
All settings moved from the top-level CycloneDDS element to the
CycloneDDS/Domain element. The CycloneDDS/Domain/Id element moved to
become the "id" attribute of CycloneDDS/Domain. The old locations still
work, with appropriate deprecation warnings.
Signed-off-by: Erik Boasson <eb@ilities.com>
* Change default behaviour with no arguments to print help text;
"ddsperf sanity" now gives the behaviour that it used to give when run
with no arguments;
* Include context switch rate in output;
* Allow suffixing frequencies and sizes with standard units (so "size
1kB" is now allowed);
* Add missing option to help text, extend help text with some additional
informationr.
Signed-off-by: Erik Boasson <eb@ilities.com>
* per-thread CPU usage (only those threads where the load is over 0.5%,
if the sum of threads below that threshold exceeds 0.5%, it prints an
aggregate for those threads);
* also report RSS;
* network load (only on request, as percentage of specified network
bandwidth and actual bytes in/out, with the output suppressed if it is
0%);
* publish CPU usage so a ddsperf instance can display CPU loads for
its peers;
* handle SIGXFSZ (file size exceeded) by displaying one last line of
statistics before killing itself; this simply a debugging tool to make
it easier to get a trace covering a high sample-rate start-up issue;
* default topic changed to "KS" because that allows all the options to
be used, this has a negative impact on performance (both latency and
small-sample throughput) but it should be less surprising to users;
* specifying a size is now done by appending "size N" (where N is the
size in bytes) after a "ping" or "pub" command, rather than it having
to set it via a command-line option;
Note that some of this is platform-dependent -- SIGXFSZ is currently
only on Linux and macOS, and CPU and network load reporting is currently
only on Linux, macOS and Windows.
Signed-off-by: Erik Boasson <eb@ilities.com>
* Move the project top-level CMakeLists.txt to the root of the project;
this allows building Cyclone as part of ROS2 without any special
tricks;
* Clean up the build options:
ENABLE_SSL: whether to check for and include OpenSSL support if a
library can be found (default = ON); this used to be
called DDSC_ENABLE_OPENSSL, the old name is deprecated
but still works
BUILD_DOCS: whether to build docs (default = OFF)
BUILD_TESTING: whether to build test (default = OFF)
* Collect all documentation into top-level "docs" directory;
* Move the examples to the top-level directory;
* Remove the unused and somewhat misleading pseudo-default
cyclonedds.xml;
* Remove unused cmake files
Signed-off-by: Erik Boasson <eb@ilities.com>
It is an excellent platform for catching bugs: big-endian, slow enough
that a context switch in the middle of an operation becomes a regular
occurrence, and all that on a SMP box. Or: I just wanted to see if it
would work.
Signed-off-by: Erik Boasson <eb@ilities.com>
The two do essentially the same think, and ddsrt_strtok_r was only used
in one place. (Triggered by Solaris 2.6 not providing strtok_r.)
Signed-off-by: Erik Boasson <eb@ilities.com>
Tracking pings and expected number of pongs was done without holding the
correct locks. Terminate flag was also not a ddsrt_atomic... and hence
flagged by thread sanitizer as a race condition.
Signed-off-by: Erik Boasson <eb@ilities.com>
Missing prototypes for exported functions cause a really huge issue on
Windows. Enabling the "missing prototypes" warning makes it much easier
to catch this problem. Naturally, any warnings caused by this have been
fixed.
Signed-off-by: Erik Boasson <eb@ilities.com>
* Remove dds_return_t / dds_retcode_t distinction (now there is only
dds_return_t and all error codes are always negative)
* Remove Q_ERR_... error codes and replace them by DDS_RETCODE_...
ones so that there is only one set of error codes
* Replace a whole bunch "int" return types that were used to return
Q_ERR_... codes by "dds_return_t" return types
Signed-off-by: Erik Boasson <eb@ilities.com>
* As a simple matter of code hygiene, in particular to aid in checking for
leaks, ddsperf should free all memory it allocates on exit.
* Remove spurious mutex unlock in ddsperf
* Removing a participant means removing one or two entries from the "pong
writers" array ("pong wr"), and there it read 1 element beyond the end
of the array while moving the remaining elements forward.
* Constant-rate pinging was broken because of two reasons, one worse than
the other:
* setting the rate had a mismatch in variables (publication rate and
command-line argument) resulting in a completely wrong ping interval;
the code now has a bit more clear variable naming ...
* the timing of the pings was relative to the current time, but the
wakeup a little delayed, resulting in a lower rate than requested.
It now simply adds the ping interval to the scheduled ping time, rather
than the time at which the ping is being sent. To guard against really
late wakeups, rates that are too high, suspending the machine, &c. it
will in extremis delay the next ping.
Signed-off-by: Erik Boasson <eb@ilities.com>
Multiplying time-in-ns since previous output line by 1e9 instead of
dividing it by 1e9 resulted in bit rate showing up as 0Mb/s.
Signed-off-by: Erik Boasson <eb@ilities.com>
The current situation for performance measurements and checking network
behaviour is rather unsatisfactory, as the only tools available are
``pubsub`` and the ``roundtrip`` and ``throughput`` examples. The first
can do many things thanks to its thousand-and-one options, but its
purpose really is to be able to read/write arbitrary data with arbitrary
QoS -- though the arbitrary data bit was lost in the hacked conversion
from the original code. The latter two have a terrible user interface,
don't perform any verification that the measurement was successful and
do not provide the results in a convenient form.
Furthermore, the abuse of the two examples as the primary means for
measuring performance has resulted in a reduction of their value as an
example, e.g., they can do waitset- or listener-based reading (and the
throughput one also polling-based), but that kind of complication does
not help a new user understand what is going on. Especially not given
that these features were simply hacked in.
Hence the need for a new tool, one that integrates the common
measurements and can be used to verify that the results make sense. It
is not quite done yet, in particular it is lacking in a number of
aspects:
* no measurement of CPU- and network load, memory usage and context
switches yet;
* very limited statistics (min/max/average, if you're lucky; no
interesting things such as jitter on a throughput test yet);
* it can't yet gather the data from all participants in the network
using DDS;
* it doesn't output the data in a convenient file format yet;
* it doesn't allow specifying boundaries within which the results
must fall for the run to be successful.
What it does verify is that all the endpoint matches that should exist
given the discovered participant do in fact come into existence,
reporting an error (and exiting with an exit status code of 1) if they
don't, as well as checking the number of participants. With the way the
DDSI protocol works, this is a pretty decent network connectivity check.
The raw measurements needed for the desired statistics (apart from
system-level measurements) are pretty much made, so the main thing that
still needs to be done is exploit them and output them. It can already
replace the examples for most benchmarks (only the 50%/90%/99%
percentiles are still missing for a complete replacement).
Signed-off-by: Erik Boasson <eb@ilities.com>
The old entity handle mechanism suffered from a number of problems, the
most terrible one being that it would only ever allocate 1000 handles
(not even have at most 1000 in use at the same time). Secondarily, it
was protected by a single mutex that actually does show up as a limiting
factor in, say, a polling-based throughput test with small messages.
Thirdly, it tried to provide for various use cases that don't exist in
practice but add complexity and overhead.
This commit totally rewrites the mechanism, by replacing the old array
with a hash table and allowing a near-arbitrary number of handles as
well as reuse of handles. It also removes the entity "kind" bits in the
most significant bits of the handles, because they only resulted in
incorrect checking of argument validity. All that is taken out, but
there is still more cleaning up to be done. It furthermore removes an
indirection in the handle-to-entity lookup by embedding the
"dds_handle_link" structure in the entity.
Handle allocation is randomized to avoid the have a high probability of
quickly finding an available handle (the total number of handles is
limited to a number much smaller than the domain from which they are
allocated). The likelihood of handle reuse is still dependent on the
number of allocated handles -- the fewer handles there are, the longer
the expected time to reuse. Non-randomized handles would give a few
guarantees more, though.
It moreover moves the code from the "util" to the "core/ddsc" component,
because it really is only used for entities, and besides the new
implementation relies on the deferred freeing (a.k.a. garbage collection
mechanism) implemented in the core.
The actual handle management has two variants, selectable with a macro:
the preferred embodiment uses a concurrent hash table, the actually used
one performs all operations inside a single mutex and uses a
non-concurrent version of the hash table. The reason the
less-predeferred embodiment is used is that the concurrent version
requires the freeing of entity objects to be deferred (much like the
GUID-to-entity hash tables in DDSI function, or indeed the key value to
instance handle mapping). That is a fair bit of work, and the
non-concurrent version is a reasonable intermediate step.
Signed-off-by: Erik Boasson <eb@ilities.com>
- Replace os_result by dds_retcode_t and move DDS return code defines down.
Eliminates the need to convert between different return code types.
- Move dds_time_t down and remove os_time.
Eliminates the need to convert between different time representations and
reduces code duplication.
- Remove use of Microsoft source-code annotation language (SAL).
SAL annotations are Microsoft specific and not very well documented. This
makes it very difficult for contributers to write.
- Rearrange the abstraction layer to be feature-based. The previous layout
falsely assumed that the operating system dictates which implementation is
best suited. For general purpose operating systems this is mostly true, but
embedded targets require a slightly different approach and may not even offer
all features. The new layout makes it possible to mix-and-match feature
implementations and allows for features to not be implemented at all.
- Replace the os prefix by ddsrt to avoid name collisions.
- Remove various portions of unused and unwanted code.
- Export thread names on all supported platforms.
- Return native thread identifier on POSIX compatible platforms.
- Add timed wait for condition variables that takes an absolute time.
- Remove system abstraction for errno. The os_getErrno and os_setErrno were
incorrect. Functions that might fail now simply return a DDS return code
instead.
- Remove thread-specific memory abstraction. os_threadMemGet and accompanying
functions were a mess and their use has been eliminated by other changes in
this commit.
- Replace attribute (re)defines by ddsrt_ prefixed equivalents to avoid name
collisions and problems with faulty __nonnull__ attributes.
Signed-off-by: Jeroen Koekkoek <jeroen@koekkoek.nl>
Make function names more consistent:
* ``dds_topic_{get,set}_filter`` -> ``dds_{get,set}_topic_filter``
* ``dds_{get,set}_enabled_status`` -> ``dds_{get,set}_status_mask``
* ``dds_instance_lookup`` -> ``dds_lookup_instance``
Plus removal of defunct ddsv2.h.
The main naming scheme is OPER_TYPE (i.e., dds_create_participant) but the listener operations were named TYPE_OPER (i.e., dds_listener_create). This commit brings the listener scheme in line with the rest, retaining the old names as a deprecated interface.
Signed-off-by: Erik Boasson <eb@ilities.com>
The main naming scheme is OPER_TYPE (i.e., dds_create_participant) but the QoS operations were named TYPE_OPER (i.e., dds_qos_create). This commit brings the QoS scheme in line with the rest, retaining the old names as a deprecated interface.
Signed-off-by: Erik Boasson <eb@ilities.com>
Automatic batching is not yet implemented but: it dramatically enhances performance in throughput tests, and enabling it in the config breaks manual publication of data in pubsub. For the time being, it makes more sense to enable manual batching programmatically and force the data out when needed.
Signed-off-by: Erik Boasson <eb@ilities.com>
