The participant listener creates a pong writer, setting a publication
matched listener on it. That listener can be invoked immediately and as
it queries the subscriptions reader, it must not be enabled before the
latter reader has been created.
Signed-off-by: Erik Boasson <eb@ilities.com>
* IPv6 extensions to patterns
* use full GUID prefix for Cyclone
* pattern fixes to deal with small changes in the formatting of QoS
* suppressinof local built-in topic publications
* asymmetrical disconnect detection improvements (better chance of
detecting it, plus better suppression of spurious notifications)
Signed-off-by: Erik Boasson <eb@ilities.com>
```
/opt/ros/master/src/eclipse-cyclonedds/cyclonedds/src/tools/pubsub/common.c:586:28: warning: implicit conversion from 'long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-int-float-conversion]
if(nanosec > nextafter(INT64_MAX, 0)) {
~~~~~~~~~ ^~~~~~~~~
/usr/include/stdint.h:134:22: note: expanded from macro 'INT64_MAX'
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/stdint.h:116:24: note: expanded from macro '__INT64_C'
^~~~~~
<scratch space>:345:1: note: expanded from here
9223372036854775807L
^~~~~~~~~~~~~~~~~~~~
1 warning generated.
```
Signed-off-by: Dan Rose <dan@digilabs.io>
This adds options to check for "unreasonable" RSS growth, receipt of a
minimum number of samples and having run a minimum number of roundtrips.
Signed-off-by: Erik Boasson <eb@ilities.com>
The status mask on some readers got reduced to just "data available"
when used in conjunction with a waitset, but the consequence is that the
"subscription matched" listener would be suppressed.
Signed-off-by: Erik Boasson <eb@ilities.com>
* Use the parameter tables to pretty-print QoS and plist, rather than a
hard-coded function supporting only the QoS.
* Support diffing two plists: a single table-driven function can handle
both nn_plist_t and ddsi_qos_t, and it removes the discrepancy between
the two types.
* Log content of discovery samples in trace rather than merely printing
"(plist)"
Signed-off-by: Erik Boasson <eb@ilities.com>
This commit adds support for liveliness QoS when using local readers.
The implementation for (liveliness) expiration of writers used here is
similar to that used with proxy writers, and it also supports the three
liveliness kinds (1) automatic, which is trivial when using a local
reader and writer, (2) manual-by-participant and (3) manual-by-topic.
In addition, these changes and fixes are included in this commit:
- Fixed a bug in heartbeat handling in the reader: for manual-by-
participant writers the lease was not updated on reception of a
heartbeat message with liveliness flag set. This is fixed and a
test-case is added.
- Include the liveliness flag in a heartbeat message to the trace
- Trace all lease renewals, including liveliness leases
- Replaced liveliness changed state 'twitch' by 2 subsequent calls
to the status callback
- Added a test for liveliness duration 0 and 1ns (for both local
and remote readers)
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
* Add PropertyPolicy to QoS API for Security settings
This commit adds the public API for PropertyQosPolicy including
tests. This policy can be used to set the parameters for the DDS security
implementation, as an alternative for using the xml configuration.
Tests are also inlcuded for setting security properties and conflict
resolving when both security configuration and qos properties are present.
Finally, the pubsub tool is updated so that is handles this qos correctly.
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
* Init binary_value.props to fix failing qos merge and moved init code in qset_prop functions
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
* Add additional test and some validation improvements based on review comments
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
* Refactoring of qos property
Refactored the qos property handling based on review comments. Setting
and unsettings functions are simplified and now use helper functions
for lookup, property initialisation is simplified. Added an additional
check for required security properties when creating participant using
security settings from qos, and added a test-case for this code.
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
* Check for qos flag before getting property index from qos
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
* Participant creation should fail on inconsistent security qos properties, and some minor code improvements in property qos api functions
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
* Update log message in test security_config_qos
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
* Fixed unused label compiler error in q_entity.c when security is disabled
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
* Refactored qprop functions with macros to avoid code duplicate code
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
CMAKE_PROJECT_NAME refers to the top-level project name, not the most recent project. So any CMake project that pulls this in as a dependency was in for a nasty surprise.
https://cmake.org/cmake/help/latest/variable/CMAKE_PROJECT_NAME.html
Signed-off-by: Dan Rose <dan@digilabs.io>
The ``docs/makernc.pl`` perl script can extract the options from the
source code for the configuration processing and turn these into a
GitHub MarkDown document and a RELAX NG Compact form (RNC) description
of the options. Standard conversion tools can be used to turn the RNC
file into an XSD.
Although they are generated by a mechanical transformation, it is not
(yet) part of the build process, in no small part because there no
proper generation and publishing system has yet been set up for the
documentation.
Now that there is an XSD and a Markdown file for discovering the
options, there is no need to retain the configuration editing tool. Any
decent editor will do a better job.
Signed-off-by: Erik Boasson <eb@ilities.com>
Change the structure of the configuration file (in a backwards
compatible manner) to allow specifying configurations for multiple
domains in a file. (Listing multiple files in CYCLONEDDS_URI was
already supported.) A configuration specifies an id, with a default of
any, configurations for an incompatible id are ignored.
If the application specifies an id other than DDS_DOMAIN_DEFAULT in the
call to create_participant, then only configuration specifications for
Domain elements with that id or with id "any" will be used. If the
application does specify DDS_DOMAIN_DEFAULT, then the id will be taken
from the first Domain element that specifies an id. If none do, the
domain id defaults to 0. Each applicable domain specification is taken
as a separate source and may override settings made previously.
All settings moved from the top-level CycloneDDS element to the
CycloneDDS/Domain element. The CycloneDDS/Domain/Id element moved to
become the "id" attribute of CycloneDDS/Domain. The old locations still
work, with appropriate deprecation warnings.
Signed-off-by: Erik Boasson <eb@ilities.com>
* Change default behaviour with no arguments to print help text;
"ddsperf sanity" now gives the behaviour that it used to give when run
with no arguments;
* Include context switch rate in output;
* Allow suffixing frequencies and sizes with standard units (so "size
1kB" is now allowed);
* Add missing option to help text, extend help text with some additional
informationr.
Signed-off-by: Erik Boasson <eb@ilities.com>
* per-thread CPU usage (only those threads where the load is over 0.5%,
if the sum of threads below that threshold exceeds 0.5%, it prints an
aggregate for those threads);
* also report RSS;
* network load (only on request, as percentage of specified network
bandwidth and actual bytes in/out, with the output suppressed if it is
0%);
* publish CPU usage so a ddsperf instance can display CPU loads for
its peers;
* handle SIGXFSZ (file size exceeded) by displaying one last line of
statistics before killing itself; this simply a debugging tool to make
it easier to get a trace covering a high sample-rate start-up issue;
* default topic changed to "KS" because that allows all the options to
be used, this has a negative impact on performance (both latency and
small-sample throughput) but it should be less surprising to users;
* specifying a size is now done by appending "size N" (where N is the
size in bytes) after a "ping" or "pub" command, rather than it having
to set it via a command-line option;
Note that some of this is platform-dependent -- SIGXFSZ is currently
only on Linux and macOS, and CPU and network load reporting is currently
only on Linux, macOS and Windows.
Signed-off-by: Erik Boasson <eb@ilities.com>
* Move the project top-level CMakeLists.txt to the root of the project;
this allows building Cyclone as part of ROS2 without any special
tricks;
* Clean up the build options:
ENABLE_SSL: whether to check for and include OpenSSL support if a
library can be found (default = ON); this used to be
called DDSC_ENABLE_OPENSSL, the old name is deprecated
but still works
BUILD_DOCS: whether to build docs (default = OFF)
BUILD_TESTING: whether to build test (default = OFF)
* Collect all documentation into top-level "docs" directory;
* Move the examples to the top-level directory;
* Remove the unused and somewhat misleading pseudo-default
cyclonedds.xml;
* Remove unused cmake files
Signed-off-by: Erik Boasson <eb@ilities.com>
It is an excellent platform for catching bugs: big-endian, slow enough
that a context switch in the middle of an operation becomes a regular
occurrence, and all that on a SMP box. Or: I just wanted to see if it
would work.
Signed-off-by: Erik Boasson <eb@ilities.com>
The two do essentially the same think, and ddsrt_strtok_r was only used
in one place. (Triggered by Solaris 2.6 not providing strtok_r.)
Signed-off-by: Erik Boasson <eb@ilities.com>
Tracking pings and expected number of pongs was done without holding the
correct locks. Terminate flag was also not a ddsrt_atomic... and hence
flagged by thread sanitizer as a race condition.
Signed-off-by: Erik Boasson <eb@ilities.com>
Missing prototypes for exported functions cause a really huge issue on
Windows. Enabling the "missing prototypes" warning makes it much easier
to catch this problem. Naturally, any warnings caused by this have been
fixed.
Signed-off-by: Erik Boasson <eb@ilities.com>
* Remove dds_return_t / dds_retcode_t distinction (now there is only
dds_return_t and all error codes are always negative)
* Remove Q_ERR_... error codes and replace them by DDS_RETCODE_...
ones so that there is only one set of error codes
* Replace a whole bunch "int" return types that were used to return
Q_ERR_... codes by "dds_return_t" return types
Signed-off-by: Erik Boasson <eb@ilities.com>
* As a simple matter of code hygiene, in particular to aid in checking for
leaks, ddsperf should free all memory it allocates on exit.
* Remove spurious mutex unlock in ddsperf
* Removing a participant means removing one or two entries from the "pong
writers" array ("pong wr"), and there it read 1 element beyond the end
of the array while moving the remaining elements forward.
* Constant-rate pinging was broken because of two reasons, one worse than
the other:
* setting the rate had a mismatch in variables (publication rate and
command-line argument) resulting in a completely wrong ping interval;
the code now has a bit more clear variable naming ...
* the timing of the pings was relative to the current time, but the
wakeup a little delayed, resulting in a lower rate than requested.
It now simply adds the ping interval to the scheduled ping time, rather
than the time at which the ping is being sent. To guard against really
late wakeups, rates that are too high, suspending the machine, &c. it
will in extremis delay the next ping.
Signed-off-by: Erik Boasson <eb@ilities.com>
Multiplying time-in-ns since previous output line by 1e9 instead of
dividing it by 1e9 resulted in bit rate showing up as 0Mb/s.
Signed-off-by: Erik Boasson <eb@ilities.com>
The current situation for performance measurements and checking network
behaviour is rather unsatisfactory, as the only tools available are
``pubsub`` and the ``roundtrip`` and ``throughput`` examples. The first
can do many things thanks to its thousand-and-one options, but its
purpose really is to be able to read/write arbitrary data with arbitrary
QoS -- though the arbitrary data bit was lost in the hacked conversion
from the original code. The latter two have a terrible user interface,
don't perform any verification that the measurement was successful and
do not provide the results in a convenient form.
Furthermore, the abuse of the two examples as the primary means for
measuring performance has resulted in a reduction of their value as an
example, e.g., they can do waitset- or listener-based reading (and the
throughput one also polling-based), but that kind of complication does
not help a new user understand what is going on. Especially not given
that these features were simply hacked in.
Hence the need for a new tool, one that integrates the common
measurements and can be used to verify that the results make sense. It
is not quite done yet, in particular it is lacking in a number of
aspects:
* no measurement of CPU- and network load, memory usage and context
switches yet;
* very limited statistics (min/max/average, if you're lucky; no
interesting things such as jitter on a throughput test yet);
* it can't yet gather the data from all participants in the network
using DDS;
* it doesn't output the data in a convenient file format yet;
* it doesn't allow specifying boundaries within which the results
must fall for the run to be successful.
What it does verify is that all the endpoint matches that should exist
given the discovered participant do in fact come into existence,
reporting an error (and exiting with an exit status code of 1) if they
don't, as well as checking the number of participants. With the way the
DDSI protocol works, this is a pretty decent network connectivity check.
The raw measurements needed for the desired statistics (apart from
system-level measurements) are pretty much made, so the main thing that
still needs to be done is exploit them and output them. It can already
replace the examples for most benchmarks (only the 50%/90%/99%
percentiles are still missing for a complete replacement).
Signed-off-by: Erik Boasson <eb@ilities.com>
The old entity handle mechanism suffered from a number of problems, the
most terrible one being that it would only ever allocate 1000 handles
(not even have at most 1000 in use at the same time). Secondarily, it
was protected by a single mutex that actually does show up as a limiting
factor in, say, a polling-based throughput test with small messages.
Thirdly, it tried to provide for various use cases that don't exist in
practice but add complexity and overhead.
This commit totally rewrites the mechanism, by replacing the old array
with a hash table and allowing a near-arbitrary number of handles as
well as reuse of handles. It also removes the entity "kind" bits in the
most significant bits of the handles, because they only resulted in
incorrect checking of argument validity. All that is taken out, but
there is still more cleaning up to be done. It furthermore removes an
indirection in the handle-to-entity lookup by embedding the
"dds_handle_link" structure in the entity.
Handle allocation is randomized to avoid the have a high probability of
quickly finding an available handle (the total number of handles is
limited to a number much smaller than the domain from which they are
allocated). The likelihood of handle reuse is still dependent on the
number of allocated handles -- the fewer handles there are, the longer
the expected time to reuse. Non-randomized handles would give a few
guarantees more, though.
It moreover moves the code from the "util" to the "core/ddsc" component,
because it really is only used for entities, and besides the new
implementation relies on the deferred freeing (a.k.a. garbage collection
mechanism) implemented in the core.
The actual handle management has two variants, selectable with a macro:
the preferred embodiment uses a concurrent hash table, the actually used
one performs all operations inside a single mutex and uses a
non-concurrent version of the hash table. The reason the
less-predeferred embodiment is used is that the concurrent version
requires the freeing of entity objects to be deferred (much like the
GUID-to-entity hash tables in DDSI function, or indeed the key value to
instance handle mapping). That is a fair bit of work, and the
non-concurrent version is a reasonable intermediate step.
Signed-off-by: Erik Boasson <eb@ilities.com>
