There is the occasional failure on Travis CI since 66daba9f, which
appears to be caused by a legitimate increase in memory usage but not
quite covered by the raised limits in that same commit. This raises the
limit further, but also lengthens the test to 20s to increase the
likelihood of catching non-legitimate memory growth.
Signed-off-by: Erik Boasson <eb@ilities.com>
The default value was zero, which means only specifying -Qrss:N would
require the memory usage at the end to be <= N MB, rather than the
delta.
Signed-off-by: Erik Boasson <eb@ilities.com>
This adds a set of functions:
* dds_create_statistics
* dds_refresh_statistics
* dds_delete_statistics
* dds_lookup_statistic
to poll entities for information on their state, returned as a set of
name-value pairs. The interface and selection of statistics (and
naming) is all provisional, and for this reason the
dds/ddsc/dds_statistisc.h file is not included by dds.h.
Currently, the only statistics available relate to retansmits and are
optionally output by ddsperf.
Signed-off-by: Erik Boasson <eb@ilities.com>
* Bandwidth usage is now printed in Mb/s if no reference rate is given
* Trailing average rate over the last 10s (approximated as the last 10
lines of output) is printed
* An option to wait until the expected number of peers is present
* The test script now pushes data to the remotes, instead of using the
first remote as the publisher
Signed-off-by: Erik Boasson <eb@ilities.com>
This changes a few intertwined things at the same time:
* It allows configuring sending a partial message for large messages,
with a maximum derived from the discovered receive buffer sizes;
* It uses a different message size limit for datagrams that include
retransmits than for those that don't. The argument here is that,
having seen flaky networks where large datagrams cause trouble, it
makes sense to default to sending retransmits as datagrams that fit in
individual packets.
* The best performance is generally obtained using the maximum data gram
size, but the benefits do fall off quite quickly once they are
largish. For flaky networks, it doesn't make sense to go for 64kB
datagrams. This tries to find a reasonable compromise.
* It now packs mutiple fragments into a single DATAFRAG message to
eliminate the cost of using small fragment sizes.
The changes in buffer sizes cause the ddsperf sanity check to fail:
* The larger amounts of unacknowledged data cause the used memory to be
higher, failing the RSS check. Raising the limit seems
reasonable (the alternative would be to configure it back to the old
values, but it is all empirically determined anyway).
* The same also causes the publisher thread to get to run more and the
ping/pong bit gets less of a chance. Using fixed-frequency bursts
helps with this.
This therefore also adjust the test configuration and the thresholds a
bit.
Signed-off-by: Erik Boasson <eb@ilities.com>
The former name should be less confusing. Backwards compatibility is
preserved by only adding the sensible name as a typedef.
Signed-off-by: Erik Boasson <eb@ilities.com>
The participant listener creates a pong writer, setting a publication
matched listener on it. That listener can be invoked immediately and as
it queries the subscriptions reader, it must not be enabled before the
latter reader has been created.
Signed-off-by: Erik Boasson <eb@ilities.com>
* IPv6 extensions to patterns
* use full GUID prefix for Cyclone
* pattern fixes to deal with small changes in the formatting of QoS
* suppressinof local built-in topic publications
* asymmetrical disconnect detection improvements (better chance of
detecting it, plus better suppression of spurious notifications)
Signed-off-by: Erik Boasson <eb@ilities.com>
```
/opt/ros/master/src/eclipse-cyclonedds/cyclonedds/src/tools/pubsub/common.c:586:28: warning: implicit conversion from 'long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-int-float-conversion]
if(nanosec > nextafter(INT64_MAX, 0)) {
~~~~~~~~~ ^~~~~~~~~
/usr/include/stdint.h:134:22: note: expanded from macro 'INT64_MAX'
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/stdint.h:116:24: note: expanded from macro '__INT64_C'
^~~~~~
<scratch space>:345:1: note: expanded from here
9223372036854775807L
^~~~~~~~~~~~~~~~~~~~
1 warning generated.
```
Signed-off-by: Dan Rose <dan@digilabs.io>
This adds options to check for "unreasonable" RSS growth, receipt of a
minimum number of samples and having run a minimum number of roundtrips.
Signed-off-by: Erik Boasson <eb@ilities.com>
The status mask on some readers got reduced to just "data available"
when used in conjunction with a waitset, but the consequence is that the
"subscription matched" listener would be suppressed.
Signed-off-by: Erik Boasson <eb@ilities.com>
* Use the parameter tables to pretty-print QoS and plist, rather than a
hard-coded function supporting only the QoS.
* Support diffing two plists: a single table-driven function can handle
both nn_plist_t and ddsi_qos_t, and it removes the discrepancy between
the two types.
* Log content of discovery samples in trace rather than merely printing
"(plist)"
Signed-off-by: Erik Boasson <eb@ilities.com>
This commit adds support for liveliness QoS when using local readers.
The implementation for (liveliness) expiration of writers used here is
similar to that used with proxy writers, and it also supports the three
liveliness kinds (1) automatic, which is trivial when using a local
reader and writer, (2) manual-by-participant and (3) manual-by-topic.
In addition, these changes and fixes are included in this commit:
- Fixed a bug in heartbeat handling in the reader: for manual-by-
participant writers the lease was not updated on reception of a
heartbeat message with liveliness flag set. This is fixed and a
test-case is added.
- Include the liveliness flag in a heartbeat message to the trace
- Trace all lease renewals, including liveliness leases
- Replaced liveliness changed state 'twitch' by 2 subsequent calls
to the status callback
- Added a test for liveliness duration 0 and 1ns (for both local
and remote readers)
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
* Add PropertyPolicy to QoS API for Security settings
This commit adds the public API for PropertyQosPolicy including
tests. This policy can be used to set the parameters for the DDS security
implementation, as an alternative for using the xml configuration.
Tests are also inlcuded for setting security properties and conflict
resolving when both security configuration and qos properties are present.
Finally, the pubsub tool is updated so that is handles this qos correctly.
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
* Init binary_value.props to fix failing qos merge and moved init code in qset_prop functions
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
* Add additional test and some validation improvements based on review comments
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
* Refactoring of qos property
Refactored the qos property handling based on review comments. Setting
and unsettings functions are simplified and now use helper functions
for lookup, property initialisation is simplified. Added an additional
check for required security properties when creating participant using
security settings from qos, and added a test-case for this code.
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
* Check for qos flag before getting property index from qos
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
* Participant creation should fail on inconsistent security qos properties, and some minor code improvements in property qos api functions
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
* Update log message in test security_config_qos
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
* Fixed unused label compiler error in q_entity.c when security is disabled
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
* Refactored qprop functions with macros to avoid code duplicate code
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
CMAKE_PROJECT_NAME refers to the top-level project name, not the most recent project. So any CMake project that pulls this in as a dependency was in for a nasty surprise.
https://cmake.org/cmake/help/latest/variable/CMAKE_PROJECT_NAME.html
Signed-off-by: Dan Rose <dan@digilabs.io>
The ``docs/makernc.pl`` perl script can extract the options from the
source code for the configuration processing and turn these into a
GitHub MarkDown document and a RELAX NG Compact form (RNC) description
of the options. Standard conversion tools can be used to turn the RNC
file into an XSD.
Although they are generated by a mechanical transformation, it is not
(yet) part of the build process, in no small part because there no
proper generation and publishing system has yet been set up for the
documentation.
Now that there is an XSD and a Markdown file for discovering the
options, there is no need to retain the configuration editing tool. Any
decent editor will do a better job.
Signed-off-by: Erik Boasson <eb@ilities.com>
Change the structure of the configuration file (in a backwards
compatible manner) to allow specifying configurations for multiple
domains in a file. (Listing multiple files in CYCLONEDDS_URI was
already supported.) A configuration specifies an id, with a default of
any, configurations for an incompatible id are ignored.
If the application specifies an id other than DDS_DOMAIN_DEFAULT in the
call to create_participant, then only configuration specifications for
Domain elements with that id or with id "any" will be used. If the
application does specify DDS_DOMAIN_DEFAULT, then the id will be taken
from the first Domain element that specifies an id. If none do, the
domain id defaults to 0. Each applicable domain specification is taken
as a separate source and may override settings made previously.
All settings moved from the top-level CycloneDDS element to the
CycloneDDS/Domain element. The CycloneDDS/Domain/Id element moved to
become the "id" attribute of CycloneDDS/Domain. The old locations still
work, with appropriate deprecation warnings.
Signed-off-by: Erik Boasson <eb@ilities.com>
* Change default behaviour with no arguments to print help text;
"ddsperf sanity" now gives the behaviour that it used to give when run
with no arguments;
* Include context switch rate in output;
* Allow suffixing frequencies and sizes with standard units (so "size
1kB" is now allowed);
* Add missing option to help text, extend help text with some additional
informationr.
Signed-off-by: Erik Boasson <eb@ilities.com>
* per-thread CPU usage (only those threads where the load is over 0.5%,
if the sum of threads below that threshold exceeds 0.5%, it prints an
aggregate for those threads);
* also report RSS;
* network load (only on request, as percentage of specified network
bandwidth and actual bytes in/out, with the output suppressed if it is
0%);
* publish CPU usage so a ddsperf instance can display CPU loads for
its peers;
* handle SIGXFSZ (file size exceeded) by displaying one last line of
statistics before killing itself; this simply a debugging tool to make
it easier to get a trace covering a high sample-rate start-up issue;
* default topic changed to "KS" because that allows all the options to
be used, this has a negative impact on performance (both latency and
small-sample throughput) but it should be less surprising to users;
* specifying a size is now done by appending "size N" (where N is the
size in bytes) after a "ping" or "pub" command, rather than it having
to set it via a command-line option;
Note that some of this is platform-dependent -- SIGXFSZ is currently
only on Linux and macOS, and CPU and network load reporting is currently
only on Linux, macOS and Windows.
Signed-off-by: Erik Boasson <eb@ilities.com>
* Move the project top-level CMakeLists.txt to the root of the project;
this allows building Cyclone as part of ROS2 without any special
tricks;
* Clean up the build options:
ENABLE_SSL: whether to check for and include OpenSSL support if a
library can be found (default = ON); this used to be
called DDSC_ENABLE_OPENSSL, the old name is deprecated
but still works
BUILD_DOCS: whether to build docs (default = OFF)
BUILD_TESTING: whether to build test (default = OFF)
* Collect all documentation into top-level "docs" directory;
* Move the examples to the top-level directory;
* Remove the unused and somewhat misleading pseudo-default
cyclonedds.xml;
* Remove unused cmake files
Signed-off-by: Erik Boasson <eb@ilities.com>
It is an excellent platform for catching bugs: big-endian, slow enough
that a context switch in the middle of an operation becomes a regular
occurrence, and all that on a SMP box. Or: I just wanted to see if it
would work.
Signed-off-by: Erik Boasson <eb@ilities.com>
The two do essentially the same think, and ddsrt_strtok_r was only used
in one place. (Triggered by Solaris 2.6 not providing strtok_r.)
Signed-off-by: Erik Boasson <eb@ilities.com>
Tracking pings and expected number of pongs was done without holding the
correct locks. Terminate flag was also not a ddsrt_atomic... and hence
flagged by thread sanitizer as a race condition.
Signed-off-by: Erik Boasson <eb@ilities.com>
Missing prototypes for exported functions cause a really huge issue on
Windows. Enabling the "missing prototypes" warning makes it much easier
to catch this problem. Naturally, any warnings caused by this have been
fixed.
Signed-off-by: Erik Boasson <eb@ilities.com>
