Commit graph

660 commits

Author SHA1 Message Date
Erik Boasson
8964c0b1bc Handle malformed pre-emptive ACKNACK from FastRTPS
The pre-emptive ACKNACK messages from FastRTPS have a base sequence
number of 0, which is malformed and must be rejected according to DDSI
8.3.7.1 / 8.3.5.5.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-26 11:50:26 +02:00
Jeroen Koekkoek
93addbbda0 Fix shadow warnings in CUnit tests
Signed-off-by: Jeroen Koekkoek <jeroen@koekkoek.nl>
2019-08-23 12:37:15 +02:00
Jeroen Koekkoek
4e741e9137 Synchronize warning flags between Xcode and Clang
Signed-off-by: Jeroen Koekkoek <jeroen@koekkoek.nl>
2019-08-23 12:37:15 +02:00
Jeroen Koekkoek
1200bfd109 Set project name via string literal
Some tools (e.g. colcon) require the project name to be a string literal in
order to extract it. Fix proposed by Dirk Thomas.

Signed-off-by: Jeroen Koekkoek <jeroen@koekkoek.nl>
2019-08-23 12:37:15 +02:00
Erik Boasson
6f33dc0e6d CI tests: log DDSI configuration to stderr
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
f06d3bf9ad FreeRTOS: replace DDS_TRACE+abort by DDS_FATAL
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
5b2cc4e6f3 Work around FreeRTOS header trouble
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
da05024fe3 Fixup for repository reorg: Solaris 2.6 makefile
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
1129939865 Cleanup assertions in serializer
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
891fc2b12f Support multiple domains in configuration
Change the structure of the configuration file (in a backwards
compatible manner) to allow specifying configurations for multiple
domains in a file.  (Listing multiple files in CYCLONEDDS_URI was
already supported.)  A configuration specifies an id, with a default of
any, configurations for an incompatible id are ignored.

If the application specifies an id other than DDS_DOMAIN_DEFAULT in the
call to create_participant, then only configuration specifications for
Domain elements with that id or with id "any" will be used.  If the
application does specify DDS_DOMAIN_DEFAULT, then the id will be taken
from the first Domain element that specifies an id.  If none do, the
domain id defaults to 0.  Each applicable domain specification is taken
as a separate source and may override settings made previously.

All settings moved from the top-level CycloneDDS element to the
CycloneDDS/Domain element.  The CycloneDDS/Domain/Id element moved to
become the "id" attribute of CycloneDDS/Domain.  The old locations still
work, with appropriate deprecation warnings.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
70a342991f Build no-SSL version on CI as well
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
711026114b Add multi-domain version of topic_data test
This is the same test for verifying set_qos on topic_data is reflected
in the DCPSPublication and DCPSSubscription topics as the one that
already exists, but it uses two threads in two domains.  A barrier is
used to ensure the threads in the two processes indeed execute
concurrently, and different topic data sequences are used to increase
the likelihood of detecting leakage from one domain into the other.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
5e31a3df90 Use separate plist for local process settings
The default participant QoS/plist that is used for defaulting received
QoS and for determining which QoS/plist entries to send in discovery
data was mixed up with the one that contains local process information
such as hostname and process id.

It moreover was modified after starting up the protocol stack, and hence
after discovery of remote participants.  While unlikely, this could lead
to an assertion in plist_or_xqos_mergein_missing.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
09e08f7778 Lock entity QoS while accessing it
A QoS change can happen at the same time that a new reader for a
built-in topic is provisioned with historical data, and so cause reading
in inconsistent QoS, use-after-free or other fun things.

During QoS matching it is also necessary to guarantee the QoS doesn't
change (QoS changes affecting matching will be supported at some point,
and manipulating complex data structures where bitmasks determine which
parts are defined while reading the same data concurrently is a recipe
for disaster.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
103210bf8e Check sequence number of SEDP messages
Historical data can processed after new data, effectively going backward
in time, so only process data that is newer than the current state.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
966ec0dda7 Make logging config per-domain
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
7190bb3d3e Pacification of native compiler on OpenIndiana
* constness in ternary expressions
* removal of OS_MAX_INTEGER
* inclusion of dds/ddsrt/attributes.h everywhere DDS_EXPORT inline
  occurs
* _POSIX_PTHREAD_SEMANTICS in ddsperf

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
c9f04ee5bd Allow coexisting domains and do a minimal test
The big issue is the there is still only a single log output that gets
opened on creating a domain and closed on deleting one, but otherwise at
least this minimal test works.

The other issue is that the GC waits until threads in all domains have
made sufficient progress, rather than just the threads in its own
domain.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
c1f3ad8a22 Eliminate domain-specific global variables
This commit moves all but a handful of the global variables into the
domain object, in particular including the DDSI configuration, globals
and all transport internal state.

The goal of this commit is not to produce the nicest code possible, but
to get a working version that can support multiple simultaneous domains.
Various choices are driven by this desire and it is expected that some
of the changes will have to be undone.  (E.g., passing the DDSI globals
into address set operations and locator printing because there is no
other way to figure out what transport to use for a given locator;
storing the transport pointer inside the locator would solve that.)

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
8a591fdc9b Liveliness monitoring to dds_global
Thread liveliness monitoring moves to dds_global and there is one
monitor running if there is at least one domain that requests it.  The
synchronization over freeing the thread name when reaping the thread
state is gone by no longer dynamically allocating the thread name.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
f05dd6ac07 Remove use of globals from config processing
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
782f032df8 DDSI stack init/fini in domain_create/free
This moves DDSI stack initialisation and finalisation to the creating
and deleting of a domain, and modifies the related code to trigger all
that from creating/deleting participants.

Built-in topic generation is partially domain-dependent, so that moves
as well.  The underlying ddsi_sertopics can be created are domain
independent and created without initialising DDSI, which necessitates
moving the IID generation (and thus init/fini) out of the DDSI stack and
to what will remain global data.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
cf46ddbb7b Move deleted participants admin into "gv"
That reduces the number of global variables in preparation for
supporting multi domains simultaneously.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
59fb537ba3 Cleanup duplicate deinit code in UDP support
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
ced9b1c219 Remove global temporaries from config parsing
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
16f1df74f1 Remove unused "durability service" global variables
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
2e9ce9b4c1 Abstract RHC interface
This makes it possible to use a different RHC implementations for
different readers and removes the need for the RHC interface to be part
of the global state.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Thijs Sassen
483f4d2b77 Fallback to zip installer for non Linux/Windows builds
Signed-off-by: Thijs Sassen <thijs.sassen@adlinktech.com>
2019-08-12 11:31:29 +02:00
Erik Boasson
f7941b98a9 Fix crash dropping sample missing first fragment
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-07 13:43:41 +02:00
Erik Boasson
0e83982aeb Correct ACKNACK, GAP, NACKFRAG size calculation
Commit 3afce30c37 introduced an error in
the calculation of the size of these submessages, making them (and
requiring them to be) larger than correct and putting the "count" field
at the wrong offset, breaking interoperability.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-07 13:43:41 +02:00
Erik Boasson
40ba6b207f Small ddsperf enhancements
* Change default behaviour with no arguments to print help text;
  "ddsperf sanity" now gives the behaviour that it used to give when run
  with no arguments;

* Include context switch rate in output;

* Allow suffixing frequencies and sizes with standard units (so "size
  1kB" is now allowed);

* Add missing option to help text, extend help text with some additional
  informationr.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-06 08:32:14 +02:00
Erik Boasson
ca04ac48de Parallelize builds & tests
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-06 08:32:14 +02:00
Erik Boasson
e143caa106 Reduce likelihood of ddsrt_select_timeout failure
The main change is a longer sleep in the main thread while waiting for
the select call to timeout.  Still not perfect, but more practical than
checking the stacktrace of the thread calling ddsrt_select() to see
whether it has spent too much time in the call.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-06 08:32:14 +02:00
Erik Boasson
774dae135b Assertion rsample_is_singleton fails
Caused by the changes in a652ecb78e; the
sample that matters is the first in what may now be a chain of samples,
which requires some overlooked adjustments.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-06 08:32:14 +02:00
Erik Boasson
d494ba4eda Make qos MPTs more robust
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-06 08:32:14 +02:00
Erik Boasson
3afce30c37 C99 forbids flex array members in nested structs
Even when they are at the end.  GCC & Clang allow it, but not all
compilers do.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-06 08:32:14 +02:00
Erik Boasson
ecbee32422 Ignore EPERM on sendmsg
Some Linux versions sometimes return this (undocumented) error,
presumably because of firewalling.  Better to ignore it.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-06 08:32:14 +02:00
Thijs Sassen
c0b0db3ae2 Fixed not building freertos target due to conflict in print definition
Signed-off-by: Thijs Sassen <thijs.sassen@adlinktech.com>
2019-08-05 13:56:53 +02:00
Jeroen Koekkoek
15e68152c9
Merge pull request #225 from k0ekk0ek/documentation
Generate documentation using the FindSphinx.cmake module
2019-08-02 14:10:58 +02:00
Jeroen Koekkoek
0351963670 Fix version number parse bug in FindSphinx.cmake
Signed-off-by: Jeroen Koekkoek <jeroen@koekkoek.nl>
2019-08-02 13:06:43 +02:00
Erik Boasson
af19c5681c Use ddsperf in perf scripts and update README
* the scripted throughput test originally used for the throughput graph
  in the README now uses ddsperf;

* a scripted latency test has been added;

* updated the README with the results of these tests (and so now gives
  easy access not only to throughput, but also to latency and memory
  usage, as well as to latency over GbE.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-02 09:53:36 +02:00
Erik Boasson
952029dba0 ddsperf enhancements
* per-thread CPU usage (only those threads where the load is over 0.5%,
  if the sum of threads below that threshold exceeds 0.5%, it prints an
  aggregate for those threads);

* also report RSS;

* network load (only on request, as percentage of specified network
  bandwidth and actual bytes in/out, with the output suppressed if it is
  0%);

* publish CPU usage so a ddsperf instance can display CPU loads for
  its peers;

* handle SIGXFSZ (file size exceeded) by displaying one last line of
  statistics before killing itself; this simply a debugging tool to make
  it easier to get a trace covering a high sample-rate start-up issue;

* default topic changed to "KS" because that allows all the options to
  be used, this has a negative impact on performance (both latency and
  small-sample throughput) but it should be less surprising to users;

* specifying a size is now done by appending "size N" (where N is the
  size in bytes) after a "ping" or "pub" command, rather than it having
  to set it via a command-line option;

Note that some of this is platform-dependent -- SIGXFSZ is currently
only on Linux and macOS, and CPU and network load reporting is currently
only on Linux, macOS and Windows.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-02 09:53:36 +02:00
Erik Boasson
ecb77d481c Add network statistics to ddsrt
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-02 09:53:36 +02:00
Erik Boasson
f9808c7656 Add gathering per-thread CPU usage to ddsrt
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-02 09:53:36 +02:00
Erik Boasson
9b1920862e Init all sample pointers when reusing loan
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-02 09:53:36 +02:00
Jeroen Koekkoek
b916f0bfb7 Generate documentation using the FindSphinx.cmake module
Signed-off-by: Jeroen Koekkoek <jeroen@koekkoek.nl>
2019-07-31 20:12:11 +02:00
Erik Boasson
9cf4b97f1a Reorganize repository
* Move the project top-level CMakeLists.txt to the root of the project;
  this allows building Cyclone as part of ROS2 without any special
  tricks;

* Clean up the build options:

  ENABLE_SSL:    whether to check for and include OpenSSL support if a
                 library can be found (default = ON); this used to be
                 called DDSC_ENABLE_OPENSSL, the old name is deprecated
                 but still works
  BUILD_DOCS:    whether to build docs (default = OFF)
  BUILD_TESTING: whether to build test (default = OFF)

* Collect all documentation into top-level "docs" directory;

* Move the examples to the top-level directory;

* Remove the unused and somewhat misleading pseudo-default
  cyclonedds.xml;

* Remove unused cmake files

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-07-30 10:52:44 +02:00
Erik Boasson
4e80559763 Improve multicast related defaults
* use multicast only for participant discovery if using a WiFi network
* default to using unicast for retransmits

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-07-25 11:31:51 +02:00
Erik Boasson
fda285e2f5 Add support for Solaris 2.6 on sun4m builds
It is an excellent platform for catching bugs: big-endian, slow enough
that a context switch in the middle of an operation becomes a regular
occurrence, and all that on a SMP box.  Or: I just wanted to see if it
would work.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-07-25 10:59:09 +02:00
Erik Boasson
47920df65c Remove qoslock
Creating a reader/writer in a listener for a built-in topic (as ddsperf
does) recursively ends up in reader/writer matching, recursively
read-locking qoslock.  As nobody ever takes a write-lock, it is a
non-issue provided the rwlock really is an rwlock.  On Solaris 2.6 those
don't exist, and mapping it onto a mutex deadlocks.

This commit removes the thing in its entirety, the fact that it is
currently only ever locked for reading is hint that perhaps it is not
that valuable a thing.  The way it was used in the code would in any
case not have helped with re-matching on QoS changes (save for
duplicating all the matching code), and it is doubtful that serializing
the matching in that case would be necessary in the first place.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-07-25 10:59:09 +02:00