Commit graph

700 commits

Author SHA1 Message Date
Erik Boasson
70a342991f Build no-SSL version on CI as well
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
711026114b Add multi-domain version of topic_data test
This is the same test for verifying set_qos on topic_data is reflected
in the DCPSPublication and DCPSSubscription topics as the one that
already exists, but it uses two threads in two domains.  A barrier is
used to ensure the threads in the two processes indeed execute
concurrently, and different topic data sequences are used to increase
the likelihood of detecting leakage from one domain into the other.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
5e31a3df90 Use separate plist for local process settings
The default participant QoS/plist that is used for defaulting received
QoS and for determining which QoS/plist entries to send in discovery
data was mixed up with the one that contains local process information
such as hostname and process id.

It moreover was modified after starting up the protocol stack, and hence
after discovery of remote participants.  While unlikely, this could lead
to an assertion in plist_or_xqos_mergein_missing.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
09e08f7778 Lock entity QoS while accessing it
A QoS change can happen at the same time that a new reader for a
built-in topic is provisioned with historical data, and so cause reading
in inconsistent QoS, use-after-free or other fun things.

During QoS matching it is also necessary to guarantee the QoS doesn't
change (QoS changes affecting matching will be supported at some point,
and manipulating complex data structures where bitmasks determine which
parts are defined while reading the same data concurrently is a recipe
for disaster.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
103210bf8e Check sequence number of SEDP messages
Historical data can processed after new data, effectively going backward
in time, so only process data that is newer than the current state.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
966ec0dda7 Make logging config per-domain
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
7190bb3d3e Pacification of native compiler on OpenIndiana
* constness in ternary expressions
* removal of OS_MAX_INTEGER
* inclusion of dds/ddsrt/attributes.h everywhere DDS_EXPORT inline
  occurs
* _POSIX_PTHREAD_SEMANTICS in ddsperf

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
c9f04ee5bd Allow coexisting domains and do a minimal test
The big issue is the there is still only a single log output that gets
opened on creating a domain and closed on deleting one, but otherwise at
least this minimal test works.

The other issue is that the GC waits until threads in all domains have
made sufficient progress, rather than just the threads in its own
domain.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
c1f3ad8a22 Eliminate domain-specific global variables
This commit moves all but a handful of the global variables into the
domain object, in particular including the DDSI configuration, globals
and all transport internal state.

The goal of this commit is not to produce the nicest code possible, but
to get a working version that can support multiple simultaneous domains.
Various choices are driven by this desire and it is expected that some
of the changes will have to be undone.  (E.g., passing the DDSI globals
into address set operations and locator printing because there is no
other way to figure out what transport to use for a given locator;
storing the transport pointer inside the locator would solve that.)

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
8a591fdc9b Liveliness monitoring to dds_global
Thread liveliness monitoring moves to dds_global and there is one
monitor running if there is at least one domain that requests it.  The
synchronization over freeing the thread name when reaping the thread
state is gone by no longer dynamically allocating the thread name.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
f05dd6ac07 Remove use of globals from config processing
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
782f032df8 DDSI stack init/fini in domain_create/free
This moves DDSI stack initialisation and finalisation to the creating
and deleting of a domain, and modifies the related code to trigger all
that from creating/deleting participants.

Built-in topic generation is partially domain-dependent, so that moves
as well.  The underlying ddsi_sertopics can be created are domain
independent and created without initialising DDSI, which necessitates
moving the IID generation (and thus init/fini) out of the DDSI stack and
to what will remain global data.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
cf46ddbb7b Move deleted participants admin into "gv"
That reduces the number of global variables in preparation for
supporting multi domains simultaneously.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
59fb537ba3 Cleanup duplicate deinit code in UDP support
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
ced9b1c219 Remove global temporaries from config parsing
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
16f1df74f1 Remove unused "durability service" global variables
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
2e9ce9b4c1 Abstract RHC interface
This makes it possible to use a different RHC implementations for
different readers and removes the need for the RHC interface to be part
of the global state.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Thijs Sassen
483f4d2b77 Fallback to zip installer for non Linux/Windows builds
Signed-off-by: Thijs Sassen <thijs.sassen@adlinktech.com>
2019-08-12 11:31:29 +02:00
Erik Boasson
f7941b98a9 Fix crash dropping sample missing first fragment
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-07 13:43:41 +02:00
Erik Boasson
0e83982aeb Correct ACKNACK, GAP, NACKFRAG size calculation
Commit 3afce30c37 introduced an error in
the calculation of the size of these submessages, making them (and
requiring them to be) larger than correct and putting the "count" field
at the wrong offset, breaking interoperability.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-07 13:43:41 +02:00
Erik Boasson
40ba6b207f Small ddsperf enhancements
* Change default behaviour with no arguments to print help text;
  "ddsperf sanity" now gives the behaviour that it used to give when run
  with no arguments;

* Include context switch rate in output;

* Allow suffixing frequencies and sizes with standard units (so "size
  1kB" is now allowed);

* Add missing option to help text, extend help text with some additional
  informationr.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-06 08:32:14 +02:00
Erik Boasson
ca04ac48de Parallelize builds & tests
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-06 08:32:14 +02:00
Erik Boasson
e143caa106 Reduce likelihood of ddsrt_select_timeout failure
The main change is a longer sleep in the main thread while waiting for
the select call to timeout.  Still not perfect, but more practical than
checking the stacktrace of the thread calling ddsrt_select() to see
whether it has spent too much time in the call.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-06 08:32:14 +02:00
Erik Boasson
774dae135b Assertion rsample_is_singleton fails
Caused by the changes in a652ecb78e; the
sample that matters is the first in what may now be a chain of samples,
which requires some overlooked adjustments.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-06 08:32:14 +02:00
Erik Boasson
d494ba4eda Make qos MPTs more robust
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-06 08:32:14 +02:00
Erik Boasson
3afce30c37 C99 forbids flex array members in nested structs
Even when they are at the end.  GCC & Clang allow it, but not all
compilers do.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-06 08:32:14 +02:00
Erik Boasson
ecbee32422 Ignore EPERM on sendmsg
Some Linux versions sometimes return this (undocumented) error,
presumably because of firewalling.  Better to ignore it.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-06 08:32:14 +02:00
Thijs Sassen
c0b0db3ae2 Fixed not building freertos target due to conflict in print definition
Signed-off-by: Thijs Sassen <thijs.sassen@adlinktech.com>
2019-08-05 13:56:53 +02:00
Jeroen Koekkoek
15e68152c9
Merge pull request #225 from k0ekk0ek/documentation
Generate documentation using the FindSphinx.cmake module
2019-08-02 14:10:58 +02:00
Jeroen Koekkoek
0351963670 Fix version number parse bug in FindSphinx.cmake
Signed-off-by: Jeroen Koekkoek <jeroen@koekkoek.nl>
2019-08-02 13:06:43 +02:00
Erik Boasson
af19c5681c Use ddsperf in perf scripts and update README
* the scripted throughput test originally used for the throughput graph
  in the README now uses ddsperf;

* a scripted latency test has been added;

* updated the README with the results of these tests (and so now gives
  easy access not only to throughput, but also to latency and memory
  usage, as well as to latency over GbE.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-02 09:53:36 +02:00
Erik Boasson
952029dba0 ddsperf enhancements
* per-thread CPU usage (only those threads where the load is over 0.5%,
  if the sum of threads below that threshold exceeds 0.5%, it prints an
  aggregate for those threads);

* also report RSS;

* network load (only on request, as percentage of specified network
  bandwidth and actual bytes in/out, with the output suppressed if it is
  0%);

* publish CPU usage so a ddsperf instance can display CPU loads for
  its peers;

* handle SIGXFSZ (file size exceeded) by displaying one last line of
  statistics before killing itself; this simply a debugging tool to make
  it easier to get a trace covering a high sample-rate start-up issue;

* default topic changed to "KS" because that allows all the options to
  be used, this has a negative impact on performance (both latency and
  small-sample throughput) but it should be less surprising to users;

* specifying a size is now done by appending "size N" (where N is the
  size in bytes) after a "ping" or "pub" command, rather than it having
  to set it via a command-line option;

Note that some of this is platform-dependent -- SIGXFSZ is currently
only on Linux and macOS, and CPU and network load reporting is currently
only on Linux, macOS and Windows.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-02 09:53:36 +02:00
Erik Boasson
ecb77d481c Add network statistics to ddsrt
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-02 09:53:36 +02:00
Erik Boasson
f9808c7656 Add gathering per-thread CPU usage to ddsrt
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-02 09:53:36 +02:00
Erik Boasson
9b1920862e Init all sample pointers when reusing loan
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-02 09:53:36 +02:00
Jeroen Koekkoek
b916f0bfb7 Generate documentation using the FindSphinx.cmake module
Signed-off-by: Jeroen Koekkoek <jeroen@koekkoek.nl>
2019-07-31 20:12:11 +02:00
Erik Boasson
9cf4b97f1a Reorganize repository
* Move the project top-level CMakeLists.txt to the root of the project;
  this allows building Cyclone as part of ROS2 without any special
  tricks;

* Clean up the build options:

  ENABLE_SSL:    whether to check for and include OpenSSL support if a
                 library can be found (default = ON); this used to be
                 called DDSC_ENABLE_OPENSSL, the old name is deprecated
                 but still works
  BUILD_DOCS:    whether to build docs (default = OFF)
  BUILD_TESTING: whether to build test (default = OFF)

* Collect all documentation into top-level "docs" directory;

* Move the examples to the top-level directory;

* Remove the unused and somewhat misleading pseudo-default
  cyclonedds.xml;

* Remove unused cmake files

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-07-30 10:52:44 +02:00
Erik Boasson
4e80559763 Improve multicast related defaults
* use multicast only for participant discovery if using a WiFi network
* default to using unicast for retransmits

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-07-25 11:31:51 +02:00
Erik Boasson
fda285e2f5 Add support for Solaris 2.6 on sun4m builds
It is an excellent platform for catching bugs: big-endian, slow enough
that a context switch in the middle of an operation becomes a regular
occurrence, and all that on a SMP box.  Or: I just wanted to see if it
would work.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-07-25 10:59:09 +02:00
Erik Boasson
47920df65c Remove qoslock
Creating a reader/writer in a listener for a built-in topic (as ddsperf
does) recursively ends up in reader/writer matching, recursively
read-locking qoslock.  As nobody ever takes a write-lock, it is a
non-issue provided the rwlock really is an rwlock.  On Solaris 2.6 those
don't exist, and mapping it onto a mutex deadlocks.

This commit removes the thing in its entirety, the fact that it is
currently only ever locked for reading is hint that perhaps it is not
that valuable a thing.  The way it was used in the code would in any
case not have helped with re-matching on QoS changes (save for
duplicating all the matching code), and it is doubtful that serializing
the matching in that case would be necessary in the first place.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-07-25 10:59:09 +02:00
Erik Boasson
96e09d2d4e Use ddsrt_strsep instead of ddsrt_strtok_r
The two do essentially the same think, and ddsrt_strtok_r was only used
in one place.  (Triggered by Solaris 2.6 not providing strtok_r.)

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-07-25 10:59:09 +02:00
Erik Boasson
0d33462664 Use uintptr_t rather than uintmax_t for thread ids
On 32-bit machines uintmax_t is likely to be slower than uintptr_t, and
for that reason, using an uintmax_t for a thread id seems highly
unlikely.  For the current platforms, uintptr_t works.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-07-25 10:59:09 +02:00
Erik Boasson
bf5920385f Account for DDSI_INCLUDE_SSM in parameterlist size
The parameter list table indices ought to be a small as possible to
avoid wasting space, and that means the index size is dependent on
whether or not DDSI_INCLUDE_SSM is set.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-07-25 10:59:09 +02:00
Erik Boasson
5a83d422ea Do not use msg_flags if DDSRT_MSGHDR_FLAGS not set
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-07-25 10:59:09 +02:00
Erik Boasson
20b91796b1 Fix big-endian build failure
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-07-25 10:59:09 +02:00
TheFixer
1f5083aa44 #213 - Allow readconditions and queryconditions as valid entities for… (#214)
Allow readconditions and queryconditions as valid entities for dds_instance_get_key

Signed-off-by: TheFixer <thefixer@iteazz.com>
2019-07-16 13:09:06 +02:00
Erik Boasson
0dd2155f99 64-bit alignment in serialised data
The payload in a struct serdata_default is assumed to be at a 64-bit
offset for conversion to/from a dds_{i,o}stream_t and getting padding
calculations in the serialised representation correct.  The definition
did not guarantee this and got it wrong on a 32-bit release build.

This commit computes the required padding at compile time and at
verifies the assumption holds where it matters.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-07-16 12:37:33 +02:00
Thijs Sassen
2fe4a4ca35 Fixed failing FreeRTOS target due to recent code refactors
Signed-off-by: Thijs Sassen <thijs.sassen@adlinktech.com>

Adjusted the close methode not to expand by the lwip close macro and added a check for DDSI_INCLUDE_SSM to match the correct pid table size.

Signed-off-by: Thijs Sassen <thijs.sassen@adlinktech.com>
2019-07-11 22:45:17 +02:00
Erik Boasson
ee9a12b469 Allocate xpack::iov once writer has to send data
Currently each DDSC (not DDSI) writer has its own "xpack" for packing
submessages into larger messages, but that is a bit wasteful, especially
when a lot of samples are being generated that never need to go onto the
wire.  Lazily allocating them and only pushing message into them when
they have a destination address saves memory and improves speed for
local communications.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-06-28 12:47:27 +02:00
Erik Boasson
260f8cd86b Lazily allocate state for multi-writer instances
Multiple writers for a single instance is pretty rare, so it makes sense
to lazily allocate the tables for keeping track of them.  The more
elegant solution would be to have a single lock-free table.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-06-28 12:47:27 +02:00