Commit graph

128 commits

Author SHA1 Message Date
eboasson
50b0611ad1
macOS pre-10.13 support (#264)
* macOS < 10.13: skip log tests for want of fmemopen

There is a dependency on fmemopen in the tests for the logging
functions, which macOS introduced only in version 10.13.  There should
be no other obstacle to building/running/testing on older versions of
macOS.  This commit does the pragmatic thing: remove the dependency and
turns the tests into no-ops in this one case.

Signed-off-by: Erik Boasson <eb@ilities.com>

* Fix macOS clock_gettime_nsec_np availability test

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-09-30 09:13:49 +02:00
Martin Bremmer
919850232c Merge branch 'master' into merge
Signed-off-by: Martin Bremmer <martin.bremmer@adlinktech.com>
2019-09-27 14:45:16 +02:00
Erik Boasson
94483e3371 Address Coverity, Clang static analyzer warnings
* Fix type of num reliable readers (int to int32_t)

* Conversion codes in debug monitor printf formats

* Dead code elimination

* Skipping a test case where SIZE_MAX is assumed > INT32_MAX if
  assumption is false on target platform

* Error handling in os_sockWaitsetNew

* Stick to unsigned in fragment size calculations

  This check is actually guarded by valid_DataFrag and was safe for
  datagrams up to 2GB, but the unintended and implicit conversion to is
  still best eliminated.

* A "server" connection never has an invalid socket in TCP wrapper

* Handle error return from gethostname in SPDP write (CID 248183)

* Handle extended retcodes in dds_strretcode

  CID 248131, introduced by 19aec98b8a

* Remove dead code in ddsrt logging test (CID 248195)

* Validate command-line argument in process test (CID 248117)

* Allow for extremely delayed store in test

  Test is constructed to have the events trigger only at the appropriate
  times, but it does assume that the store to cb_called becomes visible
  prior to the listener callback.  I'm pretty sure that will always be
  the case in practice, but I'm also pretty sure there is no formal
  guarantee without a memory barrier, which mutex_unlock provides.

  CID 248088, 248136, 248177, 253590, 253591, 253593

* Check unsetenv return value in test (CID 248099)

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-09-25 10:46:40 +02:00
Erik Boasson
2996a6b5f8 Let Coverity ignore sleep in ddsrt init "spinlock"
Initialisation of ddsrt layer uses a hand-rolled CAS/sleep device not
unlike a spin lock.  This so initialisation doesn't depend on, e.g.,
ddsrt_once.

Checking or changing thread states between "awake" and "asleep" can end
up in ddsrt_init if the thread is unknown at the time of the call.
Once really only ends up in those cases when the library is initialised
already, in which case no sleeping occurs.

In any case, the sleep is just a friendly yielding of the CPU.  Coverity
will still see the loop, just not the sleep.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-09-25 10:46:40 +02:00
Erik Boasson
81be40ec0e Fix dropping of first digit of time stamp in log
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-09-11 10:06:24 +02:00
Erik Boasson
801ae26872 Optionally include sample content in trace
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-09-11 10:06:24 +02:00
Erik Boasson
0b12ff5cfc Promote domains and Cyclone library to entities
This commit adds two entity types: a "domain", which is the parent of
participants and which is instantiated for each domain that has at least
one participant in it; and "cyclonedds", which is a representation of
the (initialized) Cyclone DDS library in the process and that is the
parent of all domain entities.  The handle of the latter is a
compile-constant, DDS_CYCLONEDDS_HANDLE.

This changes the return value from dds_get_parent when executed on a
participant: it now returns the handle of the entity representing the
domain the participant is attached to.  Two participants in the same
domain self-evidently return the same domain entity.

This allows deleting all participants in a domain by calling dds_delete
on the domain entity, or tearing down everything and deinitializing the
library by calling dds_delete on the top-level entity.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-09-11 10:06:24 +02:00
dennis-adlink
0c23eea7cb Fix compilation errors for RMW build on Windows (#248)
* Fix compilation errors for RMW build on Windows

This commit fixes the compilation errors that occur when building
the ROS2 rmw_cyclonedds_cpp module on Windows with the msvc C++
compiler. The error are fixed by adding explicit casts when calling
operations on atomics and a different syntax is used for compound
literals (the c99 syntax is not supported in msvc in C++ mode).

In additionally some warnings related to emtpy array usage
in structs are suppressed for msvc.

Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>

* Restructured macros for vendor id cast and casts for atomic operations

To improve readability I've restructured the macros that are used
for vendor id casts on msvc in c++ mode and macros for type-casting
arguments in operations on atomics (as suggested in the review of
my previous commit)

Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>

* Moved atomic function suffix to macro

Moved the suffix for (msvc) 64 bit atomic functions to the
DDSRT_ATOMIC_OP64 macro

Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
2019-09-09 14:18:13 +02:00
Erik Boasson
a632f80000 Add a global counter in ddsrt_random_init fallback
It currently uses process id and timestamp, but on a low-resolution
clock that might result in two subsequent initialisations in the same
process yielding the same seed, and that wasn't the intent.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-09-03 12:23:50 +02:00
Martin Bremmer
09eba90c90 Fixed dynlib Windows issues after merge.
Signed-off-by: Martin Bremmer <martin.bremmer@adlinktech.com>
2019-09-03 12:02:20 +02:00
Martin Bremmer
3fc777e631 Merge branch 'master' into merge2
Signed-off-by: Martin Bremmer <martin.bremmer@adlinktech.com>
2019-09-03 11:44:50 +02:00
Jeroen Koekkoek
4e741e9137 Synchronize warning flags between Xcode and Clang
Signed-off-by: Jeroen Koekkoek <jeroen@koekkoek.nl>
2019-08-23 12:37:15 +02:00
Erik Boasson
f06d3bf9ad FreeRTOS: replace DDS_TRACE+abort by DDS_FATAL
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
891fc2b12f Support multiple domains in configuration
Change the structure of the configuration file (in a backwards
compatible manner) to allow specifying configurations for multiple
domains in a file.  (Listing multiple files in CYCLONEDDS_URI was
already supported.)  A configuration specifies an id, with a default of
any, configurations for an incompatible id are ignored.

If the application specifies an id other than DDS_DOMAIN_DEFAULT in the
call to create_participant, then only configuration specifications for
Domain elements with that id or with id "any" will be used.  If the
application does specify DDS_DOMAIN_DEFAULT, then the id will be taken
from the first Domain element that specifies an id.  If none do, the
domain id defaults to 0.  Each applicable domain specification is taken
as a separate source and may override settings made previously.

All settings moved from the top-level CycloneDDS element to the
CycloneDDS/Domain element.  The CycloneDDS/Domain/Id element moved to
become the "id" attribute of CycloneDDS/Domain.  The old locations still
work, with appropriate deprecation warnings.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
966ec0dda7 Make logging config per-domain
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
7190bb3d3e Pacification of native compiler on OpenIndiana
* constness in ternary expressions
* removal of OS_MAX_INTEGER
* inclusion of dds/ddsrt/attributes.h everywhere DDS_EXPORT inline
  occurs
* _POSIX_PTHREAD_SEMANTICS in ddsperf

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
c1f3ad8a22 Eliminate domain-specific global variables
This commit moves all but a handful of the global variables into the
domain object, in particular including the DDSI configuration, globals
and all transport internal state.

The goal of this commit is not to produce the nicest code possible, but
to get a working version that can support multiple simultaneous domains.
Various choices are driven by this desire and it is expected that some
of the changes will have to be undone.  (E.g., passing the DDSI globals
into address set operations and locator printing because there is no
other way to figure out what transport to use for a given locator;
storing the transport pointer inside the locator would solve that.)

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-21 14:16:51 +02:00
Erik Boasson
e143caa106 Reduce likelihood of ddsrt_select_timeout failure
The main change is a longer sleep in the main thread while waiting for
the select call to timeout.  Still not perfect, but more practical than
checking the stacktrace of the thread calling ddsrt_select() to see
whether it has spent too much time in the call.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-06 08:32:14 +02:00
Erik Boasson
ecbee32422 Ignore EPERM on sendmsg
Some Linux versions sometimes return this (undocumented) error,
presumably because of firewalling.  Better to ignore it.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-06 08:32:14 +02:00
Erik Boasson
ecb77d481c Add network statistics to ddsrt
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-02 09:53:36 +02:00
Erik Boasson
f9808c7656 Add gathering per-thread CPU usage to ddsrt
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-08-02 09:53:36 +02:00
Erik Boasson
4e80559763 Improve multicast related defaults
* use multicast only for participant discovery if using a WiFi network
* default to using unicast for retransmits

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-07-25 11:31:51 +02:00
Erik Boasson
fda285e2f5 Add support for Solaris 2.6 on sun4m builds
It is an excellent platform for catching bugs: big-endian, slow enough
that a context switch in the middle of an operation becomes a regular
occurrence, and all that on a SMP box.  Or: I just wanted to see if it
would work.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-07-25 10:59:09 +02:00
Erik Boasson
96e09d2d4e Use ddsrt_strsep instead of ddsrt_strtok_r
The two do essentially the same think, and ddsrt_strtok_r was only used
in one place.  (Triggered by Solaris 2.6 not providing strtok_r.)

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-07-25 10:59:09 +02:00
Erik Boasson
0d33462664 Use uintptr_t rather than uintmax_t for thread ids
On 32-bit machines uintmax_t is likely to be slower than uintptr_t, and
for that reason, using an uintmax_t for a thread id seems highly
unlikely.  For the current platforms, uintptr_t works.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-07-25 10:59:09 +02:00
Thijs Sassen
2fe4a4ca35 Fixed failing FreeRTOS target due to recent code refactors
Signed-off-by: Thijs Sassen <thijs.sassen@adlinktech.com>

Adjusted the close methode not to expand by the lwip close macro and added a check for DDSI_INCLUDE_SSM to match the correct pid table size.

Signed-off-by: Thijs Sassen <thijs.sassen@adlinktech.com>
2019-07-11 22:45:17 +02:00
Kurtulus Oksuztepe
a65d3db7c8 Dynamic library loading functions were added to ddsrt
ddsrt_dlopen ddsrt_dlclose ddsrt_dlsym and ddsrt_dlerror functions can be used
in posix and windows platforms that support dynamic loading

Signed-off-by: Kurtulus Oksuztepe <kurtulus.oksuztepe@adlinktech.com>
2019-07-01 16:05:22 +02:00
Erik Boasson
0e888eb2ec Special-case size-1 sequential hopscotch hash table
Rather than allocate a HH_HOP_RANGE large array of buckets, allocate
just 1 if the initial size is 1, then jump to HH_HOP_RANGE as soon as a
second element is added to the table.  There are quite a few cases where
hash tables are created where there never be more than 1 (or even 0)
elements in the table (e.g., a writer without readers, a reader for a
keyless topic).

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-06-28 12:47:27 +02:00
Erik Boasson
f61b2d54da Add some atomic operations returning old value
* ddsrt_atomic_inc32_ov
* ddsrt_atomic_add32_ov
* ddsrt_atomic_sub32_ov

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-06-28 12:47:27 +02:00
Erik Boasson
559c325307 Emulate 64-bit atomic ops if hardware doesn't support them
Signed-off-by: Erik Boasson <eb@ilities.com>
2019-06-28 12:47:27 +02:00
Erik Boasson
c6c5a872eb Trivial changes for thread sanitizer
Thread sanitizer warns about reads and writes of variables that are
meant to be read without holding a lock:

* Global "keep_going" is now a ddsrt_atomic_uint32_t

* Thread "vtime" is now a ddsrt_atomic_uint32_t

Previously the code relied on the assumption that a 32-bit int would be
treated as atomic, now that is all wrapped in ddsrt_atomic_{ld,st}32.
These being inline functions doing exactly the same thing, there is no
functional change, but it does allow annotating the loads and stores for
via function attributes on the ddsrt_atomic_{ld,st}X.

The concurrent hashtable implementation is replaced by a locked version
of the non-concurrent implementation if thread sanitizer is used.  This
changes eliminates the scores of problems signalled by thread sanitizer
in the GUID-to-entity translation and the key-to-instance id lookups.

Other than that, this replaces a flag used in a waitset test case to be
a ddsrt_atomic_uint32_t.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-06-28 12:47:27 +02:00
Erik Boasson
32b683bf37 Enable "missing prototypes" warning for gcc, clang
Missing prototypes for exported functions cause a really huge issue on
Windows.  Enabling the "missing prototypes" warning makes it much easier
to catch this problem.  Naturally, any warnings caused by this have been
fixed.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-06-13 12:54:35 +02:00
Erik Boasson
a4d8aba4f9 Add limited support for QoS changes
This commit adds support for changing all mutable QoS except those that
affect reader/writer matching (i.e., deadline, latency budget and
partition).  This is simply because the recalculation of the matches
hasn't been implemented yet, it is not a fundamental limitation.

Implementing this basically forced fixing up a bunch of inconsistencies
in handling QoS in entity creation.  A silly multi-process ping-pong
test built on changing the value of user data has been added.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-06-13 12:54:35 +02:00
Erik Boasson
3322fc086d Table-driven parameter list handling
The old parameter list parsing was a mess of custom code with tons of
duplicated checks, even though parameter list parsing really is a fairly
straightforward affair.  This commit changes it to a mostly table-driven
implementation, where the vast majority of the settings are handled by a
generic deserializer and the irregular ones (like reliability, locators)
are handled by custom functions.  The crazy ones (IPv4 address and port
rely on additional state and are completely special-cased).

Given these tables, the serialization, finalisation, validation,
merging, unalias'ing can all be handled by a very small amount of custom
code and an appropriately defined generic function for the common cases.
This also makes it possible to have all QoS validation in place, and so
removes the need for the specialized implementations for the various
entity kinds in the upper layer.

QoS inapplicable to an entity were previously ignored, allowing one to
have invalid values set in a QoS object when creating an entity,
provided that the invalid values are irrelevant to that entity.  Whether
this is a good thing or not is debatable, but certainly it is a good
thing to avoid copying in inapplicable QoS settings.  That in turn means
the behaviour of the API can remain the same.

It does turn out that the code used to return "inconsistent QoS" also
for invalid values.  That has now been rectified, and it returns
"inconsistent QoS" or "bad parameter" as appropriate.  Tests have been
updated accordingly.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-06-10 10:45:53 +02:00
Erik Boasson
ffbf3d7843 Avoid implementation defined types, e.g. unsigned
There are some cases where "int" or "unsigend" actually makes sense, but
in a large number of cases it is really supposed to be either a 32-bit
integer, or, in some cases, at least a 32-bit integer.  It is much to be
preferred to be clear about this.

Another reason is that at least some embedded platforms define, e.g.,
int32_t as "long" instead of "int".  For the ones I am aware of the
"int" and "long" are actually the same 32-bit integer, but that
distinction can cause trouble with printf format specifications.  So
again a good reason to be consistent in avoiding the
implementation-defined ones.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-06-10 10:42:52 +02:00
Erik Boasson
13480616e0 Consistent code formatting for the core code
Code formatting was quite a mess (different indentation, completely
different ideas on where opening braces should go, spacing in various
places, early out versus single return or goto-based error handling,
&c.).  This commit cleans it up.

A few doxygen comment fixes allowed turning on Clang's warnings for
doxygen comments, so those are no enabled by default as least on
Xcode-based builds.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-06-10 10:42:52 +02:00
Erik Boasson
19aec98b8a Clean up return code types
* Remove dds_return_t / dds_retcode_t distinction (now there is only
  dds_return_t and all error codes are always negative)

* Remove Q_ERR_... error codes and replace them by DDS_RETCODE_...
  ones so that there is only one set of error codes

* Replace a whole bunch "int" return types that were used to return
  Q_ERR_... codes by "dds_return_t" return types

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-06-10 10:42:52 +02:00
Erik Boasson
7fbe49c267 Fix broken check for multi-process support
With the introduction of FreeRTOS support, the requirement that the
underlying platform supports multiple processes became untenable, and
therefore the code for dealing with multiple processes is now
automatically detected via some CMake code and then effectuated in the
sources via a macro (DDSRT_HAVE_MULTI_PROCESS).  Unfortunately, a typo
resulted in all platforms being treated as if without support for
multiple processes.  Fortunately, at this stage the only consequence was
the disabling of the first few multi-process tests.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-05-29 13:20:37 +02:00
Erik Boasson
35fcc013af use clock_gettime on macOS >= 10.12
Since macOS 10.12, the POSIX clock_gettime interface with various
different clocks is supported on macOS, so use those when building for
10.12 or newer.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-05-24 07:48:45 +02:00
Erik Boasson
a6d92aac8c config input handling improvements
This commit changes a few things in the config handling:

* When reading the configuration from multiple sources, a source can now
  override settings already set by a preceding source for settings that
  are not lists.  Previously, trying to change the value of a setting in a
  subsequence file would be considered an error, just like trying to set
  the value of a particular setting multiple times in a single
  configuration file.

* A configuration fragment in CYCLONEDDS_URI now no longer requires the
  CycloneDDS top-level tag to be specified.  If it is missing it will be
  assumed.  This is only true for configuration fragments contained in
  CYCLONEDDS_URI, not for data read from a file.

* A configuration fragment in CYCLONEDDS_URI no longer requires that all
  elements are properly closed: a missing close tag is treated as-if it
  is the end of the fragment and any elements are implicitly closed.
  Again this does not apply to files.

* The configuration dump now lists explicitly which sources affected
  each setting, with a default value indicated by an empty set.

The result of the latter two is that one can almost pretend that it is a
sane format instead of XML.  For example, if one would like to override
tracing settings, one could just write:

  CYCLONEDDS_URI="$CYCLONEDDS_URI,<Tracing><Verbosity>finest"

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-05-24 07:48:45 +02:00
Kurtulus Oksuztepe
98cf8e2ae5 Security API has been added for OMG DDS Security Specification compliance.
This API consists of only header files which are grabbed from spec IDL. The dynamically loaded plugins should implement API functions.

Built-in plugins will be added to the repository later. Third party plugins will also be able to be integrated.

Signed-off-by: Kurtulus Oksuztepe <kurtulus.oksuztepe@adlinktech.com>
2019-05-23 18:51:23 +02:00
Erik Boasson
326ab93572 Allow closing config elems with </> if from envvar
The Cyclone DDS configuration is in principle an XML document, but it is
possible to write configuration fragments directly in the CYCLONEDDS_URI
environment variable.  In that case, it is quite annoying to have to
enter the full closing tags all the time, and so it now allows closing
elements with a simple </> when not reading them from a file.

While it would be trivial to also allow this when reading the
configuration from a file, it seems that promulgating invalid XML would
be bad form ... and besides, in that case editors can help keep
everything in order.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-05-23 18:51:23 +02:00
Erik Boasson
f601beb907 don't modify input string in XML parser
The XML parser has two modes: it can parse a file or parse a
caller-owned string.  In the former case, it owns its buffer and shifts
the contents to make room for more data read in from the file.  This
shifting may not happen when parsing a string.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-05-23 18:51:23 +02:00
Erik Boasson
ae323ab0ca make internal header files more C++ friendly
Generally one doesn't need to include any internal header files in an
application, but the (unstable) interface for application-defined sample
representation and serialization does require including some.  It turns
out a keyword clash had to be resolved (typename => type_name) and that
a whole bunch of them were missing the #ifdef __cplusplus / extern "C"
bit.

It further turned out that one had to pull in nearly all of the type
definitions, including some typedefs that are illegal in C++, e.g.,

  typedef struct os_sockWaitset *os_sockWaitset;

C++ is right to forbid this, but Cyclone's header files were wrong to
force inclusion of so much irrelevant stuff.  This commit leaves these
typedefs in place, but eliminates a few header file inclusions to avoid
the problem.

Signed-off-by: Erik Boasson <eb@ilities.com>
2019-05-23 18:51:23 +02:00
Martin Bremmer
14a62ae3cb Removed expand_envvars.h
Signed-off-by: Martin Bremmer <martin.bremmer@adlinktech.com>
2019-05-23 18:51:23 +02:00
Martin Bremmer
5a04db973a Be sure to not trigger the SIGCHLD
Signed-off-by: Martin Bremmer <martin.bremmer@adlinktech.com>
2019-05-23 18:51:23 +02:00
Martin Bremmer
955f933ae2 Fixed proc compile warning.
Signed-off-by: Martin Bremmer <martin.bremmer@adlinktech.com>
2019-05-23 18:51:23 +02:00
Martin Bremmer
0779399391 Moved expand_envvars.
Signed-off-by: Martin Bremmer <martin.bremmer@adlinktech.com>
2019-05-23 18:51:23 +02:00
Martin Bremmer
597ca25287 Multi Process Testing framework
Signed-off-by: Martin Bremmer <martin.bremmer@adlinktech.com>
2019-05-23 18:51:23 +02:00
Martin Bremmer
5a8197fa2b Rudimentary process management.
Signed-off-by: Martin Bremmer <martin.bremmer@adlinktech.com>
2019-05-23 18:51:23 +02:00