This already was leaking out in the interface, so this name change was
needed too. The relationship between plist and xqos being so intimate,
doing the one but not the other made no sense.
Signed-off-by: Erik Boasson <eb@ilities.com>
This commit changes the implementation of topics so that multiple topic
entities can exist in a single participant for the same topic.
Different entities may refer to different topic implementations
(sertopics, akin to a type support in the DDS specification). All
entities (for the same participant) always have the same QoS, via the
new "ktopic" table in the participant.
Readers and writers are bound to a topic entity and inherit its
properties. If a topic comes in two definitions, say one for C and one
for C++, one can have a single participant with a reader delivering the
data in C representation and another reader delivering it in C++
representation.
This changes the behaviour of create_topic and find_topic: these now (on
successful return) always return a new entity (and thus with a unique
handle), where previously these would simply return a existing one when
possible.
This also requires some small additions to the sertopic/serdata
interface.
Signed-off-by: Erik Boasson <eb@ilities.com>
And updated dds_read_instance_mask and dds_take_instance_mask to work with DDS_READ_WITHOUT_LOCK
Signed-off-by: Thijs Sassen <thijs.sassen@adlinktech.com>
This commit adds support for the liveliness QoS for the liveliness
kinds automatic and manual-by-participant. It also implements the
lease_duration from this QoS, which was ignored until now. In the
api the function dds_assert_liveliness is added to assert liveliness
on a participant, which can be used when using liveliness kind
manual-by-participant.
Liveliness kind manual-by-topic is not yet supported, this will be
added in a later commit.
* Proxy participants now have 2 fibheaps to keep leases: one for leases
of pwrs with automatic liveliness and one for leases of the pwrs with
manual-by-participant liveliness (both protected by the proxypp lock).
The minl_auto and minl_man members represent the shortest lease from
these fibheaps and these leases are renewed when receiving data.
Replacing the minl_ leases is now done by replacing the lease object
(atomic ptr) with delayed deletion of the old lease using the gc.
* Proxy writers are set not-alive when the lease expired, and reset to
alive then data is received. When data is received by a pwr, the other
pwrs in the proxypp might also be set alive. I think the specification
is not clear at this point, and for now I have not implemented this
* I refactored out the counter for man-by-pp proxy writers and improved
locking when updating the min-leases on the proxy participant, so I
think this fixes the race conditions.
Some additional tests are required, e.g. to test the not-alive->alive
transition for pwrs. I will add these in short term, as well as the
implementation of the manual-by-topic liveliness kind.
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
For targets that do not support ddsrt_setenv and ddsrt_getenv, an alternative
method is needed to supply an application specific configuration. One way to
implement this, is to add a function for creating a domain with a string
arguments, which needs to be called before any call to dds_create_participant
for given domain identifier.
The function dds_create_domain has been added, which has as arguments a domain
identifier and a configuration string. The string is treated in the same way
as the string that is retrieved from the environment variable, in that it may
containt a comma separated list of file names and/or XML fragments for the
configuration.
Two tests have been added. One limits the number of participants to two and
verifies that creating a third participant fails. The other tests checks
incorrect calls to dds_create_domain.
An assert in dds_handle_delete has been weakened.
Signed-off-by: Frans Faase <frans.faase@adlinktech.com>
The default RHC implementation is not always ideal and rather than
trying to squeeze everything in a fixed interface it makes more sense to
allow the caller to provide an arbitrary implementation of the
interface.
This is not yet a stable interface.
Signed-off-by: Erik Boasson <eb@ilities.com>
This makes it possible to write one's own RHC implementation. This is
not a stable interface. It shuffles a few things around and renames
some types used throughout the code to stick to having a "dds" prefix
for all the external things.
Signed-off-by: Erik Boasson <eb@ilities.com>
This commit adds two entity types: a "domain", which is the parent of
participants and which is instantiated for each domain that has at least
one participant in it; and "cyclonedds", which is a representation of
the (initialized) Cyclone DDS library in the process and that is the
parent of all domain entities. The handle of the latter is a
compile-constant, DDS_CYCLONEDDS_HANDLE.
This changes the return value from dds_get_parent when executed on a
participant: it now returns the handle of the entity representing the
domain the participant is attached to. Two participants in the same
domain self-evidently return the same domain entity.
This allows deleting all participants in a domain by calling dds_delete
on the domain entity, or tearing down everything and deinitializing the
library by calling dds_delete on the top-level entity.
Signed-off-by: Erik Boasson <eb@ilities.com>
There were inconsistencies in the order in which entity locks were taken
when multiple entities needed to be locked at the same time. In most
cases, the order was first locking entity X, then locking the parent
entity of X. However, in some cases the order was reversed, a likely
cause of deadlocks.
This commit sorts these problems, and in particular propagating
operations into children. The entity refcount is now part of the handle
administration so that it is no longer necessary to lock an entity to
determine whether it is still allowed to be used (previously it had to
check the CLOSED flag afterward). This allows recursing into the
children while holding handles and the underlying objects alive, but
without violating lock order.
Attendant changes that would warrant there own commits but are too hard
to split off:
* Children are now no longer in a singly linked list, but in an AVL
tree; this was necessary at some intermediate stage to allow unlocking
an entity and restarting iteration over all children at the "next"
child (all thanks to the eternally unique instance handle);
* Waitsets shifted to using arrays of attached entities instead of
linked lists; this was a consequence of dealing with some locking
issues in reading triggers and considering which operations on the
"triggered" and "observed" sets are actually needed.
* Entity status flags and waitset/condition trigger counts are now
handled using atomic operations. Entities are now classified as
having a "status" with a corresponding mask, or as having a "trigger
count" (conditions). As there are fewer than 16 status bits, the
status and its mask can squeeze into the same 32-bits as the trigger
count. These atomic updates avoid the need for a separate lock just
for the trigger/status values and results in a significant speedup
with waitsets.
* Create topic now has a more rational behaviour when multiple
participants attempt to create the same topic: each participant now
gets its own topic definition, but the underlying type representation
is shared.
Signed-off-by: Erik Boasson <eb@ilities.com>
* dds_set_allocator
* dds_set_aligned_allocator
The intent behind them is good, but the approach too primitive ... there
is far more work to be done for managing dynamic allocation in a
meaningful way.
Signed-off-by: Erik Boasson <eb@ilities.com>
The old parameter list parsing was a mess of custom code with tons of
duplicated checks, even though parameter list parsing really is a fairly
straightforward affair. This commit changes it to a mostly table-driven
implementation, where the vast majority of the settings are handled by a
generic deserializer and the irregular ones (like reliability, locators)
are handled by custom functions. The crazy ones (IPv4 address and port
rely on additional state and are completely special-cased).
Given these tables, the serialization, finalisation, validation,
merging, unalias'ing can all be handled by a very small amount of custom
code and an appropriately defined generic function for the common cases.
This also makes it possible to have all QoS validation in place, and so
removes the need for the specialized implementations for the various
entity kinds in the upper layer.
QoS inapplicable to an entity were previously ignored, allowing one to
have invalid values set in a QoS object when creating an entity,
provided that the invalid values are irrelevant to that entity. Whether
this is a good thing or not is debatable, but certainly it is a good
thing to avoid copying in inapplicable QoS settings. That in turn means
the behaviour of the API can remain the same.
It does turn out that the code used to return "inconsistent QoS" also
for invalid values. That has now been rectified, and it returns
"inconsistent QoS" or "bad parameter" as appropriate. Tests have been
updated accordingly.
Signed-off-by: Erik Boasson <eb@ilities.com>
All this duplication was rather useless: the values are standardized
anyway and the conversion was a simple type cast without any check.
This commit unifies the definitions.
* DDSI now uses the definitions of the various QoS "kind" values from
the header file;
* The durations in the QoS objects are no longer in wire-format
representation, the conversions now happen only in conversion to/from
wire format;
* The core DDSI stack no longer uses DDSI time representations for time
stamps, instead using the "native" one;
* QoS policy ids duplication has been eliminated, again using the IDs
visible in the API -- the actual values are meaningless to the DDSI
stack anyway.
Signed-off-by: Erik Boasson <eb@ilities.com>
Code formatting was quite a mess (different indentation, completely
different ideas on where opening braces should go, spacing in various
places, early out versus single return or goto-based error handling,
&c.). This commit cleans it up.
A few doxygen comment fixes allowed turning on Clang's warnings for
doxygen comments, so those are no enabled by default as least on
Xcode-based builds.
Signed-off-by: Erik Boasson <eb@ilities.com>
* Remove dds_return_t / dds_retcode_t distinction (now there is only
dds_return_t and all error codes are always negative)
* Remove Q_ERR_... error codes and replace them by DDS_RETCODE_...
ones so that there is only one set of error codes
* Replace a whole bunch "int" return types that were used to return
Q_ERR_... codes by "dds_return_t" return types
Signed-off-by: Erik Boasson <eb@ilities.com>
The CDR deserializer failed to check it was staying within the bounds of
the received data, and it turns out it also was inconsistent in its
interpretation of the (undocumented) serializer instructions. This
commit adds some information on the instruction format obtained by
reverse engineering the code and studying the output of the IDL
preprocessor, and furthermore changes a lot of the types used in the
(de)serializer code to have some more compiler support. The IDL
preprocessor is untouched and the generated instructinos do exactly the
same thing (except where change was needed).
The bulk of this commit replaces the implementation of the
(de)serializer. It is still rather ugly, but at least the very long
functions with several levels of nested conditions and switch statements
have been split out into multiple functions. Most of these have single
call-sites, so the compiler hopefully inlines them nicely.
The other important thing is that it adds a "normalize" function that
validates the structure of the CDR and performs byteswapping if
necessary. This means the deserializer can now assume a well-formed
input in native byte-order. Checks and conditional byteswaps have been
removed accordingly.
It changes some types to make a compile-time distinction between
read-only, native-endianness input, a native-endianness output, and a
big-endian output for dealing with key hashes. This should reduce the
risk of accidentally mixing endianness or modifying an input stream.
The preprocessor has been modified to indicate the presence of unions in
a topic type in the descriptor flags. If a union is present, any
memory allocated in a sample is freed first and the sample is zero'd out
prior to deserializing the new value. This is to prevent reading
garbage pointers for strings and sequences when switching union cases.
The test tool has been included in the commit but it does not get run by
itself. Firstly, it requires the presence of OpenSplice DDS as an
alternative implementation to check the CDR processing against.
Secondly, it takes quite a while to run and is of no interest unless one
changes something in the (de)serialization.
Finally, I have no idea why there was a "CDR stream" interface among the
public functions. The existing interfaces are fundamentally broken by
the removal of arbitrary-endianness streams, and the interfaces were
already incapable of proper error notification. So, they have been
removed.
Signed-off-by: Erik Boasson <eb@ilities.com>
The name parameter and the name in the sertopic parameter had to match
because it used the one as a key in a lookup checking whether the topic
exists already, and the other as key for the nodes in that index. As
the name is (currently) included in the sertopic, it shouldn't be passed
in separately as well.
Signed-off-by: Erik Boasson <eb@ilities.com>
Changes the semantics of dds_qget_{user,group,topic}data to always
append a 0 byte to any non-empty value without counting it in the size.
(An empty value is always represented by a null pointer and a size of
0). The advantage is that any code treating the data as the octet
sequence it formally is will do exactly the same, but any code written
with the knowledge that it should be a string can safely interpret it as
one.
Signed-off-by: Erik Boasson <eb@ilities.com>
Extend the endpoint built-in topic data with the participant instance
handle (the GUID was already present). Having the instance handle
available makes it trivial to look up the participant, whereas a lookup
of the GUID is rather impractical.
Signed-off-by: Erik Boasson <eb@ilities.com>
Adds a new "ignorelocal" QoS to the readers/writers to ignore local
matching readers/writers, with three settings:
* DDS_IGNORELOCAL_NONE: default
* DDS_IGNORELOCAL_PARTICIPANT: ignores readers/writers in the same
participant
* DDS_IGNORELOCAL_PROCESS: ignores readers/writers in the same process
These can be set/got using dds_qset_ignorelocal and
dds_qget_ignorelocal.
If a matching reader or writer is ignored because of this setting, it is
as-if that reader or writer doesn't exist. No traffic will be generated
or data retained on its behalf.
There are no consequences for interoperability as this is (by
definition) a local affair.
Signed-off-by: Erik Boasson <eb@ilities.com>
The old entity handle mechanism suffered from a number of problems, the
most terrible one being that it would only ever allocate 1000 handles
(not even have at most 1000 in use at the same time). Secondarily, it
was protected by a single mutex that actually does show up as a limiting
factor in, say, a polling-based throughput test with small messages.
Thirdly, it tried to provide for various use cases that don't exist in
practice but add complexity and overhead.
This commit totally rewrites the mechanism, by replacing the old array
with a hash table and allowing a near-arbitrary number of handles as
well as reuse of handles. It also removes the entity "kind" bits in the
most significant bits of the handles, because they only resulted in
incorrect checking of argument validity. All that is taken out, but
there is still more cleaning up to be done. It furthermore removes an
indirection in the handle-to-entity lookup by embedding the
"dds_handle_link" structure in the entity.
Handle allocation is randomized to avoid the have a high probability of
quickly finding an available handle (the total number of handles is
limited to a number much smaller than the domain from which they are
allocated). The likelihood of handle reuse is still dependent on the
number of allocated handles -- the fewer handles there are, the longer
the expected time to reuse. Non-randomized handles would give a few
guarantees more, though.
It moreover moves the code from the "util" to the "core/ddsc" component,
because it really is only used for entities, and besides the new
implementation relies on the deferred freeing (a.k.a. garbage collection
mechanism) implemented in the core.
The actual handle management has two variants, selectable with a macro:
the preferred embodiment uses a concurrent hash table, the actually used
one performs all operations inside a single mutex and uses a
non-concurrent version of the hash table. The reason the
less-predeferred embodiment is used is that the concurrent version
requires the freeing of entity objects to be deferred (much like the
GUID-to-entity hash tables in DDSI function, or indeed the key value to
instance handle mapping). That is a fair bit of work, and the
non-concurrent version is a reasonable intermediate step.
Signed-off-by: Erik Boasson <eb@ilities.com>
- Replace os_result by dds_retcode_t and move DDS return code defines down.
Eliminates the need to convert between different return code types.
- Move dds_time_t down and remove os_time.
Eliminates the need to convert between different time representations and
reduces code duplication.
- Remove use of Microsoft source-code annotation language (SAL).
SAL annotations are Microsoft specific and not very well documented. This
makes it very difficult for contributers to write.
- Rearrange the abstraction layer to be feature-based. The previous layout
falsely assumed that the operating system dictates which implementation is
best suited. For general purpose operating systems this is mostly true, but
embedded targets require a slightly different approach and may not even offer
all features. The new layout makes it possible to mix-and-match feature
implementations and allows for features to not be implemented at all.
- Replace the os prefix by ddsrt to avoid name collisions.
- Remove various portions of unused and unwanted code.
- Export thread names on all supported platforms.
- Return native thread identifier on POSIX compatible platforms.
- Add timed wait for condition variables that takes an absolute time.
- Remove system abstraction for errno. The os_getErrno and os_setErrno were
incorrect. Functions that might fail now simply return a DDS return code
instead.
- Remove thread-specific memory abstraction. os_threadMemGet and accompanying
functions were a mess and their use has been eliminated by other changes in
this commit.
- Replace attribute (re)defines by ddsrt_ prefixed equivalents to avoid name
collisions and problems with faulty __nonnull__ attributes.
Signed-off-by: Jeroen Koekkoek <jeroen@koekkoek.nl>
Listener/status management invocation was rather expensive, and
especially the cost of checking listeners, then setting status flags and
triggering waitsets ran into severe lock contention.
A major cost was the repeated use of dds_entity_lock and
dds_entity_unlock, these have been eliminated. Another cost was that
each time an event occurred (with DATA_AVAILABLE the most problematic
one) it would walk the chain of ancestors to see if any had a relevant
listener, and only if none of them had any, it would set the status
flags.
The locking/unlocking of the entity has been eliminated by moving the
listener/status flag manipulation from the general entity lock to its
m_observers_lock. That lock has a much smaller scope, and consequently
contention has been significantly reduced.
Instead of walking the entity hierarchy looking for listeners, an entity
now inherits the ancestors' listeners. The set_listener operation has
been made a little more complicated by the need to not only set the
listeners for the specified entity, but to also update any inherited
listeners its descendants.
The commit is a bit larger than strictly needed ... I've started
reformatting the code to reduce the variety of styles ... as there I
haven't been able to find a single tool that does what I want, it may
well end up as manual work.
Signed-off-by: Erik Boasson <eb@ilities.com>
Move details of built-in topics out of the DDSI core (so the only hooks
remain). For this, rtps_term had to be split, so now it is "stop"
followed by "fini".
Add a notion of local writers that are not bound to a participant ("local
orphans"), so that the local built-in topic writers can be created during
initialization. This eliminates the "builtin" participant. This
uncovered in inconsistency in the unit tests: on the one hand, a newly
created participant is expected to have no child entities; on the other
hand, the built-in topics were expected to be returned by find_topic ...
This inconsistency has been resolved by creating them lazily and
accepting that find_topic can't return them until they have been
created. Special code was in place in dds_create_reader anyway, so it
is not expected to have any real consequence for applications.
Use a special WHC implementation that regenerates the data on the fly
using the internal discovery tables of DDSI, so that the samples are only
stored by readers. This eliminates the memory overhead of that existed
previously when the WHC of the writers stored the data.
No longer return topic name and type name in the built-in topics, they
have been extracted already and are not accessible through the normal
interface but do cause problems when comparing QoS.
Signed-off-by: Erik Boasson <eb@ilities.com>
Make function names more consistent:
* ``dds_topic_{get,set}_filter`` -> ``dds_{get,set}_topic_filter``
* ``dds_{get,set}_enabled_status`` -> ``dds_{get,set}_status_mask``
* ``dds_instance_lookup`` -> ``dds_lookup_instance``
Plus removal of defunct ddsv2.h.
The main naming scheme is OPER_TYPE (i.e., dds_create_participant) but the listener operations were named TYPE_OPER (i.e., dds_listener_create). This commit brings the listener scheme in line with the rest, retaining the old names as a deprecated interface.
Signed-off-by: Erik Boasson <eb@ilities.com>
The main naming scheme is OPER_TYPE (i.e., dds_create_participant) but the QoS operations were named TYPE_OPER (i.e., dds_qos_create). This commit brings the QoS scheme in line with the rest, retaining the old names as a deprecated interface.
Signed-off-by: Erik Boasson <eb@ilities.com>
