The DATA_AVAILABLE status was reset by read and take while holding the
upper-layer reader lock, but after completing the read/take operation on
the RHC. As data can be written into the RHC without holding the
upper-layer reader lock, new data could arrive in between the
reading/taking and the resetting of the DATA_AVAILABLE status, leading
to a missed detection. Resetting DATA_AVAILABLE prior to accessing the
RHC solves this.
Signed-off-by: Erik Boasson <eb@ilities.com>
Changes the semantics of dds_qget_{user,group,topic}data to always
append a 0 byte to any non-empty value without counting it in the size.
(An empty value is always represented by a null pointer and a size of
0). The advantage is that any code treating the data as the octet
sequence it formally is will do exactly the same, but any code written
with the knowledge that it should be a string can safely interpret it as
one.
Signed-off-by: Erik Boasson <eb@ilities.com>
As was the plan with the introduction of ddsrt; this includes renaming
the identifiers to match the capitalization style and removes old junk.
Signed-off-by: Erik Boasson <eb@ilities.com>
Extend the endpoint built-in topic data with the participant instance
handle (the GUID was already present). Having the instance handle
available makes it trivial to look up the participant, whereas a lookup
of the GUID is rather impractical.
Signed-off-by: Erik Boasson <eb@ilities.com>
Adds a new "ignorelocal" QoS to the readers/writers to ignore local
matching readers/writers, with three settings:
* DDS_IGNORELOCAL_NONE: default
* DDS_IGNORELOCAL_PARTICIPANT: ignores readers/writers in the same
participant
* DDS_IGNORELOCAL_PROCESS: ignores readers/writers in the same process
These can be set/got using dds_qset_ignorelocal and
dds_qget_ignorelocal.
If a matching reader or writer is ignored because of this setting, it is
as-if that reader or writer doesn't exist. No traffic will be generated
or data retained on its behalf.
There are no consequences for interoperability as this is (by
definition) a local affair.
Signed-off-by: Erik Boasson <eb@ilities.com>
The DDSI reader/writer pointers are now returned as out parameters
instead of as a return value, so that the upper-layer reference is set
before any listener can be invoked.
Signed-off-by: Erik Boasson <eb@ilities.com>
It was called strangely early in the deleting of the reader, even before
the DDSI reader was no longer being accessed by other threads. The
immediate and obvious problem is that it resets the pointer to the
upper-layer entity even though this can still be dereferenced in
invoking a listener, resulting in a crash.
Secondly it blocks until there are no listener calls any more (and the
resetting of that pointer will prevent any further listener
invocations), but a similar piece of logic is already in generic entity
code that resets the mask and then waits for all listener invocations to
complete. Having both is a problem.
Signed-off-by: Erik Boasson <eb@ilities.com>
If a (proxy) writer delivers data to a reader that has a data_available
listener calling read/take while that reader is being deleted, blocked
in set_listener waiting for the listeners to complete, then a deadlock
can occur:
* listener calling read/take then attempt to lock reader;
* deleting the reader locks the reader, then waits for the listeners to
complete while holding the lock
This commits unlocks the reader before waiting for the listeners to
complete.
Signed-off-by: Erik Boasson <eb@ilities.com>
Remove all the "if asleep then awake ..." stuff from the code by making
awake/asleep calls nestable, whereas before it "awake ; awake" really
meant a transition through "asleep". This self-evidently necessitates
fixing those places where the old behaviour was relied on upon, but
fortunately those are few.
Signed-off-by: Erik Boasson <eb@ilities.com>
The old entity handle mechanism suffered from a number of problems, the
most terrible one being that it would only ever allocate 1000 handles
(not even have at most 1000 in use at the same time). Secondarily, it
was protected by a single mutex that actually does show up as a limiting
factor in, say, a polling-based throughput test with small messages.
Thirdly, it tried to provide for various use cases that don't exist in
practice but add complexity and overhead.
This commit totally rewrites the mechanism, by replacing the old array
with a hash table and allowing a near-arbitrary number of handles as
well as reuse of handles. It also removes the entity "kind" bits in the
most significant bits of the handles, because they only resulted in
incorrect checking of argument validity. All that is taken out, but
there is still more cleaning up to be done. It furthermore removes an
indirection in the handle-to-entity lookup by embedding the
"dds_handle_link" structure in the entity.
Handle allocation is randomized to avoid the have a high probability of
quickly finding an available handle (the total number of handles is
limited to a number much smaller than the domain from which they are
allocated). The likelihood of handle reuse is still dependent on the
number of allocated handles -- the fewer handles there are, the longer
the expected time to reuse. Non-randomized handles would give a few
guarantees more, though.
It moreover moves the code from the "util" to the "core/ddsc" component,
because it really is only used for entities, and besides the new
implementation relies on the deferred freeing (a.k.a. garbage collection
mechanism) implemented in the core.
The actual handle management has two variants, selectable with a macro:
the preferred embodiment uses a concurrent hash table, the actually used
one performs all operations inside a single mutex and uses a
non-concurrent version of the hash table. The reason the
less-predeferred embodiment is used is that the concurrent version
requires the freeing of entity objects to be deferred (much like the
GUID-to-entity hash tables in DDSI function, or indeed the key value to
instance handle mapping). That is a fair bit of work, and the
non-concurrent version is a reasonable intermediate step.
Signed-off-by: Erik Boasson <eb@ilities.com>
The introduction of properly functioning query conditions adds some
overhead, this commit removes some of that cost by avoiding some calls
to update_conditions when there are no query conditions.
It also removes the has_changed field from the instance, instead using a
local boolean to track whether DATA_AVAILABLE should be raised or not.
Signed-off-by: Erik Boasson <eb@ilities.com>
Deleting a writer causes unregisters (and possibly disposes) in the rest
of the network, and these updates to the instances should trigger
DATA_AVAILABLE.
Signed-off-by: Erik Boasson <eb@ilities.com>
The rtps_init function used to initialize all data structures and start
all threads used by the protocol stack, allowing discovery of remote
entities before the built-in topic data structures were initialized.
(Very) early discovery of a remote participant thus led to a crash.
This commit splits the initialisation, providing a separate function for
starting, in particular, the threads receiving data from the network.
In terms of threads created, it matches exactly with the rtps_stop /
rtps_fini split that already existed to address the exact same problem
on termination.
Signed-off-by: Erik Boasson <eb@ilities.com>
This adds an Internal/EnableExpensiveChecks setting for enabling some or all expensive run-time checks to avoid a massive slowdown when assertions are enabled at compile-time. Currently these cover only the writer and reader-history cache checking.
Signed-off-by: Erik Boasson <eb@ilities.com>
- Replace os_result by dds_retcode_t and move DDS return code defines down.
Eliminates the need to convert between different return code types.
- Move dds_time_t down and remove os_time.
Eliminates the need to convert between different time representations and
reduces code duplication.
- Remove use of Microsoft source-code annotation language (SAL).
SAL annotations are Microsoft specific and not very well documented. This
makes it very difficult for contributers to write.
- Rearrange the abstraction layer to be feature-based. The previous layout
falsely assumed that the operating system dictates which implementation is
best suited. For general purpose operating systems this is mostly true, but
embedded targets require a slightly different approach and may not even offer
all features. The new layout makes it possible to mix-and-match feature
implementations and allows for features to not be implemented at all.
- Replace the os prefix by ddsrt to avoid name collisions.
- Remove various portions of unused and unwanted code.
- Export thread names on all supported platforms.
- Return native thread identifier on POSIX compatible platforms.
- Add timed wait for condition variables that takes an absolute time.
- Remove system abstraction for errno. The os_getErrno and os_setErrno were
incorrect. Functions that might fail now simply return a DDS return code
instead.
- Remove thread-specific memory abstraction. os_threadMemGet and accompanying
functions were a mess and their use has been eliminated by other changes in
this commit.
- Replace attribute (re)defines by ddsrt_ prefixed equivalents to avoid name
collisions and problems with faulty __nonnull__ attributes.
Signed-off-by: Jeroen Koekkoek <jeroen@koekkoek.nl>
The "rhc" test runs a random sequence of operations (writes, reads, &c.)
through an RHC with conditions attached to it. All possible state masks
are used, and query conditions are tried with a condition that only
tests the key value, and one that tests attribute values. It depends on
the internal checking logic of the RHC, which is currently enabled only
in Debug builds because of the associated run-time overhead.
Signed-off-by: Erik Boasson <eb@ilities.com>
Various details:
- index replaced throughout by bitmask
- caching a single sample in RHC for deserialising samples when query conditions are used
- short-circuiting the trivial case of an instance matched neither before nor after a change to its state
- combining inc/dec counters into a single delta in condition evaluation
- major speed-up of rhc_check_counts by not checking the condition bit masks every single time
Signed-off-by: Erik Boasson <eb@ilities.com>
The specification says the data in an invalid sample must not be inspected, but that means retrieving the key values from a disposed/unregistered instance after a take is impossible. So, the elegant thing to do is to always provide the key values in the data. The remaining question is then whether the non-key fields should be left in whatever state they happened to be in, or be set to zero. The latter is more consistent in the interface, and that is almost invariably a good thing.
Signed-off-by: Erik Boasson <eb@ilities.com>
Whenever a sample or an instance is added, check it against the attached query conditions and indicate which ones match
Signed-off-by: Erik Boasson <eb@ilities.com>
This index can then be used as an index into a bitmap to keep track
which query conditions are matched by a sample or an instance.
Signed-off-by: Erik Boasson <eb@ilities.com>
The read condition and the query condition are represented by the same data type internally, and a read condition therefore has a "m_filter" attribute. It makes more sense to initialise this properly as part of the read condition, instead of initialisation-by-memset in the dds_create_readcond, then overwriting it in dds_create_querycond.
Signed-off-by: Erik Boasson <eb@ilities.com>
The RHC tracing produces so much junk that is hardly ever useful that a
normal trace should definitely not include it.
Signed-off-by: Erik Boasson <eb@ilities.com>
A read restricted to samples in "read" state would not enter the condition update code on the false assumption that no read conditions could become triggered if the number of read samples remained the same, but it is nonetheless possible that the instance was transitions from "new" to "old" as a consequence, at least in my interpretation of the spec and the current implementation of read() in Cyclone. This commit brings consistency to the implementation without the intention of confirming the current behaviour as being desirable.
Signed-off-by: Erik Boasson <eb@ilities.com>
Fixing that produces a lot of noise on stderr because of inappropriate use of the "info" category in various place and, on macOS, because of a rather stupid way of messing with thread scheduling priorities even when none have been specified explicitly in the configuration.
Signed-off-by: Erik Boasson <eb@ilities.com>
Getting a QoS from an entity is akin to reading, and all read/take operations reuse or free/reallocate memory to avoid memory leaks, and so it is a reasonable assumption that calling dds_get_qos repeatedly without intervening calls to dds_reset_qos would not leak any memory either. (This was actually an assumption in the builtin topics test.) Therefore, it is reasonable to first call dds_reset_qos in dds_get_qos. All operations in the API that yield or modify a QoS object result in a properly initialised one, therefore the input to dds_get_qos is necessarily initialised, and so this is safe.
Signed-off-by: Erik Boasson <eb@ilities.com>
Listener/status management invocation was rather expensive, and
especially the cost of checking listeners, then setting status flags and
triggering waitsets ran into severe lock contention.
A major cost was the repeated use of dds_entity_lock and
dds_entity_unlock, these have been eliminated. Another cost was that
each time an event occurred (with DATA_AVAILABLE the most problematic
one) it would walk the chain of ancestors to see if any had a relevant
listener, and only if none of them had any, it would set the status
flags.
The locking/unlocking of the entity has been eliminated by moving the
listener/status flag manipulation from the general entity lock to its
m_observers_lock. That lock has a much smaller scope, and consequently
contention has been significantly reduced.
Instead of walking the entity hierarchy looking for listeners, an entity
now inherits the ancestors' listeners. The set_listener operation has
been made a little more complicated by the need to not only set the
listeners for the specified entity, but to also update any inherited
listeners its descendants.
The commit is a bit larger than strictly needed ... I've started
reformatting the code to reduce the variety of styles ... as there I
haven't been able to find a single tool that does what I want, it may
well end up as manual work.
Signed-off-by: Erik Boasson <eb@ilities.com>
dds_init generates an entity name for the participant derived from the
process name and process id, but the name is currently uninitialised
(temporarily so, caused by a recent update to the OS abstraction layer).
Signed-off-by: Erik Boasson <eb@ilities.com>
Move details of built-in topics out of the DDSI core (so the only hooks
remain). For this, rtps_term had to be split, so now it is "stop"
followed by "fini".
Add a notion of local writers that are not bound to a participant ("local
orphans"), so that the local built-in topic writers can be created during
initialization. This eliminates the "builtin" participant. This
uncovered in inconsistency in the unit tests: on the one hand, a newly
created participant is expected to have no child entities; on the other
hand, the built-in topics were expected to be returned by find_topic ...
This inconsistency has been resolved by creating them lazily and
accepting that find_topic can't return them until they have been
created. Special code was in place in dds_create_reader anyway, so it
is not expected to have any real consequence for applications.
Use a special WHC implementation that regenerates the data on the fly
using the internal discovery tables of DDSI, so that the samples are only
stored by readers. This eliminates the memory overhead of that existed
previously when the WHC of the writers stored the data.
No longer return topic name and type name in the built-in topics, they
have been extracted already and are not accessible through the normal
interface but do cause problems when comparing QoS.
Signed-off-by: Erik Boasson <eb@ilities.com>
simply switching from dds_alloc to os_malloc in alloc_sample removes a redundant memset, which gives 5% improvement in a throughput test (on my laptop); other analogous changes for consistency
Signed-off-by: Erik Boasson <eb@ilities.com>
