Some of the liveliness qos tests were not using unique topic names
for the tests for local and remote readers. Re-using the participant
for these 2 tests results in unexpected reader-proxywriter matches
in the latter test. Fixed by adding a sequence number in the topic name.
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
This commit adds support for liveliness QoS when using local readers.
The implementation for (liveliness) expiration of writers used here is
similar to that used with proxy writers, and it also supports the three
liveliness kinds (1) automatic, which is trivial when using a local
reader and writer, (2) manual-by-participant and (3) manual-by-topic.
In addition, these changes and fixes are included in this commit:
- Fixed a bug in heartbeat handling in the reader: for manual-by-
participant writers the lease was not updated on reception of a
heartbeat message with liveliness flag set. This is fixed and a
test-case is added.
- Include the liveliness flag in a heartbeat message to the trace
- Trace all lease renewals, including liveliness leases
- Replaced liveliness changed state 'twitch' by 2 subsequent calls
to the status callback
- Added a test for liveliness duration 0 and 1ns (for both local
and remote readers)
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
And updated dds_read_instance_mask and dds_take_instance_mask to work with DDS_READ_WITHOUT_LOCK
Signed-off-by: Thijs Sassen <thijs.sassen@adlinktech.com>
This commit contains the implementation of the deadline QoS
for readers and writers. The description of this QoS in
the DDS specification (section 2.2.3.7):
"This policy is useful for cases where a Topic is expected to
have each instance updated periodically. On the publishing side this
setting establishes a contract that the application must meet.
On the subscribing side the setting establishes a minimum
requirement for the remote publishers that are expected to supply
the data values."
On the writer side, the deadline missed event also needs to trigger in
case only local readers exist. The implementation for this inserts
the sample in the writer history cache temporary, so that an instance
is created in the whc. Immediately after inserting the sample, it is
removed again. With the creation of the instance, the deadline missed event
is created, which will take care of triggering the deadline missed
callback if required. In case the instance already existed, the timer
of the event is renewed.
To verify the changes to the writer history cache, add an additional
test to check the write history cache state. This test checks the state
of the whc after writing samples by a writer with specific combinations
of qos settings. The state of the whc is checked for stored
samples (min/max sequence number) and the absence of unacked data, after
writing samples and wait for acks by the local and/or remote
readers (which is also a parameter for this test). This test is
introduced as part of the deadline implementation, but its scope is
wider than only the changes that were made in the whc implementation for
the deadline qos.
This test showed that even before the deadline support was added,
whc_default_remove_acked_messages_full data was not marked as acked in
case of transient-local keep-all. This resulted in data in whc that
never gets in acked state. This has been fixed as well.
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
These topics are non-standard and not actually used anywhere in Cyclone,
so it is rather silly to keep them.
Signed-off-by: Erik Boasson <eb@ilities.com>
This commit enables specifying a duration for data to be valid when writing
samples. After this duration, samples are dropped from the reader and writer
history cache. See section 2.2.3.16 of the DDS specification for more
details on this QoS.
The expiration of samples in the reader history cache is calculated
based on the reception timestamp of the sample and uses the monotonic
clock. As a result, the current implementation does not rely on clock
synchronisation between reader and writer. There may be reasons to
change this behavior in future and use the source timestamp instead.
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
Maintain an index of all entities on (kind, topic, GUID) in addition to
the concurrent hash table on GUID and use that when looking for matching
entities. For endpoint matching, restrict the scan to readers/writers
of the right topic, for regular iterators, restrict it to the range of
the correct entity kind.
Signed-off-by: Erik Boasson <eb@ilities.com>
* Fix array keys on little-endian machines
Signed-off-by: Erik Boasson <eb@ilities.com>
* Test for writing topics with an array as key
Signed-off-by: Erik Boasson <eb@ilities.com>
- moved de-registration of a lease to a separate function, so that
it is called synchronously when a pwr is deleted, to prevent the
lease from expiring (before this change, the lease was renewed with
t_never, but that does not work because of a check with early out
in lease_renew)
- handle proxypp->owns_lease correctly: when an OpenSplice instance
was running in the same network and participants from OpenSplice were
discovered, the lease-renewal fails in case the proxy participant's
lease was not registered in minl_auto, which happens when the
proxypp depends on its parent (ddsi2) participant.
- increased lease duration in create_delete_writer stress test
to avoid failed tests due to delayed pmd messages
- fixed the indenting in liveliness tests source file
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
This commit adds the implementation for the liveliness kind manual-by-topic.
With these changes, the api function dds_assert_liveliness now accepts a
writer as entity. Asserting liveliness on a writer triggers sending a
heartbeat message with the liveliness flag set.
The code for handling reception of a heartbeat message checks for this flag and
if set the lease for the proxy writer is renewed (and the shortest manual-by-participant
lease on the proxy participant as well, because the message also indicates that the
remote participant is alive). Receiving data (handle_regular) also renews the
lease on the proxy writer in case it has the manual-by-topic liveliness kind.
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
Refactored locking for pwr->alive so that locking order is consistent (locking
pwr->c.proxypp->e.lock while holding pwr->e.lock is the expected order). And
processed other review comments: removed lock for ephash_lookup, added
additional comments, set pwr->lease to null if not initialised.
Because of intermittent timing issues with liveliness expiry test in Travis, I've
increase the time-out and retry limit for this test.
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
Check that proxy writer is still alive (could be not-alive due to deleting) in code path for proxy writer's lease expiry
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
Some additional refactoring in locking for pwr->alive for liveliness qos, moved lease free to gc_delete_pwr, refactored the set pwr alive/not alive functions and some minor changes in liveliness tests
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
Fixed building liveliness tests on Windows and some cleaning up in liveliness test code
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
Added unit tests for (1) testing the scenario that a proxy writer writes data
after its lease is expired, to check that the status for the pwr is set to alive
again and (2) stress-testing the creation and deletetion of writers with
decreasing lease duration. In addition I've optimized the locking in
unref_proxy_participant a bit and fixed the liveliness changed callback
when a writer with expired lease (not-alive) gets alive again.
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
This commit adds support for the liveliness QoS for the liveliness
kinds automatic and manual-by-participant. It also implements the
lease_duration from this QoS, which was ignored until now. In the
api the function dds_assert_liveliness is added to assert liveliness
on a participant, which can be used when using liveliness kind
manual-by-participant.
Liveliness kind manual-by-topic is not yet supported, this will be
added in a later commit.
* Proxy participants now have 2 fibheaps to keep leases: one for leases
of pwrs with automatic liveliness and one for leases of the pwrs with
manual-by-participant liveliness (both protected by the proxypp lock).
The minl_auto and minl_man members represent the shortest lease from
these fibheaps and these leases are renewed when receiving data.
Replacing the minl_ leases is now done by replacing the lease object
(atomic ptr) with delayed deletion of the old lease using the gc.
* Proxy writers are set not-alive when the lease expired, and reset to
alive then data is received. When data is received by a pwr, the other
pwrs in the proxypp might also be set alive. I think the specification
is not clear at this point, and for now I have not implemented this
* I refactored out the counter for man-by-pp proxy writers and improved
locking when updating the min-leases on the proxy participant, so I
think this fixes the race conditions.
Some additional tests are required, e.g. to test the not-alive->alive
transition for pwrs. I will add these in short term, as well as the
implementation of the manual-by-topic liveliness kind.
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
Signed-off-by: Erik Boasson <eb@ilities.com>
Use static assert instead of run-time assert
Signed-off-by: Erik Boasson <eb@ilities.com>
Use static assertion to verify nn_rmsg offset calc
Signed-off-by: Erik Boasson <eb@ilities.com>
* Moved bswap functions to ddsrt
Moved the byte swapping functions from ddsi to ddsrt so that
these can be re-used in e.g. the security plugins and tests.
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
* Moved decarations for bswap functions to ddsrt
Signed-off-by: Dennis Potman <dennis.potman@adlinktech.com>
CMAKE_PROJECT_NAME refers to the top-level project name, not the most recent project. So any CMake project that pulls this in as a dependency was in for a nasty surprise.
https://cmake.org/cmake/help/latest/variable/CMAKE_PROJECT_NAME.html
Signed-off-by: Dan Rose <dan@digilabs.io>
* Fix type of num reliable readers (int to int32_t)
* Conversion codes in debug monitor printf formats
* Dead code elimination
* Skipping a test case where SIZE_MAX is assumed > INT32_MAX if
assumption is false on target platform
* Error handling in os_sockWaitsetNew
* Stick to unsigned in fragment size calculations
This check is actually guarded by valid_DataFrag and was safe for
datagrams up to 2GB, but the unintended and implicit conversion to is
still best eliminated.
* A "server" connection never has an invalid socket in TCP wrapper
* Handle error return from gethostname in SPDP write (CID 248183)
* Handle extended retcodes in dds_strretcode
CID 248131, introduced by 19aec98b8a
* Remove dead code in ddsrt logging test (CID 248195)
* Validate command-line argument in process test (CID 248117)
* Allow for extremely delayed store in test
Test is constructed to have the events trigger only at the appropriate
times, but it does assume that the store to cb_called becomes visible
prior to the listener callback. I'm pretty sure that will always be
the case in practice, but I'm also pretty sure there is no formal
guarantee without a memory barrier, which mutex_unlock provides.
CID 248088, 248136, 248177, 253590, 253591, 253593
* Check unsetenv return value in test (CID 248099)
Signed-off-by: Erik Boasson <eb@ilities.com>
In the case of when a DATA_ON_READERS listener is set, but with the
corresponding status mask is set to suppress the event, the reader lock
would not be in locked, resulting in a unlocked access of status flags
and a double unlock.
Signed-off-by: Erik Boasson <eb@ilities.com>
For targets that do not support ddsrt_setenv and ddsrt_getenv, an alternative
method is needed to supply an application specific configuration. One way to
implement this, is to add a function for creating a domain with a string
arguments, which needs to be called before any call to dds_create_participant
for given domain identifier.
The function dds_create_domain has been added, which has as arguments a domain
identifier and a configuration string. The string is treated in the same way
as the string that is retrieved from the environment variable, in that it may
containt a comma separated list of file names and/or XML fragments for the
configuration.
Two tests have been added. One limits the number of participants to two and
verifies that creating a third participant fails. The other tests checks
incorrect calls to dds_create_domain.
An assert in dds_handle_delete has been weakened.
Signed-off-by: Frans Faase <frans.faase@adlinktech.com>
Rare intermittent failure appears to be timing. Increasing the timeout
doesn't affect the duration of a successful run and will still signal an
missing trigger.
Signed-off-by: Erik Boasson <eb@ilities.com>
So dds_read/dds_take should pass it the address of the first pointer,
rather than the first pointer itself, or the freeing of memory allocated
for samples because of an outstanding loan will crash. Add a test that
reliable detects this case when no other participants are around.
Signed-off-by: Erik Boasson <eb@ilities.com>
Deleting entities concurrently with operating on them or creating child
entities should work properly, even if it is essentially abuse. This
commit fixes (most?) of them, with exception of some nastiness when
deleting the last entity, and thus deinitialising the library, in
parallel to attempting to operate on a (by definition invalid) handle.
* Interrupting a blocked operation at the beginning of "delete" is now a
separate operation. E.g., a wait call on a waitset must be interrupted,
but the data structures can't be touched yet because other threads may
be doing an attach/detach in parallel.
* DDSI writer can now be switched to an intermediate state,
"INTERRUPTED", to indicate that it should unblock any waiting threads
and refuse to transmit any further data, but without actually
embarking on the path of deleting data structures.
* The extra "pinning" of readers and writers is now gone, they remain
fully functional until the no other threads can still access the
entity.
* Future listener invocations are prevented as part of deleting the
entity, but now it also guarantees the application can no longer
re-enable them. It furthermore waits until there are no further
current or pending listener invocations, rather than simply no current
ones.
* The internal state of the waitset now has its own lock, otherwise
attaching the parent entity of the waitset can require locking the
waitset after having locking the parent, which violates locking order.
* Handles are created in a pending state, where they are not included in
a dds_get_children operation and refuse to be pinned. This makes it
possible (in a future commit) to undo deletion of complex entities.
* There is a test (ddsc_waitset_torture) that exercises some of these
corner cases.
Signed-off-by: Erik Boasson <eb@ilities.com>
This fixes a possible deadlock when detaching an entity at the same time
it is triggering: a triggering entity holds its m_observers_lock while
trying to acquire waiset::m_mutex, and so attach and detach must not do
the opposite. The deadlock had excellent reproducibility in a seemingly
unrelated ROS2 application; this changes fixes it.
Signed-off-by: Erik Boasson <eb@ilities.com>
The default RHC implementation is not always ideal and rather than
trying to squeeze everything in a fixed interface it makes more sense to
allow the caller to provide an arbitrary implementation of the
interface.
This is not yet a stable interface.
Signed-off-by: Erik Boasson <eb@ilities.com>
This makes it possible to write one's own RHC implementation. This is
not a stable interface. It shuffles a few things around and renames
some types used throughout the code to stick to having a "dds" prefix
for all the external things.
Signed-off-by: Erik Boasson <eb@ilities.com>
All functions creating an entity pin the parent, which guarantees that
the CLOSED is not set yet, but as setting it occurs within the mutex, it
may be set by the time the parent has been locked. One should not try
to add a child entity at that point, so check it again.
Signed-off-by: Erik Boasson <eb@ilities.com>
Require that all entities attached to waitset are in the same
participant (if the waitset's parent is a participant) or in the same
domain (if the waitset's parent is a domain). There used to be no check
on this, but it always seemed unhygenic and now that domains and the
entire library instantiation can be used as parents, it seems much
better to enforce this scoping rule.
Signed-off-by: Erik Boasson <eb@ilities.com>
