niklas/cyclonedds
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Adapt message and burst sizes to receive buffers

Browse source 
This changes a few intertwined things at the same time:

* It allows configuring sending a partial message for large messages,
with a maximum derived from the discovered receive buffer sizes;

* It uses a different message size limit for datagrams that include
  retransmits than for those that don't.  The argument here is that,
  having seen flaky networks where large datagrams cause trouble, it
  makes sense to default to sending retransmits as datagrams that fit in
  individual packets.

* The best performance is generally obtained using the maximum data gram
  size, but the benefits do fall off quite quickly once they are
  largish.  For flaky networks, it doesn't make sense to go for 64kB
  datagrams.  This tries to find a reasonable compromise.

* It now packs mutiple fragments into a single DATAFRAG message to
  eliminate the cost of using small fragment sizes.

The changes in buffer sizes cause the ddsperf sanity check to fail:

* The larger amounts of unacknowledged data cause the used memory to be
  higher, failing the RSS check.  Raising the limit seems
  reasonable (the alternative would be to configure it back to the old
  values, but it is all empirically determined anyway).

* The same also causes the publisher thread to get to run more and the
  ping/pong bit gets less of a chance.  Using fixed-frequency bursts
  helps with this.

This therefore also adjust the test configuration and the thresholds a
bit.

Signed-off-by: Erik Boasson <eb@ilities.com>
This commit is contained in:
Erik Boasson 2020-06-22 17:15:48 +02:00 • committed by eboasson
parent 82e6a7972c
commit 66daba9f2f
13 changed files with 321 additions and 106 deletions
Download patch file Download diff file Expand all files Collapse all files

43
etc/cyclonedds.xsd
View file

@ -313,6 +313,7 @@ CycloneDDS configuration</xs:documentation>
<xs:element minOccurs="0" ref="config:ExternalNetworkMask"/>
<xs:element minOccurs="0" ref="config:FragmentSize"/>
<xs:element minOccurs="0" ref="config:MaxMessageSize"/>
<xs:element minOccurs="0" ref="config:MaxRexmitMessageSize"/>
<xs:element minOccurs="0" ref="config:MulticastRecvNetworkInterfaceAddresses"/>
<xs:element minOccurs="0" ref="config:MulticastTimeToLive"/>
<xs:element minOccurs="0" ref="config:NetworkInterfaceAddress"/>
@ -382,9 +383,18 @@ CycloneDDS configuration</xs:documentation>
<xs:annotation>
<xs:documentation>
&lt;p&gt;This element specifies the maximum size of the UDP payload that Cyclone DDS will generate. Cyclone DDS will try to maintain this limit within the bounds of the DDSI specification, which means that in some cases (especially for very low values of MaxMessageSize) larger payloads may sporadically be observed (currently up to 1192 B).&lt;/p&gt;
&lt;p&gt;On some networks it may be necessary to set this item to keep the packetsize below the MTU to prevent IP fragmentation. In those cases, it is generally advisable to also consider reducing Internal/FragmentSize.&lt;/p&gt;
&lt;p&gt;On some networks it may be necessary to set this item to keep the packetsize below the MTU to prevent IP fragmentation.&lt;/p&gt;
&lt;p&gt;The unit must be specified explicitly. Recognised units: B (bytes), kB &amp; KiB (2&lt;sup&gt;10&lt;/sup&gt; bytes), MB &amp; MiB (2&lt;sup&gt;20&lt;/sup&gt; bytes), GB &amp; GiB (2&lt;sup&gt;30&lt;/sup&gt; bytes).&lt;/p&gt;
&lt;p&gt;The default value is: "4096 B".&lt;/p&gt;</xs:documentation>
&lt;p&gt;The default value is: "14720 B".&lt;/p&gt;</xs:documentation>
</xs:annotation>
</xs:element>
<xs:element name="MaxRexmitMessageSize" type="config:memsize">
<xs:annotation>
<xs:documentation>
&lt;p&gt;This element specifies the maximum size of the UDP payload that Cyclone DDS will generate for a retransmit. Cyclone DDS will try to maintain this limit within the bounds of the DDSI specification, which means that in some cases (especially for very low values) larger payloads may sporadically be observed (currently up to 1192 B).&lt;/p&gt;
&lt;p&gt;On some networks it may be necessary to set this item to keep the packetsize below the MTU to prevent IP fragmentation.&lt;/p&gt;
&lt;p&gt;The unit must be specified explicitly. Recognised units: B (bytes), kB &amp; KiB (2&lt;sup&gt;10&lt;/sup&gt; bytes), MB &amp; MiB (2&lt;sup&gt;20&lt;/sup&gt; bytes), GB &amp; GiB (2&lt;sup&gt;30&lt;/sup&gt; bytes).&lt;/p&gt;
&lt;p&gt;The default value is: "1456 B".&lt;/p&gt;</xs:documentation>
</xs:annotation>
</xs:element>
<xs:element name="MulticastRecvNetworkInterfaceAddresses" type="xs:string">
@ -465,6 +475,7 @@ CycloneDDS configuration</xs:documentation>
<xs:element minOccurs="0" ref="config:AssumeMulticastCapable"/>
<xs:element minOccurs="0" ref="config:AutoReschedNackDelay"/>
<xs:element minOccurs="0" ref="config:BuiltinEndpointSet"/>
<xs:element minOccurs="0" ref="config:BurstSize"/>
<xs:element minOccurs="0" ref="config:ControlTopic"/>
<xs:element minOccurs="0" ref="config:DDSI2DirectMaxThreads"/>
<xs:element minOccurs="0" ref="config:DefragReliableMaxSamples"/>
@ -549,6 +560,34 @@ CycloneDDS configuration</xs:documentation>
</xs:restriction>
</xs:simpleType>
</xs:element>
<xs:element name="BurstSize">
<xs:annotation>
<xs:documentation>
&lt;p&gt;Setting for controlling the size of transmit bursts.&lt;/p&gt;</xs:documentation>
</xs:annotation>
<xs:complexType>
<xs:all>
<xs:element minOccurs="0" ref="config:MaxInitTransmit"/>
<xs:element minOccurs="0" ref="config:MaxRexmit"/>
</xs:all>
</xs:complexType>
</xs:element>
<xs:element name="MaxInitTransmit" type="config:memsize">
<xs:annotation>
<xs:documentation>
&lt;p&gt;This element specifies how much more than the (presumed or discovered) receive buffer size may be sent when transmitting a sample for the first time, expressed as a percentage; the remainder will then be handled via retransmits. Usually the receivers can keep up with transmitter, at least on average, and so generally it is better to hope for the best and recover. Besides, the retransmits will be unicast, and so any multicast advantage will be lost as well.&lt;/p&gt;
&lt;p&gt;The unit must be specified explicitly. Recognised units: B (bytes), kB &amp; KiB (2&lt;sup&gt;10&lt;/sup&gt; bytes), MB &amp; MiB (2&lt;sup&gt;20&lt;/sup&gt; bytes), GB &amp; GiB (2&lt;sup&gt;30&lt;/sup&gt; bytes).&lt;/p&gt;
&lt;p&gt;The default value is: "4294967295".&lt;/p&gt;</xs:documentation>
</xs:annotation>
</xs:element>
<xs:element name="MaxRexmit" type="config:memsize">
<xs:annotation>
<xs:documentation>
&lt;p&gt;This element specifies the amount of data to be retransmitted in response to one NACK.&lt;/p&gt;
&lt;p&gt;The unit must be specified explicitly. Recognised units: B (bytes), kB &amp; KiB (2&lt;sup&gt;10&lt;/sup&gt; bytes), MB &amp; MiB (2&lt;sup&gt;20&lt;/sup&gt; bytes), GB &amp; GiB (2&lt;sup&gt;30&lt;/sup&gt; bytes).&lt;/p&gt;
&lt;p&gt;The default value is: "1 MiB".&lt;/p&gt;</xs:documentation>
</xs:annotation>
</xs:element>
<xs:element name="ControlTopic">
<xs:annotation>
<xs:documentation>