Method for transparent zero-copy distribution of data to DDS applications

Abstract

A method is provided for performing zero-copy distribution of data samples between applications running on the same node in a system using an Object Management Group (OMG) Data Distribution Service (DDS) and/or a Real-Time Publish Subscribe (RTPS) protocol. Further provided is a method for selecting the network representation to communicate with a DataReader in a system using an Object Management Group (OMG) Real-Time Publish Subscribe (RTPS) protocol. Still further provided is the combination of these two methods to communicate transparently using zero-copy within the same node and not using zero-copy for different nodes. Embodiments of this invention lead to a relatively small communication latency that is constant and independent of the data size for applications running within a single node.

Claims

1. A method for performing zero-copy distribution of data samples between applications running on the same node in a system using an Object Management Group (OMG) Data Distribution Service (DDS), a Real-Time Publish Subscribe (RTPS) protocol or a combination thereof, comprising: (a) having a node; (b) having a pool of data samples in a shared memory segment; (c) having a first application publishing the data samples from the pool with a DataWriter; (d) having a second application receiving the data samples from the pool with a DataReader; (e) having a memory representation of the data samples; (f) having a network representation of the data samples that is a reference; (g) the DataWriter sending references of the data samples to the DataReader; and (h) the DataReader using the references to access the memory representation of the data samples wherein there is zero-copying between (i) the first application publishing the samples and (ii) the second application receiving the samples.

2. The method as set forth in claim 1, wherein the memory representation is memory position independent.

3. The method as set forth in claim 1, wherein the memory representation is an Extended CDR encoding version 1 (XCDR1) or an Extended CDR encoding version 2 (XCDR2).

4. The method as set forth in claim 1, wherein the memory representation is a C/C++ plain memory representation for fixed-size DDS types.

5. The method as set forth in claim 1, wherein the memory representation uses offsets instead of memory addresses.

6. The method as set forth in claim 1, wherein the DataWriter sends to the DataReader an RTPS DATA message containing a shared memory ID, a shared memory offset, and an epoch value.

7. The method as set forth in claim 6, wherein the shared memory ID and the shared memory offset identify a data sample in a shared memory segment.

8. The method as set forth in claim 6, wherein the shared memory ID and the shared memory offset are included as part of the reference.

9. The method as set forth in claim 6, wherein the epoch value identifies the content of the data sample.

10. The method as set forth in claim 6, wherein the epoch value is included as part of an inline QoS of the RTPS DATA.

11. The method as set forth in claim 1, wherein the shared memory pool is created by the DataWriter.

12. The method as set forth in claim 1, wherein the first application borrows data samples from the pool and writes the borrowed data samples with the DataWriter.

13. The method as set forth in claim 1, wherein the data sample from the pool has a header including a state and an epoch value.

14. The method as set forth in claim 13, wherein the state indicates whether the sample can be borrowed by the first application.

15. The method as set forth in claim 13, wherein the epoch value increases every time the first application borrows the sample from the pool.

16. The method as set forth in claim 1, wherein the DataWriter sets the state of a data sample header to indicate that the data sample cannot be borrowed by the first application prior to sending the data sample to the DataReader.

17. The method as set forth in claim 1, wherein the DataWriter sets the state of a data sample header to indicate that the data sample can be borrowed by the first application after the DataReader has acknowledged the data sample.

18. The method as set forth in claim 1, wherein the DataReader uses a shared memory ID and a shared memory offset in the RTPS DATA to access the data sample in a shared memory segment.

19. The method as set forth in claim 1, wherein the DataReader compares an epoch value in the RTPS DATA with an epoch in a data sample header to check that the content of a data sample has not been modified.

20. A method for selecting the network representation to communicate with a DataReader in a system using an Object Management Group (OMG) Real-Time Publish Subscribe (RTPS) protocol, comprising: (a) having a DataWriter running on a first node; (b) having a DataReader running on a second node; (c) having a locator in which the DataReader is reachable; (d) having multiple network representations; (e) having each network representation identified by an encapsulation identifier; (f) the DataReader assigning a list of encapsulation identifiers to the locator ordered by priority, wherein the shared memory reference network representation has the highest priority; and (g) the DataWriter selecting the highest priority network representations in the list that is compatible between the DataWriter and DataReader, wherein the network representation is a shared memory reference for zero-copy distribution from the DataWriter to the DataReader in case the second node and the first node are the same nodes.

21. The method as set forth in claim 20, wherein the network representations are an Extended CDR encoding version 1 (XCDR1), an Extended CDR encoding version 2 (XCDR2), or a shared memory reference.

22. The method as set forth in claim 20, wherein the DataReader creates a shared memory segment associated with a shared memory locator.

23. The method as set forth in claim 20, wherein a shared memory locator contains a shared memory ID that can be used to attach to a shared memory segment created by the DataReader and a node identifier that can be used to identify the second node.

24. The method as set forth in claim 20, wherein the DataReader writes a node identifier in the shared memory segment.

25. The method as set forth in claim 20, wherein the DataWriter uses the shared memory ID in a locator to attach to a shared memory segment created by the DataReader.

26. The method as set forth in claim 20, wherein the DataWriter compares a node identifier in a shared memory locator with a node identifier in a shared memory segment to determine whether the first node and the second node are the same.

27. A method for performing transparent zero-copy distribution for a DataWriter and a DataReader running on the same node and a non zero-copy distribution for DataWriter and a DataReader running on different nodes, comprising the combination of the method of claim 1 or the method of claim 20.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) FIG. 1 shows according to an exemplary embodiment of the invention a number of copies in normal DDS system.

(2) FIG. 2 shows according to an exemplary embodiment of the invention a standard DDS Data Sample Transmission within the Same Node.

(3) FIG. 3 shows according to an exemplary embodiment of the invention a zero-copy DDS Data Sample Transmission within the Same Node.

(4) FIG. 4 shows according to an exemplary embodiment of the invention Transparent DDS Data Sample Transmission.

(5) FIG. 5 shows according to an exemplary embodiment of the invention a DATA RTPS Submessage.

(6) FIG. 6 shows according to an exemplary embodiment of the invention a Mapping of a Shared Memory Segment.

(7) FIG. 7 shows according to an exemplary embodiment of the invention a Shared Memory Reference.

(8) FIG. 8 shows according to an exemplary embodiment of the invention a Shared Memory Locator.

(9) FIG. 9 shows according to an exemplary embodiment of the invention a zero-copy Distribution Performance.

DETAILED DESCRIPTION

(10) Transparent Zero-Copy Delivery of Data to DDS Applications

(11) To address the CPU and memory issues resulting from doing multiple copies of a data sample, this invention provides a method and system for transparent zero-copy distribution of data samples between co-located applications (applications running on the same node).

(12) Zero-copy means that data samples are not copied by the middleware. Transparent means that an application does not need to know which applications are co-located to take advantage of the optimized transfer, since the middleware transparently uses the zero-copy distribution if possible and otherwise falls back to the standard distribution methods.

(13) DDS applications use the Real-Time Publish-Subscribe (RTPS) protocol to exchange data. The RTPS specification describes how data samples are transmitted across the network. A DataWriter marshals the data to a network representation that is then encapsulated in an RTPS message with the structure as shown in FIG. 5.

(14) An RTPS message always starts with a header followed by one or more submessage. The DATA submessage (FIG. 2) is used to transfer data samples. It should be noted that RTPS includes two types of data submessages: DATA and DATA_FRAG. This document will always use the DATA submessage for simplicity. All concepts also apply to DATA_FRAG. The DATA sub-message has the following layout. 1. DATA submessage header: The submessage header includes information such as the data sample sequence number and other metadata known as inline Qos. 2. EncapsulationId: The EncapsulationId indicates the network representation of the data sample(s) that follow. The EncapsulationId is used by the DataReaders to convert the serialized data samples back to the memory representation (e.g. C++ structure). 3. Serialized data: The serialized data consists of the data samples in a serialized format that matches the EncapsulationId.

(15) A key concept of this invention is the concept of a shared memory reference as the network representation of a data sample.

(16) The standard DDS EncapsulationIds require the serialized data samples to be part of the DATA submessage. This invention introduces a new type of EncapsulationId, a data reference encapsulation. The data that follows, a reference encapsulation, is not the serialized data sample itself, but instead a description of how to retrieve the data sample out-of-band.

(17) A particular type of reference encapsulation is the shared memory reference encapsulation distributed to DDS applications within the same nodes to achieve zero-copy distribution of data. This reference is used by DataReaders to locate data samples in a shared memory segment. For more information on the new shared memory reference encapsulation, see infra The Shared Memory Reference Network Representation.

(18) How to Select Zero-Copy Distribution for a Type

(19) This invention introduces a new annotation, transfer_mode, as a way to indicate that samples from the annotated type can benefit from zero-copy delivery for applications running on the same node. The following values are supported: INBAND: This transfer method is the standard inband transfer where data samples are serialized. SHMEM_REF: This transfer method indicates that the middleware shall, if possible, transfer data samples by sending a reference to an address in a shared memory segment (shared memory reference).

(20) For example, to enable the zero-copy transfer mode over shared memory for the CameraImage, use the following IDL:

(21) TABLE-US-00003 @transfer_mode(SHMEM_REF) struct CameraImage { long camera_id; //@key octet image_data[IMAGE_SIZE]; };

(22) Other transfer mode values are also possible. This would allow using additional out-of-band mechanisms to transfer the content of a data sample. For example: A FTP (File Transfer Protocol) transfer mode where a reference encapsulates a FTP url to a file containing the data sample. A DMA (Direct Memory Access) transfer mode where a reference encapsulates the DMA address of the data sample.
Data Samples Memory Management

(23) Standard DDS applications typically allocate data samples on the heap, populate them, and then pass them to a DataWriter for distribution by invoking the DataWriter::write( ) operation. Because a DataWriter makes a copy of a data sample when the sample is serialized, the application is free to use the sample memory again after writing the sample, without interfering with the distribution process.

(24) Consider the following IDL type:

(25) TABLE-US-00004 struct CameraImage { @key long camera_id; octet image_data[IMAGE_SIZE]; };

(26) A standard DDS application has methods to allocate a data sample before it is populated and sent. The following sequence is common:

(27) TABLE-US-00005 CameraImage *image = NULL; /* Allocate memory from heap */ image = CameraImage::create_data( ) /* Fill in data */ image−>image_data = ... /* The middleware distributes the data */ ImageDataWriter−>write(image);

(28) As shown in FIG. 1, because the middleware makes a copy of the data sample, the image can be reused after calling DataWriter::write( ) as illustrated below.

(29) TABLE-US-00006 CameraImage *image = NULL; /* Allocate memory from heap */ image = CameraImage::create_data( ) While (1) { /* Fill in data */ image−>image_data = ... /* The middleware distributes the data */ imageDataWriter−>write(image); ... }

(30) When a sample is allocated from the heap, the application owns the sample memory and the middleware owns the network representation.

(31) Allocating Data Samples from Shared Memory

(32) To achieve zero-copy data distribution, the application can no longer allocate data samples from the heap. Instead, the data samples are stored in one or more shared memory segments owned by the DataWriter.

(33) When a type is marked with @transfer_mode(SHMEM_REF), a DataWriter for that type creates a one or more shared memory segments that will be used to contain the Camera Image data samples. When data samples are allocated in shared memory, different applications can map the shared memory segments into their local address space and directly access the data samples as shown in FIG. 6.

(34) FIG. 6 shows how a DataWriter has allocated a set of samples in a shared memory segment with id=10 and how a DataReader has mapped the segment into its local memory space. Sample1_ptr in DataWriter A's address space points to the same physical memory as Sample1_ptr in DataReader B's address space.

(35) A DataWriter provides to the application a way to loan data samples from the shared memory segment. After a data sample is loaned, the application can populate it and write it. This invention introduces two new APIs for this purpose, using the following IDL type as an example:

(36) TABLE-US-00007 @transfer_mode(SHMEM_REF) struct CameraImage { long camera_id; //@key octet image_data[IMAGE_SIZE]; };

(37) The following two APIs are available:

(38) CameraImageDataWriter::get_loan( )

(39) This API loans (borrows) a data sample to the application. While an application has a loan on the data sample, a DataWriter cannot use the data sample or return it in a different call to get_loan. The application maintains the loan of the data sample until it either calls CameraImageDataWriter::write( ) or explicitly returns the loan by calling the new API CameralmageDataWriter::discard_loan( ).

(40) Once a sample is written, the DataWriter takes responsibility for returning the sample back to the shared memory segment after the sample is delivered and acknowledged by the matching DataReaders.

(41) CameraImageDataWriter::discard_loan( )

(42) This API discards a loan on a DDS data sample previously loaned with a call to CameraImageDataWriter::get_loan( ). Discarding a loan means the DDS application indicates it no longer needs access to the data sample.

(43) Format of a Data Sample

(44) Each one of the data samples in shared memory is preceded by a header that contains metadata about the sample. The header contains the following information: State: Indicates if a sample can be loaned to the application. Shared Memory Id and Sample Offset: Identify the sample in the shared memory reference sent as part of the RTPS DATA message. See section The Shared Memory Reference Network Representation. Epoch: This field is used to check data consistency. See section Detecting an Inconsistent Data Sample in the Receiving Application.
The Shared Memory Reference Network Representation

(45) With zero-copy delivery, a DataWriter does not send the serialized sample to a DataReader, but a shared memory reference to the data sample in memory. This invention introduces a new EncapsulationId sent as part of the RTPS DATA sub-message that a DataWriter uses to publish a reference to a DDS data sample in shared memory: SHMREF_PLAIN (0xc000). This encapsulation indicates that the data sample is in shared memory with a plain language binding (e.g. C++ structure).

(46) Other EncapsulationIds are Possible for Different Language Bindings

(47) The address to a data sample returned by calling CameraImageDataWriten:get_loan( ) is in the local address space of the DataWriter. However, a DataWriter does not know what the corresponding address will be in a matching DataReader's address space, because the OS may map the shared memory region to a different address space. Thus, a shared memory reference EncapsulationId cannot contain absolute addresses, but rather must provide sufficient information for a DataReader to determine the corresponding local address after the shared memory segment owned by the DataWriter is mapped into the DataReader memory space. The shared memory reference has the layout shown in FIG. 7.

(48) When a DataReader receives the reference, it uses the following steps to get an address in its local address space: 1. Attaches to the shared memory segment identified by the shared memory id. 2. Stores the base address of the mapped shared memory region in local memory. 3. Adds sample offset to the base address to obtain the absolute data sample local address.

(49) The Memory Representation of a Sample for Out-of-Band Transfer Modes

(50) The memory representation of the samples that will be used for out-of-band transfer (e.g. shared memory data sample) cannot contain memory addresses (e.g. pointers in C++). However, they may contain offsets to the beginning or some other location in the sample. The offsets can be mapped by the DataReader using the Language Binding APIs to the local addresses of the members.

(51) The Locator and Network Representation Selection Process

(52) To communicate with a DataReader, a DataWriter must choose first the locator to which to send the data samples and second the network representation used for the samples.

(53) DataReaders can be reachable on multiple locators that are announced as part of the DDS discovery process.

(54) In standard DDS, there is no way to indicate what network representation is preferred on a locator. The standard only allows a global selection of the network representation across all locators. If a DDS system contains applications running on the same node and on different nodes at the same time, with a global selection of the network representation, the DataWriters could not simultaneously support zero-copy transfer over shared memory using SHMREF_PLAIN EncapsulationId and publish data samples across the network using XCDR1 or XCDR2 EncapsulationIds.

(55) To solve this problem, this invention adds additional information to a locator ‘LA’ to allow a DataReader to request a set of encapsulations for the locator ‘LA’. A DataWriter uses this list to determine the best encapsulation to publish data samples to the DataReader on ‘LA’. The additional information for a locator is a list of preferred encapsulations in order of priority. For example, the locator for a shared memory transport may look as shown in FIG. 8.

(56) This locator informs a DataWriter that the DataReader prefers shared memory references over the standard inband XCDR1 encapsulation.

(57) To select SHMEM_REF as the network representation, the DataWriter must determine that the DataReader is co-located within the same node. This invention uses the information in the DataReader locator to make this decision; it only selects SHMEM_REF as the network representation when requested as part of a locator from which the DataWriter can infer that the DataReader is running on the same node.

(58) For a shared memory transport locator, the DataWriter can determine if the DataReader is on the same node as follows: 1. The DataWriter tries to attach to the shared memory segment identified by the shared memory id specified in the locator port. If it does not succeed, the DataWriter knows that the DataReader is on a different node and it will not send data samples to the DataReader on this locator. 2. If attaching to the shared memory segment succeeds, the DataWriter compares a sixteen-byte identifier in the header of the shared memory segment created by the shared memory transport with a sixteen-byte identifier received in the shared memory locator transport address. If these two values are different, the DataWriter knows that the DataReader is not on the same node and it will not send data samples to the DataReader on this locator. 3. Otherwise, the DataReader and DataWriter are co-located and the DataWriter can publish data samples using SHMEM_REF encapsulations to the DataReader.
Detecting an Inconsistent Data Sample in the Receiving Application

(59) Zero-copy transfer over shared memory makes no copies of the published data samples. The data samples processed by a DataReader reside in a shared memory segment created by the DataWriter.

(60) The DataWriter can decide to reuse a data sample as soon as the DataReader acknowledges the reception of the sample, but before the DataReader's application processes the sample. Reusing a sample means that the DataWriter can make it available to the DataWriter's application via the new DataWriten:get_loan( ) operation and the application may overwrite the sample contents.

(61) This invention adds a new API to the DataReader API that an application can use to determine if the data sample has been modified before the application is done processing the sample:

(62) TABLE-US-00008 DDS_ReturnCode_t FooDataReader::is_data_consistent( DDS_Boolean &is_data_consistent, const Foo &sample, const DDS_SampleInfo &sample_info);

(63) To support the implementation of this method, this invention sends an EPOCH value as part of the inline Qos of the RTPS DATA submessage containing the reference to the shared memory sample. The EPOCH value is increased each time the application gets a loan on a data sample from the DataWriter and is stored in a header associated with the data sample in shared memory. In addition, the EPOCH is sent when a loaned data sample is published using the new inline Qos parameter.

(64) When an application calls DataReader::is_data_consistent( ), the implementation compares the value of the EPOCH received on the wire in the RTPS DATA submessage with the value of the EPOCH contained in the shared memory segment for the sample. If they are different, the output value is_data_consistent is set to false.

(65) Performance

(66) The latency with zero-copy delivery is independent of the sample size, making the capability highly scalable. FIG. 9 shows a comparison for transmission latency between a zero-copy transfer using the SHMEM_REF network representation of a data sample and an inband transfer using the standard inband XCDR1 network representation of a data sample as the serialized contents of an RTPS DATA sub-message.

(67) Embodiments of this invention can be in the form of a method, system, computer-implemented method executable by computer hardware, computer code where methods steps are executable by a computer processor, distributed over the Internet where the system or method steps are executed by a computer server, or the like.