-
Notifications
You must be signed in to change notification settings - Fork 904
Media Driver Operation
This section assumes familiarity with Aeron Terminology.
The iodriver is a separate process that provides a means to demux and provide buffers of data for Aeron to process from various transmission media. It decouples the means of data transmission from protocol processing. There can be many different iodrivers for various media, to take advantage of different optimizations (such as Linux epoll and sendmmsg/recvmmsg).
The API/stack (or just API) is the Aeron library that interacts with the application and provides the API as well as the bulk of the protocol processing. The iodriver and the API/stack communicate via a set of shared memory buffers. A typical iodriver has 2 threads. 1 for sending data and 1 for receiving data.
Each Term (which is unique to a Session and unique to a Channel) is associated with a buffer. This is purely for data.
For receiving data, the iodriver recvs off a socket and must look up Session ID, Channel ID, and Term ID and send that data to a specific buffer.
For sending data, the buffer contains data to be sent to a given socket or communication endpoint. Thus the iodriver picks up the data and just sends it as there is an easy/direct association between buffer and socket or communication endpoint.
There are two control buffers between the API/stack and the iodriver. One is for the API/stack to send control Frames and instructions to the iodriver. The other is for received control Frames and notifications to the iodriver. These buffers imply no ordering. Events are events and order is not assumed. Loss recovery is not performed on these buffers either.
This means that at any one time, the number of shared memory buffers is:
- 1 per Term ID for recv (incoming data)
- 1 per Term ID for send (outgoing data)
- 1 per iodriver for control (incoming to API/stack)
- 1 per iodriver for control (outgoing from API/stack)
How do send buffers get created?
The API creates a new buffer for a new Term ID. This is irregardless of old Session and Channel or new Session and Channel.
How do recv buffers get created?
The API explicitly desires data on a given Channel ID. At this point, it does not know about what sources (Session IDs) and Term IDs will appear. But it does know that the API is interested in a specific Channel ID. The API creates a new buffer for the initial Term ID it will begin with. When data arrives on that Channel ID, the Session ID and Term ID are filled in for that buffer. The iodriver will create new buffers for Terms as needed on this Channel ID. If a new Session ID is seen, it is handled as a new Term would be and the iodriver creates it.
[TLM]: Martin, does the above work for you?
When a new send or recv buffer is created, a control message for iodriver <—> API/stack must be sent to let the other side know of the new channel and act appropriately.
Suggestion of having the end of a Term have a flag point to the next Term. This can work in both API -> iodriver and iodriver -> API directions for Term rollover.
The iodriver and API will need to know about new Term buffers. New Session IDs, Channel IDs, etc. are, in essence, just new Term buffers.
Data reception is probably the more complex case. New Terms need to be created when new Sources (i.e. new Session IDs) are seen with interesting Channel IDs. I think we have to have the ability for the iodriver to create new Term buffers and the API to notice them.
Operation: iodriver controls buffer creation for data reception
- iodriver is instructed by the API what it is interested in. This includes Channel IDs and Transmission Media specifics (such as IP addresses and ports to listen on for UDP).
- iodriver discards any data that does not match. e.g. an uninteresting Channel ID.
- iodriver creates new Term buffers as needed.
- iodriver signals new Terms to the API via Term rollover flags, Term index buffer, etc.
Data sending is easier. The API creates static Terms for each Channel it will send on. When they have to roll to new Terms, the iodriver will know via the rollover flag.
The API and the iodriver communicate with one another via the control buffer. Here is a list of commands that need to be exchangeable between the API and iodriver.
| Message | Parameters | Description |
|---|---|---|
| Add Receiver | Destination String, List of Channel IDs | Have iodriver set up state and listen to a destination and a given set of Channel IDs. |
| Add Source | Destination String, Session ID | Have iodriver set up state to be able to send on a Session |
| Add Channel | Session ID, Channel ID | Have iodriver set up state to be able to send on a Channel within a Session |
| Remove Receiver | Destination String | Have iodriver stop listening to a given destination string for the API. Removes all Channels and Terms as well. |
| Remove Source | Destination String, Session ID | Have iodriver tear down state and forget about sending on a given Session, its Channels, and its Terms totally |
| Remove Channel | Session ID, Channel ID | Have iodriver tear down state and forget about sending on a given Channel and its Terms totally. But keep Session viable. |
| Remove Term | Session ID, Channel ID, Term ID | Have iodriver stop allowing a given Term ID to be recoverable. Once created by the API, Terms stay around until they are removed by the API. |
| Message | Parameters | Description |
|---|---|---|
| Error Response | Code (int), String | Error Response to previous command from API. Indicates command can't be fulfilled. |
| Error Notification | Code (int), String | Notification of an error condition in the iodriver to the API |
As long as the iodriver has no error, it does not respond to the API commands.