Difference: TrbNetStreamedEventBuilding (1 vs. 9)

Revision 9
09 Jul 2009 - Main.JanMichel
Line: 1 to 1
 
META TOPICPARENT name="TrbNetUsersGuide"

StreamedEventBuilding

Added:
>
>
The name "streamed event building" comes from the fact that inside the network there is no place where we can guarantee to be able to store a complete event. Therefor we have to be able to generate SubEventHeaders on the fly without knowledge of the complete datastream.

The data structure sent by frontends, generated by hubs and the SubEventBuilder is described here:
  • ipu_data_format.pdf: A detailed description of the data format in each stage of the readout system.

What is described in this document, is also shown in the scheme drawn below: A drawing of the data structures through all stages of the network

The original Proposal

The part below was the first proposal to do efficient streamed event building, and may not be in complete agreement with the documents above depicting the current development status.
 

The mechanism described here are valid for the IPU Data channel only. The trigger LVL1 and slow control channels will be kept with the old behaviour.
Line: 120 to 132
  -- MichaelBoehmer - 20 Dec 2008
  • seb.pdf: Example streaming event builder
Added:
>
>

 
META FILEATTACHMENT attr="" comment="Example streaming event builder" date="1229873957" name="seb.pdf" path="seb.pdf" size="20767" user="MichaelBoehmer" version="1.1"
Added:
>
>
META FILEATTACHMENT attr="" comment="A detailed description of the data format in each stage of the readout system." date="1247136322" name="ipu_data_format.pdf" path="ipu_data_format.pdf" size="170128" user="JanMichel" version="1.1"
META FILEATTACHMENT attr="" comment="A drawing of the data structures through all stages of the network" date="1247136366" name="seb2_eb2.pdf" path="seb2_eb2.pdf" size="26866" user="JanMichel" version="1.1"
META FILEATTACHMENT attr="" comment="A drawing of the data structures through all stages of the network" date="1247136499" name="seb2_eb2.png" path="seb2_eb2.png" size="63069" user="JanMichel" version="1.1"
 
META TOPICMOVED by="JanMichel" date="1230908827" from="DaqSlowControl.TRbNetStreamedEventBuilding" to="DaqSlowControl.TrbNetStreamedEventBuilding"
Revision 8
25 Jun 2009 - Main.MichaelBoehmer
Line: 1 to 1
 
META TOPICPARENT name="TrbNetUsersGuide"

StreamedEventBuilding

Line: 77 to 77
 

The first DHDR word contains the trigger bus information belonging to this data sample, namely:
Changed:
<
<
D[31:28] reserved
>
>
D[31:29] reserved
 
D[28] pack bit (0 = don't pack, 1 = pack)
D[27:24] trigger type
D[23:16] trigger random
Revision 7
14 Jan 2009 - Main.JanMichel
Line: 1 to 1
 
META TOPICPARENT name="TrbNetUsersGuide"

StreamedEventBuilding

Line: 80 to 80
 
D[31:28] reserved
D[28] pack bit (0 = don't pack, 1 = pack)
D[27:24] trigger type
Changed:
<
<
D[23:8] trigger number
D[7:0] trigger random
>
>
D[23:16] trigger random
D[15:0] trigger number
  The second DHDR word contains the length of the data packet following, as well as the TRBnet address of the module the data originates from:
Revision 6
02 Jan 2009 - Main.JanMichel
Line: 1 to 1
 
META TOPICPARENT name="TrbNetUsersGuide"

StreamedEventBuilding

Line: 122 to 122
 
  • seb.pdf: Example streaming event builder

META FILEATTACHMENT attr="" comment="Example streaming event builder" date="1229873957" name="seb.pdf" path="seb.pdf" size="20767" user="MichaelBoehmer" version="1.1"
Added:
>
>
META TOPICMOVED by="JanMichel" date="1230908827" from="DaqSlowControl.TRbNetStreamedEventBuilding" to="DaqSlowControl.TrbNetStreamedEventBuilding"
Revision 5
23 Dec 2008 - Main.JanMichel
Line: 1 to 1
 
META TOPICPARENT name="TrbNetUsersGuide"

StreamedEventBuilding

Changed:
<
<
A proposal presented on the DAQ meeting on 18. and 19. Dec 2008 is layed out here for discussion.
>
>
The mechanism described here are valid for the IPU Data channel only. The trigger LVL1 and slow control channels will be kept with the old behaviour.
 
Changed:
<
<
A way of easy subevent building during TRBnet transport is given here. Please contribute in the discussion!
>
>
A proposal presented on the DAQ meeting on 18. and 19. Dec 2008 is layed out here for discussion. A way of easy subevent building during TRBnet transport is given here. Please contribute in the discussion!
 

Naming convections

Line: 13 to 12
 

Frame : assembly of packets, starting with a header and ending with a termination.
Changed:
<
<
The "old" behaviour of hubs is a simple concatenation of all incoming streams to one long stream:
>
>
The normal behaviour of hubs is a simple concatenation of all incoming streams to one long stream:
  H - D - D - H - D - D - H - D - T

For subevent building we need a slightly different behaviour with a common header in front and with or without the original headers from the FEE.
Line: 35 to 34
 
2 reserved    
3 slow control moderate data moderate to huge
Deleted:
<
<
Return data on channel 3 has to be assigned to individual modules, and the way of data transport can show the network connections it was transported on. Therefore the not packet frame concenation mechanism is perfect here and gives us additional information about paths in the network.
  Return data on channel 1 will be streamed out of the network at a certain point to cross over to Gigabit ethernet towards the event builder. It is recommended to have all network related parts removed from there to get rid of unneeded data which would only occupy expensive disk space. A kind of "subevent building on the fly" is necessary here.
Deleted:
<
<
The "not packed" style can be used on all channels without drawbacks, the packed style can most probably only be used on channel two with globally unique FEE channel numbers.
 

Changes in the packet format

The packet structure needs a small change for increasing data security on the network.
Line: 52 to 46
  the frame. The number of packets will exclude the HDR itself and the TRM, and therefore only denote the number of packets inside this frame.
Changed:
<
<
The length information is by no means a strict limit on TrbNet. It is only needed for building Subevent Headers on channel 2 - there is no impact to TrbNet when the number of announced data packets is exceeded! This also means that the length information is an optional feature despite on channel 1. A value of xFFFF implies an unknown length of the transfer.
>
>
The length information is by no means a strict limit on TrbNet. It is only needed for building Subevent Headers on channel 2 - there is no impact to TrbNet when the number of announced data packets is exceeded! This also means that the length information is an optional feature and only used on channel 1. A value of xFFFF implies an unknown length of the transfer, which can be used on any channel despite channel 1.
 

Streamed event building

Line: 64 to 58
  Requirements for this task:

  1. each endpoint data transmission on IPU channel must start with a fixed header (DHDR)
Changed:
<
<
  1. each endpoint must return at least the header when being ordered to deliver data (single TRM as answer are fine)
  2. empty datapackets are allowed, but must contain a header (single TRM as answer are fine)
  3. each endpoint must inform the streaming event builder if data can be packed or not (i.e. if data words are complete or need the source address information). Here, Bit 12 of the HDR can be used.
>
>
  1. each endpoint must return at least the header when being ordered to deliver data
  2. empty datapackets are allowed, but must contain a header
  3. each endpoint must inform the streaming event builder if data can be packed or not (i.e. if data words are complete or need the source address information).
 

During streaming event building it is mandatory to check trigger information of data segments when concenating data streams. In case of mismatch this event is further processed, but marked
Line: 78 to 72
 

DHDR format

Changed:
<
<
Data to be transported on IPU channel is transfered 32bit wide. The smallest unit to be used it 32bit.
>
>
Data to be transported on IPU channel is transfered 32bit wide. The smallest unit to be used is 32bit.
  The DHDR consists of two 32bit words.
Changed:
<
<
The first header word contains the trigger bus information belonging to this data sample, namely:
>
>
The first DHDR word contains the trigger bus information belonging to this data sample, namely:
 

D[31:28] reserved
D[28] pack bit (0 = don't pack, 1 = pack)
Line: 89 to 83
 
D[23:8] trigger number
D[7:0] trigger random
Changed:
<
<
The second header word contains the length of the data packet following, as well as the TRBnet address
>
>
The second DHDR word contains the length of the data packet following, as well as the TRBnet address
  of the module the data originates from:

D[31:16] length in 32bit words
D[15:0] TRBnet address
Changed:
<
<
This header must be delivered from each endpoint on each data request on channel 1 where data is returned.
>
>
This header must be delivered from each endpoint on each data request on channel 1, even when there is no data to be returned.

If the endpoint uses an intrinsic data format with a word size different to 32bit, it has to care for the padding to the next 32bit boundary.
 
Deleted:
<
<
If the endpoint uses some kind of data packing, which takes several data words into one 32bit word, it is up to the endpoint logic to pad the last 32bit word with appropriate data.
 

Procedure explained on an example

Line: 116 to 110
 

A new TrbNet HDR packet is prepared, with a new length information and a new source of data, which is the TrbNet address of the hub.
Changed:
<
<
The first two data words now will build up a data header for the concenated data stream, as defined above. This allows simple staggering several layers of hubs to concenate data streams in a transparent way.
>
>
The first two data words now will build up a new DHDR for the concatenated data stream, as defined above. This allows simple staggering several layers of hubs to concatenate data streams in a transparent way.
 
Changed:
<
<
Now data from the first FIFO is being transfered to the transmit FIFO, yielding in the data stream depictured on the right side of the sheet (named "TRBnet unpacked" and "TRBnet packed"). The TRM from the first FIFO is discarded, then data from the second FIFO is appended. The third FIFO follows.
>
>
Now data from the first FIFO is being transfered to the transmit FIFO, yielding in the data stream depicted on the right side of the sheet (named "TRBnet unpacked" and "TRBnet packed"). The TRM from the first FIFO is discarded, then data from the second FIFO is appended. The third FIFO follows.
 

Finally a TRM packet is created and inserted into the transmit FIFO. In case anything went wrong during data concenation (like: mismatching trigger information, wrong length information, broken packets, ...) the errorbits in the TRM packet will be used to inform the CTS about this failure.
Changed:
<
<
On the hub endpoint where the concenated data stream is streamed out of TrbNet and delivered to some other entitiy (preferably a TCP/IP connection to event builder) all TrbNet related stuff is taken out (named "TCP/IP unpacked" and "TCP/IP packed"). A simple logic will have to convert the leading two header words to the standard HADES subevent header.
>
>
On the hub endpoint where the concenated data stream is streamed out of TrbNet and delivered to some other entitiy (preferably a TCP/IP connection to event builder) all TrbNet related stuff is taken out (named "TCP/IP unpacked" and "TCP/IP packed"). A simple logic will have to convert the leading two DHDR words to the standard HADES subevent header.
 

-- MichaelBoehmer - 20 Dec 2008
  • seb.pdf: Example streaming event builder
Revision 4
22 Dec 2008 - Main.JanMichel
Line: 1 to 1
 
META TOPICPARENT name="TrbNetUsersGuide"

StreamedEventBuilding

Line: 9 to 9
 

Naming convections

Changed:
<
<
Packet : 80bit of data inside TRBnet, consisting of H0, F0, F1, F2 and F3.
>
>
Packet : 80bit of data inside TRBnet, consisting of H0, F0, F1, F2 and F3. The here used packet types are header (HDR), termination (TRM) and data (DAT)
 

Frame : assembly of packets, starting with a header and ending with a termination.
Deleted:
<
<
Frames may consist of several frames with their termination removed.
 
Changed:
<
<
Example: H - H - D - D - D - H - D - H - D - H - D - T
>
>
The "old" behaviour of hubs is a simple concatenation of all incoming streams to one long stream: H - D - D - H - D - D - H - D - T

For subevent building we need a slightly different behaviour with a common header in front and with or without the original headers from the FEE.

Example "not packed": H - H - D - D - D - H - D - H - D - H - D - T

Example "packed": H - D - D - D - D - D - D - D - T

A "short" frame consists only of a termination.
 
Deleted:
<
<
A "short" frame consists only of a termination and is a valid frame only on the return channel.
 

Channel usage on TRBnet

TRBnet offers four virtual channels (CH0 - CH3) with decreasing priority in order to serve for different purposes.
Channel usage uplink downlink
Changed:
<
<
0 highest priority, used as trigger distribution channel few packets just termination
>
>
0 highest priority, used as trigger distribution channel single packets just termination
 
1 IPU data channel, used for transporting data from FEs to CTS few packets huge data amounts
2 reserved    
3 slow control moderate data moderate to huge

Return data on channel 3 has to be assigned to individual modules, and the way of data transport
Changed:
<
<
can show the network connections it was transported on. Therefore the normal TRBnet frame concenation mechanism is perfect here.
>
>
can show the network connections it was transported on. Therefore the not packet frame concenation mechanism is perfect here and gives us additional information about paths in the network.
 

Return data on channel 1 will be streamed out of the network at a certain point to cross over to Gigabit ethernet towards the event builder. It is recommended to have all network related parts removed from there to get rid of unneeded data which would only occupy expensive disk space. A kind of "subevent building on the fly" is necessary here.
Added:
>
>
The "not packed" style can be used on all channels without drawbacks, the packed style can most probably only be used on channel two with globally unique FEE channel numbers.
 

Changes in the packet format

The packet structure needs a small change for increasing data security on the network. A 16bit field in the header will be used to inform about how many TRBnet packets will form
Changed:
<
<
the frame. The number of packets will exclude the header and the termination, and therefore only
>
>
the frame. The number of packets will exclude the HDR itself and the TRM, and therefore only
  denote the number of packets inside this frame.
Changed:
<
<
A HDR packet will always be followed by exactly one TRM packet, so we can exclude those two packets from the length, and keep the length information easy to handle.
>
>
The length information is by no means a strict limit on TrbNet. It is only needed for building Subevent Headers on channel 2 - there is no impact to TrbNet when the number of announced data packets is exceeded! This also means that the length information is an optional feature despite on channel 1. A value of xFFFF implies an unknown length of the transfer.
 

Streamed event building

Line: 58 to 63
 

Requirements for this task:
Changed:
<
<
  1. each endpoint data transmission on IPU channel must start with a fixed header
  2. each endpoint must return at least the header when being ordered to deliver data
  3. empty datapackets are allowed, but must contain a header
  4. each endpoint must inform the streaming event builder if data can be packed or not (i.e. if data words are complete or need the source address information)
>
>
  1. each endpoint data transmission on IPU channel must start with a fixed header (DHDR)
  2. each endpoint must return at least the header when being ordered to deliver data (single TRM as answer are fine)
  3. empty datapackets are allowed, but must contain a header (single TRM as answer are fine)
  4. each endpoint must inform the streaming event builder if data can be packed or not (i.e. if data words are complete or need the source address information). Here, Bit 12 of the HDR can be used.
 

During streaming event building it is mandatory to check trigger information of data segments when concenating data streams. In case of mismatch this event is further processed, but marked
Line: 72 to 76
  must stop the DAQ.
Changed:
<
<

Header format

>
>

DHDR format

 

Data to be transported on IPU channel is transfered 32bit wide. The smallest unit to be used it 32bit.
Changed:
<
<
The header consists of two 32bit words.
>
>
The DHDR consists of two 32bit words.
 

The first header word contains the trigger bus information belonging to this data sample, namely:
Changed:
<
<
D[31:29] reserved
>
>
D[31:28] reserved
 
D[28] pack bit (0 = don't pack, 1 = pack)
D[27:24] trigger type
D[23:8] trigger number
Line: 91 to 95
 
D[31:16] length in 32bit words
D[15:0] TRBnet address
Changed:
<
<
This header must be delivered from each endpoint on each data request.
>
>
This header must be delivered from each endpoint on each data request on channel 1 where data is returned.
 

If the endpoint uses some kind of data packing, which takes several data words into one 32bit word, it is up to the endpoint logic to pad the last 32bit word with appropriate data.
Revision 3
21 Dec 2008 - Main.MichaelBoehmer
Line: 1 to 1
 
META TOPICPARENT name="TrbNetUsersGuide"

StreamedEventBuilding

Line: 91 to 91
 
D[31:16] length in 32bit words
D[15:0] TRBnet address
Added:
>
>
This header must be delivered from each endpoint on each data request.
 
Changed:
<
<
TO BE CONTINUED
>
>
If the endpoint uses some kind of data packing, which takes several data words into one 32bit word, it is up to the endpoint logic to pad the last 32bit word with appropriate data.

Procedure explained on an example

Let's take an three endpoints of a detector, each one delivering data. Take a look into the PDF attached. The first endpoint delivers three data words, the second none, and the third two.

Data is put into the API and is sent to the hub where it arrives in the receive FIFOs as TrbNet packets.

The streaming event builder logic now removes the HDR packets from all three FIFOs, storing only the length information. Next, the DAT header is removed, and the next two trigger words from all three FIFOs are compared against each other. These two words are also stored.

The next word from the FIFOs delivers the data length of each data seqment. Together with the information from the TrbNet headers the new length of the concenated data segment is calculated. The length calculation depends on the PACK bit from the data header; if it is set to one, the logic knows that all data words are containing unique channel information, so the individual headers can be stripped from data stream. If set to zero, the original headers will stay in the data stream.

A new TrbNet HDR packet is prepared, with a new length information and a new source of data, which is the TrbNet address of the hub.

The first two data words now will build up a data header for the concenated data stream, as defined above. This allows simple staggering several layers of hubs to concenate data streams in a transparent way.

Now data from the first FIFO is being transfered to the transmit FIFO, yielding in the data stream depictured on the right side of the sheet (named "TRBnet unpacked" and "TRBnet packed"). The TRM from the first FIFO is discarded, then data from the second FIFO is appended. The third FIFO follows.

Finally a TRM packet is created and inserted into the transmit FIFO. In case anything went wrong during data concenation (like: mismatching trigger information, wrong length information, broken packets, ...) the errorbits in the TRM packet will be used to inform the CTS about this failure.

On the hub endpoint where the concenated data stream is streamed out of TrbNet and delivered to some other entitiy (preferably a TCP/IP connection to event builder) all TrbNet related stuff is taken out (named "TCP/IP unpacked" and "TCP/IP packed"). A simple logic will have to convert the leading two header words to the standard HADES subevent header.
 

-- MichaelBoehmer - 20 Dec 2008
Added:
>
>
  • seb.pdf: Example streaming event builder
 
Added:
>
>
META FILEATTACHMENT attr="" comment="Example streaming event builder" date="1229873957" name="seb.pdf" path="seb.pdf" size="20767" user="MichaelBoehmer" version="1.1"
Revision 2
21 Dec 2008 - Main.MichaelBoehmer
Line: 1 to 1
 
META TOPICPARENT name="TrbNetUsersGuide"

StreamedEventBuilding

Line: 43 to 43
 

The packet structure needs a small change for increasing data security on the network. A 16bit field in the header will be used to inform about how many TRBnet packets will form
Changed:
<
<
the frame. The number of packets will include the header and the termination (?).
>
>
the frame. The number of packets will exclude the header and the termination, and therefore only denote the number of packets inside this frame.

A HDR packet will always be followed by exactly one TRM packet, so we can exclude those two packets from the length, and keep the length information easy to handle.
 

Streamed event building

 
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Hades Wiki? Send feedback
Imprint (in German)
Privacy Policy (in German)