CWG13 meeting

Friday 12 Feb 2016, 09:30 → 11:00 Europe/Zurich

53/R-044 (CERN)

09:30 → 10:00 Investigations of msgpack/flatbuffers/multipart

Speaker: Alexey Rybalchenko (GSI - Helmholtzzentrum fur Schwerionenforschung GmbH (DE))

ser.pdf

Barth: No Root in the summary? Not tested with attachment, truncated buffers, other problems.

 

Matthias: What is the additional attachment? The use case is a message from the framework and then you add binary blob in the pre-allocated area. It is very similar to what CWG4 is discussing.

 

I am confused. We wanted to have common header and blob…

 

Mikolaj: You have two parts – header and payload. It can be serialized or not.

 

Matthias: It is existing structure + binary addition, so this is not the same use case.

 

Alexey: Sending one by one is faster that in one single part. Pre-allocating is the most generic. Multipart for data + merger dev-ce => have to distinguish which part is from the message and which part from the other device.

 

Mikolaj: Multipart message is guaranteed to be together, there is no mixing. What is the problem?

 

The API has to see if you have another part

 

Mohammad: This is guaranteed by 0MQ, but we are mixing the two things. We have one small header for system purposes and then user header. They are different.

 

Alexey: Merger and receiver exist even now in HLT and there is no problem. The do not mix, no routing problem.

 

Mikolaj: No much difference? The red bar has big advantage… Actually there is no gain between multipart and single part.

 

Pierre: Do you understand the performance? Short messages: penalty. Long messages: similar shape. What is the reason for the long messages?

 

Alexey: It is system dependent but to the TCP implementation.

 

Sylvain: Performance in MB/s? It is difficult to convert, from slide 7 I deduce 20 MB/s, which is very slow.

 

Alexey: A lot of time spent in processing…

 

Silvain: Better  test with realistic message size. What do we have to serialize?

 

It depends

 

Miklaj: Memory and CPU usage? No plot, but the load is high. With larger queue size we have bottleneck.

 

Mohammad: There was no full performance analysis, but we have just introduced the methods in ALFA. Now we have to see how we use the multipart by 0MQ or we do it ourselves.

 

Mikolaj: CWG4 discussed at length this question.

 

Peter: How do we evaluate the features?

 

Mikolaj: We decided not to use serialized headers. For the payloads we can use any.

 

David: We should decouple the serialization from the transport  buffer.

 

Giulio: In TMessage the schema evolution is not off. The comparison is a little biased.

 

Mathhias, Barth: TMessage does not send the class information, even if you ask for it.  TMessage does unneeded things.

 

Giulio: we have to take into account the persistent format at the level of the sink.

 

David: Possibility for multiple serialization at different levels.

 

Matthias: What will be the policy wrt to the external packages?  How do we make sure the projects are alive?

 

Barth: Very valid point? We need formal evaluation, metrics, etc. The maintainability is part of the metrics.

 

Thorsten: Compromise to keep the flexibility and be able to exchange the packages. The only place where we care is the sink, where we have to select the persistent format.

 

Mohammad: It is not mandatory to have only one serialization package and more then one format.

 

Do we agree on the three different components?

• transport leyer (0MQ, nanomessage);
• data protocol (what goes through the wire);
• persistent representation.

 

Yes, and each component has to be exchangeable. This limits the amount of the features we should use.

 

 

If the multipart is not in the transport layer, we can have it  an intermediate layer. This requires additional work, but it is invested only once.  The flexibility is needed, for example nanomsg does not provide multipart messages, so we can implement a protocol on top of it. The question of zero copy is important.

 

CWG4 summary: Most important to have support for multipart messages. We do not want multipart to separate headers from payload. We do not want to support multipart payloads.  This provides easy navigation in the headers. The headers will not be serialized, they are the same everywhere. The payloads will be seraialized. HLT is used as example and developed further.

 

Mohammad: This does not require multipart 0MQ.

 

David: N headers and M payloads should be send from A to B. We can use the multipart feature to transfer in one go, but we also can do it also without.

 

Thorsten: This is 

 

Mohammad: These features do not belong to the transport layer.

 

Matthias: What is transport? 0MQ, FairMW, etc.

 

Mohammad: What about the memory management? This is not in the transport. What about the manual memory management?

 

Predrag: I agree that we should not expand the abstract transport layers with memory management. We will need additional layer for it.

 

Mikolaj: We will have to do this in a realistic case.

 

David: Memory allocation on the receiving side is needed. To avoid copy, the receiver reads the same buffer where the sender puts the data.

 

Meeting in 2 weeks to present the CWG4 requirements, the features of the FairMQ and to agree on the terminology and functionality.

10:00 → 10:20 Results from the Power8 tests

Peter summarized the activities so far. Lionel will prepare a presentation with all the results. We have to provide the performance plots and plan for future investigations.

10:20 → 10:40 AOB

Additional items for the next meeting:

• Omnipath test;
• Parameter management;