Skip Ribbon Commands
Skip to main content
Navigate Up
Sign In

Quick Launch

Average Rating:

facebook Twitter
Print Bookmark Alert me when this article is updated


HOW TO: Collect SolarFlare Hardware TimeStamps From a Ultra Messaging Application for Latency Analysis

Ultra Messaging (UM) allows for collection of timestamp information from SolarFlare NICs when a packet is published onto the wire, and when that packet is received from the wire at the incoming NIC.

However, these two timestamps only reflect the elapsed time between the two NICs.  For a more complete view of where latency might be occurring, the sending application should also collect the current time prior to calling the source send API.  Likewise, the receiving application should collect the current time from within the application callback function which is handling message delivery.

Solarflare recommends the use of the clock_gettime() API call to collect the "current" time.

With these four timestamps, a more complete analysis of where latency is occurring between the two applications can be performed.

More Information
  • Use of Solarflare Enahanced PTP for clock synchronization
  • Use of Solarflare NICs on systems on which timestamps are to be collected; with Solarflare Onload 201509 or later version
  • The Solarflare adapters must have a PTP/HW timestamping license
  • Set Onload environment variable EF_TIMESTAMPING_REPORTING=1
  • Enable Onload acceleration:
    • For all applications: export
    • On a per application basis: onload <application_name> [application options]
  • For TX timestamps:
    • A SolarCapture Pro license or Performance Monitoring license must be installed.
    • Set the Onload environment variable EF_TX_TIMESTAMPING = 1
    • Set UM configuration option (context) transport_lbtrm_source_timestamp.
  • ​​​​For RX timestamps:
    • A Solarflare AppFlex license for hardware timestamps must be installed.
    • Set the Onload environment variable EF_RX_TIMESTAMPING = 1.
    • Set UM configuration option (context) transport_lbtrm_receiver_timestamp.
Details for application:
  • The collection and comparison of timestamps will need to be carefully considered by the development team.
    • Logging or sending this information to a collection point in realtime will impact performance
    • The data would be best logged to memory and then sent from a lower priority thread using separate NICs and transports from that of production data streams to a collection point where the data can be further analyzed.
  • Beyond the timestamps, the UM monitoring data should also be collected as loss events will result in large latency spikes between sending application and the receiving application.
​See Ultra Messaging Concepts Guide, section "High-resolution Timestamps" for the feature description.
See lbmsrc and lbmrcv C language example applications respectively, for source event handling of timestamps and receiver logic to access high resolution timestamps.
Applies To
Product: Ultra Messaging
Problem Type: Performance; Product Feature
User Type: Developer
Project Phase: Configure; Implement; Optimize
Product Version:
Operating System: Linux
Other Software:
Last Modified Date:7/15/2019 9:01 AMID:288807
People who viewed this also viewed


Did this KB document help you?

What can we do to improve this information (2000 or fewer characters)