Scalable and expandable audio latency testing platform for AR/XR development

Audio and video latency is a key factor in determining the quality of the user experience in communication services. With eXtended Reality (XR) systems gaining popularity in entertainment as well as professional applications such as training, education, automotive and aviation, ensuring minimal latency is a matter of not only user experience but also safety. A common challenge for all of those systems is to keep latency as low as possible when streaming video and audio, so that the user sees the XR scene in close to real time, with video and audio synchronized with each other. The low-latency synchronization challenge becomes even more complicated when the user moves in a real life scene, with physical movement translated into virtual movement.

Minimizing audio latency specifically is necessary for responsive and natural communication, especially in scenarios requiring interactivity, as significant delays might result in users talking over each other, experiencing echo, and overall poor quality of the service. For one of our customers in this space, Antmicro has created a unified, scalable and expandable audio latency testing platform based on a pair of open hardware boards with dedicated software, all available on GitHub.

In this article we will describe the components of the latency testing system and show how it can be used in various scenarios, in combination with a PC or an embedded platform such as Antmicro’s open source Jetson Orin Baseboard.

Open source Audio Latency Tester

The Audio Latency Tester hardware is composed of two boards: the Audio Latency Tester Board and the Microphone Board. The Audio Latency Tester Board includes circuitry for playing audio via an external speaker and recording audio via a pair of digital microphones. It supports inputs from I2S and PDM microphones. The microphones are placed on separate boards, connected by flex cables, which allows for adjustable spatial arrangement in a test setup.

Audio playing and recording is handled by two RP2040 MCUs with two independent USB connections to the host. The two MCUs are synchronized via GPIO in order to provide a common time base for playback and recording. The Audio Latency Tester Board is equipped with several expansion interfaces for further integration purposes: I2S output, I2C communication, and synchronization event GPIOs.

The MCU firmware is capable of recording and playing audio at different sample rates with variable volume gain, all controlled from the host PC. The user can adjust the record or playback parameters to improve compatibility and performance when interfacing with the device under test.

In order to measure audio latency of a device under test, the host uploads audio samples to the playback MCU and triggers synchronous playback and recording. The input and output audio streams are assigned timestamps and downloaded to the host. By comparing the streams, the host can then match audio events and estimate time shifts between the recordings.

Both the Audio Latency Tester Board and the Microphone Board are available on our Open Hardware Portal where you can find realistic renders with interactive component highlighting, component lists, stackups, schematics and more.

Use case: wireless headset with audio over Bluetooth

To demonstrate a use case of the Tester, the development team measured the audio latency of a wireless headset connected over Bluetooth 5.2. The detailed procedure is provided in the Tester documentation available on GitHub Pages. The instructions will guide you on how to perform measurements, process the data and interpret the results. To make this example easier to reproduce, we prepared a set of tools that automate this process, and shared the raw audio recording for reference.
The tools include a script for identifying how the recording shifts from the reference audio file by comparing the sound start and stop moments. A separate script removes unwanted overlapping of the playback samples with the recorded data, eliminating the audio spill effect.

Scalable next-gen AR/XR development

The open source Audio Latency Tester is fully customizable and can be used, for example, with a portable edge AI platform such as Antmicro’s Jetson Orin Baseboard, whose latest revision supporting both the Jetson Nano and NX SoMs in Super mode is now available for purchase from CircuitHub. In this use case, the Audio Latency Tester can be integrated into one mechanical model, connected to a remote host via Ethernet.

As the Tester can be used with both PCs and embedded platforms, it is easily embeddable into more complex testing setups that handle not only audio measurements but also video latency, which will be the topic of a follow-on note. You can run multiple Audio Latency Tester boards in parallel within larger distributed testing systems for tracking regressions and supporting advanced experimentation in a repeatable manner.

Antmicro develops complete cutting-edge devices for its customers that often combine measurement, processing, and data analysis at the edge. With full-stack development services encompassing hardware, software, OTA update systems and AI, we can help you build, test and deploy your next AR/XR, robotics or industrial project, taking full advantage of Antmicro’s off-the-shelf open source solutions such as the Latency Tester and Jetson Orin Baseboard described in this article. If you would like to learn more about our offering and discuss your specific needs, reach out to us at contact@antmicro.com.