Project: A General-Purpose Pipeline for Interfacing the Tympan with an External Computer

2021 ASA Challenge
#1

Hello!

For the 2021 ASA Challenge, @nraj and I are working on the design of “A General-Purpose Pipeline for Interfacing the Tympan with an External Computer”.

The motivation for this project was to ensure that applications which have to run certain resource-heavy algorithms (e.g. machine learning) can work with the Tympan by decoupling these algorithms from the underlying audio processing on the Tympan. The algorithms could run on an external computer, using the input audio from the Tympan, and their output could be sent to the Tympan as commands. You can read our full submission here.

We will post an update on the forum as the project progresses and, in the meantime, we’re happy to receive any feedback or questions you have :blush:

1 Like
#2

Thank you for posting. I’m excited to see how this turns out!

1 Like
#3

Yes your proposed work is needed.
I am working on something similar. My piece goes from Teensy’s external ram through the Teensy add-on 100T Ethernet, with some compression, to a Linux host where it gets processed by an Nvidia GPU CUDA process and back. USB just wasn’t fast enough and had too much delay jitter :frowning:
Looking forward to your next Github checkin.
Maybe some Architecture or SWAG docs first?

#4

Hello!

Thanks for your interest in this project. So, what I have done so far is expose the Tympan as a USB soundcard to the PC. Using the Teensy library’s AudioOutputUSB class, I have managed to send the input audio (from the Tympan’s onboard/in-line mics) to the Python script running on an external computer.

However, the catch here is that the Teensy library supports a fixed sampling rate and audio block size by default (44,117 Hz/128 samples per block). In order to change the sampling rate, I had to change a few parameters in the Teensy library code which is not a portable option. I was wondering if you used a different approach for this.

Thanks,
Nithin

#5

Hi Nithin,

QUESTION: How will you deploy you application? How is portability negatively effected by your afore-mentioned sampling rate code change? Isn’t that the essence of FOSS? Just issue a Patch file to apply when you distribute perhaps?

The approach I used is probably even less portable. As I no longer use USB for anything RealTime due to its unpredictable latency and hardware and software overhead. Although USB has sufficient bandwidth I have found it’s latency to be 0-50ms on a PC which is intolerable for a realtime Audio application ( i.e. non-steaming no buffering).

I use instead Teensy’s 4.1 100T Ethernet Interface. I can run point to point between Teensy and a host.
The latency is on the order of Microseconds with no appreciable jitter.Python has easy to use Socket classes for host processing. On the host I then use Python CUDA or Numpy with Intel MKL to do the work with plenty of time to ship the result back. In addition I use UDP for a connectionless lossy protocol that does occasionally drop-out but when working gives plenty of time for audio post processing before hitting the dreaded 0.250ms delay that starts really pissing people off.

I am trying to put together a general transport/host processing framework So others don’t have to do the schlepping but it isn’t very well abstracted or high level for efficiency’s sake and it’s definitely predicated on Ethernet.

Sorry can’t be of more help.
Is a Patch file or something of that sort really out of the question?

#6

Hi Keith,

The patch file seems to be a great idea! That makes a lot of sense in terms of portability. I’ll keep you posted regarding further updates.

Thanks,
Nithin

#7

@Keith.burden: thanks for sharing the limitations for USB latency and jitter and your success using UDP packets. Can you share more details about what you tried for USB? Was the clock rate set to the synchronous, asynchronous, or adaptive mode?

@nraj What did you end up changing to set the USB rate? Did you have to go into the processor’s clock tree to change the clocking rate for the USB peripheral?

#8

@erk1313 I basically just followed this thread, which involves changing the packet size for Rx and Tx (although I couldn’t get USB input to work properly), mentions of the sampling rate for the USB descriptors, etc, in the Teensy library code.

#12

I am new to USB protocols, but it appears that there is sync, async, and adaptive mode.

If they are using adaptive mode, then it means the clock rate is set by the data rate, rather than an explicit clock. This could explain the jitter.

The Teensy should be capable of sending async USB audio, but if you wanted to side-step the issue by buying a IC for the I2S <-> USB bridge, I believe this would work. There is even an eval board you could try out.

#13

And another IC with 8-channel I2S to USB Audio with up to 216kHz sample rate.

#14

@erk1313 USB was in async mode, which is usually the case for small packets send at indeterminate times. Mind you my measurements were NOT on a Teensy but between a Linux host and other Arduinos: Uno, Mega and Due. Which used either physical or virtual USB/Serial converters on the Arduino in USB2.0 with High-Speed, as measured with a CATC/Lecroy USB Chief.
The Ethernet link was 100T with latencies in the 1-10 uSec range, about 1000x less. Therefore this is my interface of choice at least for wired.

#15

Hi @Keith.burden ,

Thank you for the info on your testing. I misunderstood the async protocol - samples are driven by the clock to the DAC/ADC, but the USB packets are not sent on the same timing. So if you had all the time in the world to reassemble th audio, you would have little jitter. There is an older synchronous protocol, but it was problematic.

As you mentioned, a source of the latency/jitter could be on the side of the windows driver. It would be interesting to see what a high-end usb-audio card could do, with specific drivers written for low latency.

Instead on bulk transfer, I think one would want “Isochronous Transfer” rather than Bulk Transfer.

#16

As a reference, here is a off the shelf USB audio interface with 3ms round trip latency:
https://www.thewelltemperedcomputer.com/HW/USB_Interface.htm#XMOS

#17

@nraj

You posted a link to the Teensy thread on how to change the sample rate used by the Teensy Audio Library. While that method does change the rate at which audio samples are acquired by the audio hardware (via the I2S clocking), this won’t change how the Teensy Audio USB link works. To my knowledge, their USB audio classes only work at 44100 Hz. Sorry.

The folks on the Teensy forum have been working on enabling different speeds on the USB audio link. See: PJRC (Teensy) Forum

In that thread, I’m chipaudette there, too. Note my comments about having difficulty getting their modification to function. Hopefully you’ll have better success!

On a side note separate from USB…

Please be aware that, if all you want to do is change the sampling rate of the Tympan hardware for acquiring samples (again, this is different than controlling how audio flows over USB), you don’t have to use the method that you linked to in the Teensy forum. Changing the sample rate is something that we tried to make much easier in the Tympan Audio Library vs the Teensy library. To see how to change the sample rate on the Tympan, see the Tympan example sketch “ChangeSampleRate”: Tympan_Library/ChangeSampleRate.ino at master · Tympan/Tympan_Library · GitHub

Chip