Adaptive feedback cancelation example

Hey all - is there a current working implementation of the BTNHRH adaptive feedback cancelation algorithm?

The only examples I see are in the Deprecated folder on Github: https://github.com/Tympan/Tympan_Library/tree/main/examples/05-FullSystems/Deprecated

I ran the Deprecated examples on a regular Teensy 4.1 board with audio shield and just am getting loud white noise, so I also may need to update the config to use the regular Teensy Audio shield.

Any tips or thoughts here would be appreciated.

Hi Zach!

It’s deprecated?? That’s not what I remembered . I thought that we had working examples. Hmm. Let me check it out…

Chip

You’re right! I can’t find an example. Strange.

Let me see if I can make one. This might take me a day or two.

Note that feedback cancellation is something that is very sensitive to your particular hearing processing that you’re doing (multiband compression?) and on the relative location of mics and speakers (mostly, it’s the distance that matters). A lot of tuning is needed of the cancellation parameters. Hearing aid companies spend a lot of time and money tuning their algorithms for each new hearing aid.

So, to help me make a relevant example:

  • What kind of audio processing are you doing in addition to feedback cancellation?

  • Are your mic and speaker close together? Are you using the Tympan earpieces or something else?

Chip

Hey Chip!

I’m going to be doing multiband compression and some equalization in addition to the feedback cancellation - perhaps a 3 or 8 band EQ. Will likely add a high pass filter before the multiband compression, but we’ll see.

As far as how close the mic and speaker are, they are fairly close together - ranging between 10 to 15mm apart at the moment but could change slightly. I’m not using the Typman earpieces. I’m using a single mic for each ear (currently testing just a single side/single ear). In particular, the Infineon IM68A130V01 (https://www.infineon.com/dgdl/Infineon-IM68A130-DataSheet-v01_00-EN.pdf?fileId=8ac78c8c85ecb34701860371623f1204) is what I’m using for a mic.

Thanks for your help! If there is any way I can help, let me know.

Hi Zach,

I made some progress and posted some code for you!

But first, a little history:

  • The AFC algorithm is from Boys Town (BTNRH) as you mentioned. But, they have done several versions…all using the same algorithm, but they kept making it better.

  • Unfortunately, the version currently implemented in the Tympan library is quite old. It is my intention to update it, but that can’t happen today.

  • The main limitation of the current algorithm is that it becomes corrupted once it becomes overloaded. So, if it starts to feedback (despite the AFC) and you keep increasing the gain, the system will overload, which corrupts all of the algorithm states, which results in the system going silent. To recover, you have to cycle the power. Sorry.

For you to start playing with it, I added a new category of examples to the Tympan library. Here’s what you do:

  • Re-download the Tympan Library from GitHub (https://github.com/Tympan/Tympan_Library)
  • Restart the Arduino IDE
  • In the Arduino’s “Examples” menu, go to “Tympan_Library” and then you should see the new category of examples “09-FeedbackCancellation”
  • Open the “BasicGain_wAFC” example, compile it, and put it on your Tympan (assuming you have a Tympan RevE?)

This basic example is just gain plus the AFC. I’ll make more complicated examples later, but this should get you started.

This example uses the microphone built into the Tympan (you can change this in the code). I then connected some inexpensive earbuds. The example is mono (not stereo) so that you can put one earbud near your ear and you can use the other earbud near the Tympan’s mic in order to generate feedback. See this pic!

You can control the AFC via the Arduino’s serial monitor. Send an ‘h’ (no quotes) to get the help menu.

Here’s a test procedure that I’ve been using:

  1. Get everything ready: put the code on your Tympan, open the serial monitor, turn the Tympan’s volume knob until the serial monitor says 0.0 dB, put the earbud near the Tympan’s mic (like in the pic)

  2. Turn off the AFC (send a ‘A’, without quotes)

  3. Slowly increase the gain (send a ‘k’) repeatedly until you hear it start to feedback. Lower the gain (send ‘K’) until the feedback goes away. This is your “maximum stable gain” without AFC

  4. Activate the AFC (send ‘a’)

  5. Slowly increase the gain (send ‘k’) again until you start to hear it jitter into feedback. Lower the gain (send ‘K’) until the jitter/feedback goes away. This is your new maximum stable gain with AFC.

The difference between (5) and (3) is the improvement enabled by the AFC.

In my testing just now (with the built-in mic and my own earbuds), the max stable gain without AFC occured at a gain of +12 dB (digital gain). When I activated AFC, I could increase the gain to a setting of +25 dB (digital gain). This means that the AFC enabled me to use an addition 13 dB of gain.

I’ll post an improved AFC algorithm when I get a chance to work on it.

Good luck!

Chip

Chip, this is great - thank you so much! I’m gonna sit down with it over the weekend and get it working. Seriously, thanks for the updated examples.

If there is anything I can do with getting the algorithm updated to be using the latest implementation, let me know. Feel free to point me in a direction and I can see if I can update it and contribute.

Does the newest improvement to the algorithm not corrupt when it becomes overloaded?

I posted an update to GitHub regarding the code from Friday. My latest code doesn’t function any differently. In simply I renamed some things in the old algorithm to make it easier to transition to the new algorithm.

The instructions in my post above are unchanged.

Chip

Understood, thanks Chip. You had mentioned the following:

  • The AFC algorithm is from Boys Town (BTNRH) as you mentioned. But, they have done several versions…all using the same algorithm, but they kept making it better.

My question about the newest improvement to the algorithm is about that. Does the latest version of their algorithm not corrupt when overloaded?

Hi Zach,

Short answer: no, the updated NLMS algorithm (ie, example 01) still gets locked into silence after being driven into heavy overload.

Long answer: I’m not sure why it locks into silence, but this seems like it should be easily fixed. It’s only my list of things to look at. I’m assuming it’s because somewhere in the state information held within the algorithm (like the ring-buffer used to estimate the real-world’s echo path) gets filled with invalid numbers or something. It’ll take a bit of investigation.

Update: I fixed the problem! See my post a couple of posts down.

Chip

Hi Again,

I just got the second AFC algorithm working, along with an example sketch. This new algorithm “NFXLMS” has a very similar structure to the previous on (“NLMS”)…they’re both “LMS” algorithms, for example. This new algorithm has more features, including (1) a band-limiting filter to help keep the algorithm from getting distracted by low frequencies (which are unlikely to feedback) and (2) a whitening filter to help the LMS converge better.

The new code has been added to the GitHub repository, so you’ll have to download it again. The new example is example 02, in the examples directory just like the previous example.

Very helpfully, I have documentation about the algorithm! I found a short white paper written the folks at BTNRH who authored the original code (that I ported over here to Tympan). I put a PDF of white paper in the directory holding example 02. You can also access it here

In my short time testing the algorithm on my Tympan RevE, the new NFXLMS algorithm does behave a bit differently than the original NLMS algorithm. I’m not sure if it’s better or worse, it’s just different. So, try out both and see which you like better!

To answer your other question, I don’t remember what the heavy-overload behavior of the new NFXLMS algorithm is. I didn’t notice whether it locks up like the NLMS. I suspect that it does, however, since it has a very similar structure. Like I said, though, I suspect it’s a fairly easy fix by simply adding a step where it looks for any invalid numbers in the filter’s ring buffers.

Update: The new NFXLMS algorithm does NOT appear to lock-up, even when driven into heavy overload. Cool!

Enjoy the new algorithm!

Chip

Hey Zach,

I fixed the problem with the lock-up behavior with the old NLMS algorithm.
It was pretty easy to fix.

The issue was indeed as I suspected…when the system was driven into heavy acoustic overload, the algorithm was trying harder and harder to cancel out the loud audio…but all that hard work was driving the algorithm’s internal states to values of INF or NaN. Unfortunately, once you get an INF or NaN, the math in the algorithm gets stuck because you can’t add/subtract/multiple/divide away from an INF or NaN.

So, to fix this, I detect for this state and then audio-reset all of the algorithm’s states. The system sounds terrible while it’s being heavily overloaded (as you’d expect), but once you pull the mic away from the speaker (or lower the gain), the system gracefully recovers on it’s own. Excellent.

Now, you should feel free to explore both the NLMS algorithm (ie, “Example 01”) and the NFXLMS algorithm (ie, “Example 02”). Neither have a fatal flaw like the lock-up problem. You can simply choose the one that sounds better to you or that works better with your mic and speakers.

I’ve completed all the tasks that I set out to do. So, I have no more updates planned until request something.

Have fun!

Chip

Chip,

Thanks again!

I’m attempting to test things out using a regular Teensy 4.1 with the Teensy Audio board (not using Tympan hardware since it’s not available right now). Am I correct in thinking I should be able to just use TYMPAN_INPUT_JACK_AS_LINEIN for the audio to pass through from my own mic input using the Teensy Audio Board? (I’m powering the mic from the 3.3v on the audio board).

Currently just getting loud bleeps from the original NLMS algorithm and just getting some static blips from the NFXLMS algorithm.

If it’s a bit more of a lift to get this working on regular Teensy hardware/not the Tympan hardware, let me know.

Hi Zach,

I’ve not tried to run the Tympan Library on the Teensy audio board in a long, long time. Years and years. So, I don’t remember all the subtle issues.

The biggest issue is that the classes that control the hardware will need to be swapped out. So, instead of creating an instance of the Tympan class (ie, the thing that is “myTympan”), you’ll need to swap that out for an instance of AudioControlSGTL5000. As an example for how to create and use the SGTL5000 class, look at the Teensy Audio example for “PassThroughStereo” (https://github.com/PaulStoffregen/Audio/blob/master/examples/HardwareTesting/PassThroughStereo/PassThroughStereo.ino)

After swapping in the AudioControlSGTL5000, you’ll also need to swap out any function names attached to the Tympan class. The function names used by the Tympan class to control the Tympan hardware are not necessarily the same function names as used by the SGTL5000 class to control the SGTL5000 hardware. Your question about how to select the audio input is an excellent example of this. In this case, the function name is OK, but the arguments passed to the function need to be changed.

  • For the Tympan, there would be a line like: myTympan.inputSelect(TYMPAN_INPUT_JACK_AS_LINEIN);.

  • But, for the SGTL5000 class (according to the PassThroughStereo example), you’d write something like: sgtl5000_1.inputSelect(AUDIO_INPUT_LINEIN);

You’ll need to look through your code and find every function attached to the myTympan class. Each one will need to be replaced by some sort of corresponding method in the SGTL5000 class (or, you can simply try to remove the function from your sketch).

OK, great. All of the advice above relates to trying to swap any Tympan example sketch over to run with the SGTL5000. But, you may ask, are there any concerns specifically with running AFC on the SGTL5000? The short answer is that I’ve never done it, so I don’t really know what the pitfalls are. Sorry. But I can think of one potential issue…

Specific to AFC, my Tympan examples use a highpass filter prior to the AFC algorithm receiving the audio. In my examples, I happen to employ a digital filter that is built into the Tympan hardware. In my example code, look for a line like myTympan.setHPFonADC. I don’t believe that there is any corresponding function call for using a filter built into the SGTL5000. So, what do you do?

The purpose of this filter is to remove the low-frequency audio content from the audio stream. AFC algorithms often have difficulty dealing with low-frequency audio. So, to avoid this problem, I simply filtered it away. In my example, I used a digital filter that is built into the Tympan hardware. If you’re not using the Tympan hardware, you can use a (software) filter from the Tympan Library instead. No problem. You can steal the highpass filter stuff from the basic Tympan example “TrebleBoost”.

You’d add the Biquad filter to this section of code:

//create audio library objects for handling the audio
Tympan                      myTympan(TympanRev::E,audio_settings);   //only tested on Tympan RevE
AudioInputI2S_F32           i2s_in(audio_settings);                  //Digital audio *from* the Tympan AIC. 
AudioFilterBiquad_F32     hp_filt1(audio_settings);   //Biquad (IIR) filter doing a highpass filter.
AudioFeedbackCancelNLMS_F32 afc(audio_settings);                     //adaptive feedback cancelation (AFC), NLMS method
AudioEffectGain_F32         gain1(audio_settings);                   //Applies digital gain to audio data.
AudioOutputI2S_F32          i2s_out(audio_settings);                 //Digital audio *to* the Tympan AIC.  Always list last to minimize latency
AudioLoopBack_F32           afc_loopback(audio_settings);            //here's how we close the loop on the AFC

You’d connect the filter in this section of code

//Make all of the audio connections
AudioConnection_F32       patchCord10(i2s_in, 0, hp_filt1, 0);          //connect the left input to the highpass filter
AudioConnection_F32       patchCord15(hp_filt1, 0, afc, 0);          //connect the filter to the afc
AudioConnection_F32       patchCord20(afc,    0, gain1, 0);        //connect to the AFC to your audio processing
AudioConnection_F32       patchCord30(gain1,  0, i2s_out, 0);      //output to the Left output
AudioConnection_F32       patchCord31(gain1,  0, i2s_out, 1);      //output to the Right output
AudioConnection_F32       patchCord40(gain1,  0, afc_loopback, 0); //close the loop with the AFC

And, somewhere in the main sketch’s setup() function, you would add this line of code to configure the filter as highpass and to set the cutoff frequency:

float cutoff_Hz = 100.0;   //choose the desired filter cutoff frequency
hp_filt1.setHighpass(0, cutoff_Hz); //configure the biquad filter

Chip

Hey Chip!

Thanks so much for the detailed run down. Your suggested changes to the code worked and I’m able to demo the AFC algorithms. This is great. Really appreciate your time and help on the responses!

Yay!

Glad it worked out for you.

Chip