Add the exchange of a "supported codecs" list between clients and add a "PCM 8 bit" codec

[florianschmeding]

This PR consists of two commits with two consecutive features.

The first commit adds the exchange of codec support information between clients so that the clients know which codecs are supported by their peers. The logic falls back to the currently supported codecs in v1.6.2 if no codec support information is received. This feature can be used to develop new codecs, to drop support of codecs or theoretically also for an individual user to have a choice for whitelisted codecs. UUIDs are used for the unique identification of codecs and codecs that aren't supported by a peer won't be in the visible in the dropdown by the user.

The second commit uses the previously described codec support information exchange feature to add an 8-bit PCM mode to the current PCM codec without disrupting clients that don't have this implementation. The 8-bit mode lowers the bandwidth requirements without adding the 2.5ms extra latency of the Opus codec. Furthermore noise shaped dithering is applied at encoding to achieve an acceptable audio quality. Still the new mode is marked as "(*) not recommended", because it adds some noise.

I implemented these features to jam with some friends at a lower latency while maintaining compatibility with older clients. Lower latency was achieved by sending redundant UDP packets with the freed bandwidth that was gained from switching from the 16-bit PCM over to the 8-bit PCM codec. This PR contains the codec changes and I'll open a second PR for the option to enable redundant packet transmission.

PCM 8-bit selection

[essej]

Also interesting! However, in the pretty near future I'll be transitioning to a newer version of the AOO upstream library which has changed a bunch of stuff, so I'll probably wait until after that to integrate this one. I can introduce the 8bit PCM concept to the AOO developer now, although it may be more appropriate as its own codec instead of putting it into the existing PCM, because of the noise shaping, but really it's probably fine in there!

I'm curious about what aspects of what you did in this patch could affect the latency in the existing version though.... is it mostly relating to the redundant packets allowing for smaller jitter buffer size?

There are also several possible improvements in the new AOO which should affect low latency... due to a complete jitter buffer re-organization. But I'm still testing that stuff...

[Spacechild1]

I can introduce the 8bit PCM concept to the AOO developer now

@essej 8bit PCM is already supported in the AOO develop branch :-) Dither is actually a nice idea and could be added as a codec control.

[Spacechild1]

The 8-bit mode lowers the bandwidth requirements without adding the 2.5ms extra latency of the Opus codec

@florianschmeding Are you seriously trading 2.5 ms extra latency for 8-bit PCM audio? :-D

Someone requested 8-bit PCM for embedded devices which are too weak for Opus encoding/decoding. But IMO there is hardly any reason to use this on a desktop...

[florianschmeding]

@Spacechild1 hehe, valid question :)

While jamming it's 5 ms if you add both directions together and I just really hate any latency, especially at faster speeds like 180bpm+ and to me the dithered 8-bit PCM doesn't sound THAT bad.

But yeah, if enough bandwidth is available for 16-bit PCM at any time and if the redundancy packets really don't have a positive impact on latency as you've described it at the other PR, or even make the situation worse, then the 8-bit codec is really nonsense for a desktop environment under any circumstances. I just thought that the option for redundancy exists for a good reason and that it would help in any case 😄

I initially did some A/B tests over Wi-Fi, but now I'm not so sure anymore if the results were more placebo than actual results. Still I'm curious now and I'll do a real world latency test with the redundancy option enabled on Friday with a friend who lives about 200km away and report the results.

Now that I know that AOO is still under further development: Is some kind of forward error correction planned for AOO 2.0? I have some ideas in my mind which may be worth further investigation.

Regarding the noise shaping implementation

For an embedded device I wouldn't recommend my implementation with a 9 coefficient noise shaping function and instead go for the simpler one from Ardour / Audacity with only 4 coefficients to save some calculation power: https://github.com/Ardour/ardour/blob/f1f352a6e9948abec8259fe9362cc3da7a2fadf6/libs/audiographer/private/gdither/gdither.cc

The reason why I've chosen the 9-coefficient one was that, if I understood the original paper correctly, the actual perceptible noise for a human gets further minimised a teeny tiny bit with 9 coefficients over the 4 coefficients version and it's a desktop with enough power, so I thought - why not 😃

[Spacechild1]

While jamming it's 5 ms if you add both directions together

Are you sure you can add up the latencies like that? ;-)

I just thought that the option for redundancy exists for a good reason and that it would help in any case

I think I just took the idea from JackTrip, but I was always sceptical. There might be situations where it helps, but I just haven't seen any evidence yet :-)

Is some kind of forward error correction planned for AOO 2.0

I think Opus offers forward error correction. Personally, I won't attempt to roll my own audio codec, so AOO just piggy-backs on existing codecs. However, AOO has a codec plugin API, so you can actually add your own codecs! (The codecs can be added at compile time, or the host program may offer dynamic plugin loading.)

If you have more questions or ideas about AOO, here's the "official" repo: https://git.iem.at/cm/aoo/-/issues

(Regarding AOO development: I am trying to get a pre-release out this month. There are just a few items left on the TODO-list.)