nakedterew.blogg.se

1. #OPUS 5.1 CODEC DECODER RECEIVER HOW TO#
2. #OPUS 5.1 CODEC DECODER RECEIVER UPDATE#
3. #OPUS 5.1 CODEC DECODER RECEIVER CODE#

Were able to significantly improve stereo speech quality, especially in the 24-32 kb/s range. For 1.3, there's been some more tuning for low-bitrateįirst, it turns out that the Opus encoder was never actually tuned for stereo speech below 40 kb/s.īy just changing how the encoder divides the total bitrate between the SILK and CELT parts, we (combinations of SILK/CELT/hybrid, bandwidths, frame sizes, mono/stereo), it's easy to see why notĮverything was fully optimized when the 1.0 encoder came out. Combined with the fact that Opus has a total of 64 different operating modes

#OPUS 5.1 CODEC DECODER RECEIVER CODE#

We were already amazed we could code high-quality fullband speech at 32 kb/s that we didn't reallyĪttempt to go beyond that. When look-ahead is available, the decision logic is able to have the encoder switch modeĭuring a silence period before the transition between speech and music.īack in 2010-2012, when we were combining SILK and CELT into Opus, there was a lot of work to do. The encoder now minimizes a badness metric, which includes penaltiesįor using the wrong mode, but also for switching in the middle of speech or music. That decision logic has also been improved in 1.3. How it evolves over time), but also based on the bitrate and whether/when the audio is active The final 1.1 probability from the HMM (blue), and the probability computed by the 1.3 recurrent neural network (red).Ī neural network only gives us a probability that any given frame is speech or music.įrom there, we have to make an actual decision based not only on that probability (and The lower graph shows the instantaneous music probability produced by the 1.1 neural network (gray), Speech and music sections are marked on the waveform (top). Illustration of a test sample consisting of mixed voice and music alternating roughly every fifteen seconds. Although itĭoesn't look very deep, it's actually quite deep over time thanks to the GRU. The figure below shows how the layers of the network are arranged.

On a GPU, the Opus detector has just 4986 weights (that fit in less than 5 kB) and takes about 0.02% CPU Instead of thousands of neurons and millions of weights running

#OPUS 5.1 CODEC DECODER RECEIVER UPDATE#

When the update gate z is on the left, then the state can remainĬonstant over a long period of time - until a condition causes z to switch to the right.īecause distinguishing between speech and music is much easier than, say, speech recognition,

Both are soft switches (valueīetween 0 and 1) computed based on the previous state of the whole layer and the inputs, with a sigmoidĪctivation function. Gates, which make it possible to learn longer-term patterns. That allows them to combine previous information to the new information they receive. Remember information for a long period of time but also discard some of that information when appropriate.Ĭomparing the feedforward unit originally used to a simple recurrent unit and to a GRU.

#OPUS 5.1 CODEC DECODER RECEIVER HOW TO#

It not only learns how to use its inputĪnd memory at each time, but it can also learn how and when to update its memory. Unlike simpleįeedforward units, the GRU has a memory. New type of recurrent neuron: the Gated Recurrent Unit (GRU). Opus 1.3 includes a brand new speech/music detector.

Thanks toĭeep learning - and specifically recurrent neural networks - we can now Worked reasonably well most of the time, it was still far from perfect.

HMM-based layer to combine the neural network results over time. Speech or music, and choose the optimal encoding mode accordingly.ĭetector was based on a simple (non-recurrent) neural network, followed by an Here's a few of the upgrades that users and implementers willĮver since version 1.1, Opus has been able to automatically detect whether its input is This is also the first release withĪmbisonics support. To both speech and music, while remaining fully compatible with RFC 6716. Opus gets another major update with the release of version 1.3.