Fixes stack overflow for some custom modes

This only affects custom modes (builds with --enable-custom-modes) with frame
sizes 2.5, 5, 10, and 20ms and sampling rates below 40 kHz. The problem does
not affect normal use of Opus (using OpusEncoder/OpusDecoder) even when built
with custom modes enabled, but only special applications that use
OpusCustomEncoder/OpusCustomDecoder.
This commit is contained in:
Jean-Marc Valin 2023-08-11 04:06:52 -04:00 committed by Jean-Marc Valin
parent 41a4c9515d
commit a71ba10f89
No known key found for this signature in database
GPG key ID: 531A52533318F00A

View file

@ -1450,7 +1450,7 @@ void quant_all_bands(int encode, const CELTMode *m, int start, int end,
if (encode && resynth)
lowband_scratch = _lowband_scratch;
else
lowband_scratch = X_+M*eBands[m->nbEBands-1];
lowband_scratch = X_+M*eBands[m->effEBands-1];
ALLOC(X_save, resynth_alloc, celt_norm);
ALLOC(Y_save, resynth_alloc, celt_norm);
ALLOC(X_save2, resynth_alloc, celt_norm);