Audio Compression Technique for Low Delay and High Efficiency Using Complex Audio Data

1. Complex LPC based audio compression

To compress an audio signal, information reduction can be achieved by quantization process. Information reduction reduces the number of bits by performing the quantization with controlled quantized noise within a range of allowed distortion that is not perceptually degraded by humans. The quantization noise can be introduced in the frequency domain and the time domain. The quantization noise in the frequency domain can be controlled by using psychoacoustic models to perform information reduction for sound compression^[6], and by estimating spectral envelope information in the frequency domain using time-domain linear prediction coefficients^[7]. Time domain information volume estimation and quantization is mainly performed by adjusting the time domain analysis interval and changing an analysis window to a short interval. In particular, the amount of information in the time domain can be represented by the amount of signal variation over time, and if quantization is performed using a long analysis window in a section with a large amount of time variation, sound quality distortion such as pre-echo occurs^[8]. Therefore, to reduce time domain compression distortion, one alternative is to perform quantization with short analysis windows and frames. This conversion process requires future sample information for predictive processing to determine the analysis window and results in additional latency. It also reduces compression efficiency, as relatively more bits need to be allocated to short bins to minimize time base quantization distortion such as pre-echo. To improve this, a temporal noise shaping technique, Temporal Noise Shaping (TNS), has been introduced^[6]. TNS is a technique that utilizes linear prediction techniques in the frequency domain to reduce the amount of time base information. The scheme of linear prediction in frequency is based on the duality properties of the linear prediction coefficient (LPC), which enables the time envelope information in the time domain to be obtained from the linear prediction information in the frequency domain. However, the conventional technique based on the real transform has the disadvantage that the envelope in the time domain cannot be accurately predicted by performing linear prediction on the real frequency coefficients obtained by the MDCT conversion method due to incomplete phase information, and it still requires a short window and cannot significantly improve the latency^[8]. In this paper, the complex linear prediction technique adopted er is complex TNS (CTNS), which performs linear prediction in the complex frequency domain^[6]. Complex linear prediction is performed in the complex frequency domain. In general, an arbitrary signal can be represented in the time domain as an analytic signal from in the complex domain^[7].

Here, the signal x_a(n) can be represented as an analytic signal, including the imaginary part x_h(n), which is the Hilbert Transform of the original signal. The change in the amount of information on the time axis can be obtained from the envelope information |x(n)+jx_h(n)|, which is the absolute value of the analysed signal. Therefore, a prediction method that can accurately estimate the envelope information of x_a(n) is required, and to do this, it is necessary to apply CTNS technology that can perform filtering in the complex domain^[9]. Therefore, for CTNS implementation, linear prediction coefficients for complex filtering are required, and a complex number of LPCs are extracted from the DFT domain and used as filter coefficients. We define these as complex LPCs (CLPCs).

Fig. 1 shows a comparison of the time domain envelope estimation results of the adopted CTNS technique compared to conventional TNS. Since conventional TNS estimates the envelope based on the linear prediction coefficients obtained from MDCT, it cannot predict the exact envelope of the original signal due to the aliasing distortion that occurs in the time domain. In contrast, CTNS utilizing CLPC extracted from the complex domain shows that the envelope information of the predicted time base is a good estimate of the envelope of the original signal. When CTNS is applied to perform quantization, the complex linear prediction coefficient CLPC is also information that needs to be transmitted and converted into bit information through quantization^[9,10].

Fig. 1.

CTNS vs. TNS with Complex Linear Prediction Coefficients

Unlike the conventional LPCs, the complex linear prediction coefficients CLPCs do not have roots in the form of conjugate complex numbers. Therefore, a new quantization method is required for the conversion process so that they can be expressed correspondingly on the unit circle when expressed as P(z) and Q(z)^[9]. In this paper, a method of quantizing and transmitting the roots of the complex linear prediction coefficients directly is applied for the transmission technique of the complex linear prediction coefficients. The roots represented by the complex linear prediction coefficients all exist in a unit circle in the z-plane, and the magnitude and phase values of each root can be separated. Each set of the separated magnitude and phase vectors can be subjected to VQ (Vector Quantization). The VQ for each of the magnitude and phase vectors was performed hierarchically in two stages, with each stage designed to have a codebook of 10 bits for the 8th-order complex linear prediction coefficients. Thus, 40 bits of information about the complex linear prediction coefficients are transmitted per frame (4 stages are required for encoding absolute values and phases of CLPs). However, if the amount of temporal information does not change significantly, it is efficient to quantize the audio data without performing CTNS. For this purpose, the proposed compression technique is designed with a dual quantization structure, so that if the time domain information volume reduction is not required, the encoding is performed by MDCT conversion as before. A schematic block diagram is shown in Fig. 2.

Fig. 2.

Audio encoder/decoder Conceptual Block Diagram with Complex Linear Prediction Coefficients

First, let's briefly review the process of complex quantization: If the time domain information reduction is deemed effective and the quantization is performed in the complex domain, DFT is performed on the input signal x(b) with frame index ‘b’ for complex frequency domain conversion. Extract CLPC from x_f(b) with complex frequency coefficients and perform quantisation to obtain clpc_q(b) to be transmitted. Using clpc_q(b), perform filtering on the complex coefficients according to conventional LPC filtering to obtain the residual signal x_c,f(b). Finally, the complex residual signal x_c,f(b) is quantised to transmit the bit information about x_c,q(b). The method of quantising x_c,f(b) performs quantization on the real and imaginary parts with a single scale-factor based on the absolute values of the complex coefficients. The quantization method follows a similar approach to the conventional MPEG audio codec, with the difference that the quantization is performed by dividing the residual signal into multi-bands^[11]. Similarly, when MDCT is selected, the real part quantization method performed is the same as that of conventional MPEG audio codecs. The criterion for switching between the two modes is the predicted gain of the residual signal, which can be expressed in general terms as Eq (2).

In this paper, SNR (Signal-to-Noise Ratio) is used as the gain_metric, and complex domain coding is performed when the SNR difference is more than 6 dB regarding x_f(b) is considered as the original signal.

Fig. 3 shows the predicted gain of CLPC for frames actually selected with the complex value coding mode and the real value coding mode. In Fig. 3(a), it can be seen that the amount of information in the residual signal generated after applying CLPC with the complex value coding mode is significantly reduced compared with original ones. This is not only on the time domain in the upper part of (a), but also in the frequency domain plot in the lower part of (a). In Fig. 3(b), we can see that the prediction gain for CLPC is negligible. The dual-structure encoder is adopted to increase the compression efficiency, and the decoding process requires frame-to-frame compensation in the overlapping region due to the dual structure. In the case of MDCT, the phase information of the time signals is distorted, which can cause interference signals due to aliasing after inverse transform. To cancel the aliasing, MDCT information from the previous decoded frame is required. If a frame-to-frame coding scheme switch occurs during encoding, switching from a DFT complex value quantized frame to an MDCT quantized frame, the decoding process can artificially synthesize the signal of the previous MDCT quantized signals using the time-axis information obtained from the DFT complex coefficients.

Fig. 3.

CLPC Predicted Gain Example with respect to quantization mode

This synthesized signal serves to cancel out the aliasing, and the aliasing can be compensated for by an overlap-add operation. This artificially synthesized MDCT aliasing correction signal is defined as Synthesized Forward Aliasing Cancellation (S-FAC) in Fig. 2. This refers to the block that performs the operation of superposition summation of the S-FAC signal with respect to the MDCT-based reconstructed real signal. By default, both the complex signal and real signal reconstruction schemes perform a 50% overlap summation operation, which is intended to increase the quantization efficiency in the frequency domain by using a symmetrically complete analysis and synthesis window^[12].

2. Channel coding on complex domain

Most audio content is stereo by default, and to increase the efficiency of sound compression techniques, channel coding scheme must be applied to the audio channel signals. Channel coding based on the information reduction between audio channels reduces redundant information due to the correlation between the channels. The basic principle of the channel coding is to convert the highly correlated part between two channels into a principal component channel, and the remaining signal excluding the principal component into a minor component channel. The principal component channel signal is normally called the mid signal, and the remained component channel signal is called the side signal. A simple way to obtain the principal component and subcomponents from the channels is to obtain the signals of the sum and difference between the two channel signals^[13]. The sum signal is defined as the mid signal and the difference signal as the side signal, and bits are allocated according to the size of the information amount, and quantization is performed within the allocated bits for each channel. Therefore, the larger the difference between the information amount of the main and side signals, the more efficient the bit allocation is achieved, which can increase the encoding efficiency. The simplest and most common stereo channel encoding technique is the M/S (mid side) encoding technique. M/S encoding is a very simple channel encoding technique, but its encoding efficiency is not high. Therefore, a more dynamic and active conversion method for the main and sub signals is required, and the MPS (MPEG Surround) channel encoding technique meets this requirement^[14]. However, the MPS technique applies a QMF (Quadrature Mirror Filter) frequency conversion method for dynamic main/sub conversion. The QMF conversion performs a filtering operation, which requires additional analysis time and introduces significant latency.

In this paper, we apply a spatial cue-based channel encoding technique similar to MPS, and its conceptual block diagram is shown in Fig. 4. Basically, the role of unmixing the main/sub signals using spatial cues is the same as that of MPS, but the proposed method is characterized by obtaining the spatial cues in the DFT domain and then applying a unmixing matrix (unmix matrix) to generate the main/sub channels based on them. The unmixing matrix is applied in the complex or real frequency domain depending on the coding scheme. The spatial cues required to generate this unmixing matrix are the Channel Level Difference (CLD) and Inter-Channel Coherence (ICC), which are used to generate the unmixing matrix in the same way as MPS. However, the proposed audio compression technique already includes a quantization process in the complex transform domain, which has the structural advantage of facilitating the extraction of spatial cues from complex data. Thus, the unmixing matrix for channel coding can be constructed by analyzing the spatial cues without any additional delay. An example of the predicted gain of the main/side signals obtained through the dynamic unmix matrix is shown in Fig. 5, where it can be observed that the dynamic unmix matrix-based side signal (‘dynamic matrix side’) of the proposed technique has a larger information difference compared to the main signal (‘mid’), regarding the side signal (‘M/S side’) of the passive M/S method performed in the conventional MDCT domain. This difference between the main and sub-channel signals can be exploited to achieve improved encoding efficiency and sound quality by flexibly adjusting the bit allocation through joint quantization. The key strategy of a common quantizer described in Fig. 2 is to allocate a relatively large number of bits to the major component channel (‘mid’) for quantization process, and to use the residual bits for the residual component (‘side’). In this paper, we apply an eight-stage bit allocation scheme that selects the bit allocation mode based on the amount of information difference between the mid and side signals.

Fig. 4.

Block diagram of complex data-based channel encoding

Fig. 5.

M/S comparison according to the dynamic unmixing matrix

1. Latency performance evaluation

The latency of an audio coding scheme is calculated as the sum of the delays incurred during encoding and decoding. Latency in signal processing-based audio compression techniques can be calculated as a fixed value by arithmetically calculating the delay when applying the algorithm. The latency of an algorithm is calculated by summing the latency of each tool and algorithm that perform the compression. The main reason of latency is the size of the sample buffer required to perform the algorithm, which is essentially reflected in the latency. Also, if the algorithm requires future samples to perform compression, the corresponding buffer size for future samples should be reflected in the latency. The case of requiring future samples is mainly related to the signal classifier used when choosing the optimal combination of tools to increase the compression efficiency. The proposed coding compression technique performs the encoding within a single fixed frame without any classifier. The analysis for selecting the quantization scheme is also based on the predicted gain within a given current frame, which does not introduce any additional delay. The channel encoding process also performs the analysis in the same complex frequency domain, which does not need any additional delay.

Table 1 shows the processing latencies occurred during the encoding and decoding of the proposed acoustic compression technique. To increase the compression efficiency, the internal sampling frequency was converted to 38 kHz, which adds an additional 0.84 msec of latency due to the filtering operation for down-sampling. The encoding of each frame is a fixed-length sample frame, which is quantized to reduce the amount of time and frequency information to increase the compression efficiency. Based on a 38 kHz internal sampling frequency, the frame latency by the encoder was 26.9 msec, using a fixed frame length. In addition, an output superposition overlap delay of 13.5 msec was performed to eliminate time base aliasing. An additional 6.8 msec was needed as a future samples analysis during the time domain linear prediction process for performing Frequency Domain Noise Shaping (FDNS)^[7]. The CLPC performance is only within the transform block, so there is no additional latency. The final overall latency is 48.1 msec. An example of the latency difference between the decoded signal and the original signal is shown in Fig. 6. As a result, based on the subjective evaluation in the next section, it can be observed that at least 50 msec latency is achieved at 96 kbps while maintaining high-quality performance and compression efficiency equivalent to MOS 4.5.

Table 1.

Total delay samples and its configuration for proposed system

Fig. 6.

Example of proposed codec delay

2. Subjective listening Evaluation

Subjective listening experiments were conducted to evaluate the performance of the proposed audio compression encoder with low latency and high efficiency. Since the performance of audio encoders is measured by subjective quality evaluation based on bit rate, it is necessary to adopt an evaluation method that allows objective statistical analysis. We selected MUSHRA (Multiple Stimuli with Hidden Reference and Anchor) as a suitable evaluation method to assess subjective quality in a blinded manner on a scale from 0 to 100^[15]. The MUSHRA evaluation includes a hidden original signals and a low-quality anchor signals to verify the reliability of the subjects' ratings, and data from subjects whose ratings did not meet the reliability criteria were excluded from the final statistical analysis. In the end, the evaluation results of 10 subjects were selected. Five systems were included in the subjective test. The proposed system (‘sys_A’) compressed and reconstructed a 95.25 kbps stereo signal, and the comparison system, MPEG-H 3D audio (‘sys_B’) operates at 128 kbps for stereo signals and has a latency of a few hundred msec. A blind original signal (‘org’) are also included, an anchor signal (‘lp70’) with a 7 kHz band, and an anchor signal (‘lp35’) with a 3.5 kHz band are included as the test system.

Fig. 7 shows the subjective evaluation results, with scores for the test items and overall mean, including 95% confidence intervals. The detailed values of the mean scores and confidence intervals are shown in Table 2. The experimental results show that the final total average value of the proposed system is above 90 points considering the 95% confidence interval, and the reference model, 3DA 128 kbps, also shows a performance above 90 points, which corresponds to approximately MOS 4.5, considering the confidence interval. In particular, the confidence interval of the subjective evaluation average score of the proposed system and the 3DA system overlaps, which can be observed that the 96 kbps of the proposed technology and the 3DA 128 kbps of the reference system are statistically equivalent performances.

Fig. 7.

Subjective listening test results

Table 2.

MUSHRA Mean score with confidence intervals(CI) for subjective test