Advanced DSP

System Identification

AR deconvolution is as simple as MA deconvolution: we have to find the MA equivalent of the given AR system, which is just its impulse response. Same goes for ARMA deconvolution.

The problem of MA system identification is equivalent to MA deconvolution, if we interchange our understanding of filter and input signal. This we can do because of commutativity of convolution.

AR system identification is tough, if the goal is to minimize error in the output. If I pass a signal x through an AR system g, then x*g depends linearly on x but nonlinearly on g. This is because the MA equivalent of an AR system is a non-linear function. We do not know how to solve this non-linear optimization problem.

We derive a heuristic algorithm as follows. The inverse of a finite-coefficient AR system is a finite-coefficient MA system. So, we model the inverse system using MA system identification. This algorithm is heuristic, because we are minimizing errors in the input.

For AR systems, minimizing errors in the input is simple, minimizing errors in the output is impossible. For MA systems, minimizing errors in the output is simple, minimizing errors in the input is impossible. For ARMA systems, minimizing errors in both the input and output is a nonlinear optimization problem. Thus we go for heuristic algorithms.

Suppose we have a (p,q) order ARMA system, we can think of it as a cascaded system, x→s→y where s=x*f and y=s*g, where f is the MA part and g is the AR part of the filter. Let us assume that we gave an impulse input x=δ. We know that after the first p values, s becomes zero. q of these zeroes and the corresponding values of y, are used to find the values of g. After we get g, we invert it to get the nonzero values of s. These are the filter coefficients f. This method is called the Padé method.

The Padé method is not a least squares method. If we use a lot of data of y, using a lot of the s zeroes, and do least squares AR system identification for g, what we get is the Prony method.

The Prony method does least squares for the AR coefficients. Beyond this point, if we commute the system to look like x→s'→y. s'=x*g can be easily found now. f is then found using least squares MA system identification taking s' as the input and y as the output. This is the Shanks method.

What if we cannot assume impulse input? First we model the system as a large MA system. These MA coefficients are the impulse response which we can model using our ARMA models.

System identification uses LS deconvolution which uses LS matrix inversion. Levinson-Durbin is a fast algorithm for LS deconvolution, used in all of the algorithms for system identification. System identification is further used in autocorrelation estimation algorithms.