BLUE estimator

Why BLUE : We have discussed Minimum Variance Unbiased Estimator (MVUE)   in one of the previous articles. Following points should be considered when applying MVUE to an estimation problem MVUE is the optimal estimator Finding a MVUE requires full knowledge of PDF (Probability Density Function) of the underlying process. Even if the PDF is known, … Read more

Linear Models – Least Squares Estimator (LSE)

Key focus: Understand step by step, the least squares estimator for parameter estimation. Hands-on example to fit a curve using least squares estimation Background: The various estimation concepts/techniques like Maximum Likelihood Estimation (MLE), Minimum Variance Unbiased Estimation (MVUE), Best Linear Unbiased Estimator (BLUE) – all falling under the umbrella of classical estimation – require assumptions/knowledge … Read more

AutoCorrelation (Correlogram) and persistence – Time series analysis

The agenda for the subsequent series of articles is to introduce the idea of autocorrelation, AutoCorrelation Function (ACF), Partial AutoCorrelation Function (PACF) , using ACF and PACF in system identification. Introduction Given time series data (stock market data, sunspot numbers over a period of years, signal samples received over a communication channel etc.,), successive values … Read more

Yule Walker Estimation and simulation in Matlab

If a time series data is assumed to be following an Auto-Regressive (AR(N)) model of given form, the natural tendency is to estimate the model parameters a1,a2,…,aN. Least squares method can be applied here to estimate the model parameters but the computations become cumbersome as the order N increases. Fortunately, the AR model co-efficients can … Read more

Solving ARMA model – minimization of squared error

Linear-Time-Invariant-System-LTI-system-model

Key focus: Can a unique solution exists when solving ARMA (Auto Regressive Moving Average) model ? Apply minimization of squared error to find out. As discussed in the previous post, the ARMA model is a generalized model that is a mix of both AR and MA model. Given a signal x[n], AR model is easiest … Read more

Understand AR, MA and ARMA models

Key focus: AR, MA & ARMA models express the nature of transfer function of LTI system. Understand the basic idea behind those models & know their frequency responses. Introduction Signal models are used to analyze stationary univariate time series. The goal of signal modeling is to estimate the process from which the desired signal is … Read more

Cramér-Rao Lower Bound (CRLB)-Vector Parameter Estimation

Key focus: Applying Cramér-Rao Lower Bound (CRLB) for vector parameter estimation. Know about covariance matrix, Fisher information matrix & CRLB matrix. CRLB for Vector Parameter Estimation CRLB for scalar parameter estimation was discussed in previous posts. The same concept is extended to vector parameter estimation. Consider a set of deterministic parameters that we wish to … Read more

Introducing The Kalman Filter

Introducing The Kalman Filter – Ramsey Faragher PDF Text: click here PDF Text: click here Note: Click the playlist icon (located at the top left corner of the video frame) to watch all lectures Video Lectures: Watch, Listen and Learn !!! † Link will take you to external sites Disclaimer: All the materials posted in … Read more

Methods to compute linear convolution

Mathematical details of convolution, its relationship to polynomial multiplication and the application of Toeplitz matrices in computing linear convolution are discussed in the previous article. A short survey of different techniques to compute discrete linear convolution (with Matlab code) is given here. Definition Given an LTI (Linear Time Invariant) system with impulse response \(h[n]\) and … Read more

Convolution: understand the mathematics

Convolution operation is ubiquitous in signal processing applications. The mathematics of convolution is strongly rooted in operation on polynomials. The intent of this text is to enhance the understanding on mathematical details of convolution. Polynomial functions: Polynomial functions are expressions consisting of sum of terms, where each term includes one or more variables raised to … Read more