Estimator bias: Systematic deviation from the true value, either consistently overestimating or underestimating the parameter of interest.
Estimator Bias: Biased or Unbiased
Consider a simple communication system model where a transmitter transmits continuous stream of data samples representing a constant value – ‘A’. The data samples sent via a communication channel gets added with White Gaussian Noise – ‘w[n]’ (with mean=0 and variance=1). The receiver receives the samples and its goal is to estimate the actual constant value (we will call it DC component hereafter) transmitted by the transmitter in the presence of noise. This is a classical DC estimation problem.
Since the constant DC component is embedded in noise, we need to come up with an estimator function to estimate the DC component from the received samples. The goal of our estimator function is to estimate the DC component so that the mean of the estimate should be equal to the actual DC value. This is the criteria for ascertaining the unbiased-ness of an estimator.
The following figure captures the difference between a biased estimator and an unbiased estimator.
Example for understanding estimator bias
Consider that we are presented with a set of N samples of data representing x[n] at the receiver. Let’s us take the signal model that represents the received data samples :
Consider two estimator models/functions to estimate the DC component from the received samples. We will see which of the two estimator functions gives us unbiased estimate.
Computing mean for Estimator 1:
Computing mean for Estimator 2:
Summary:
| Estimator function | \(\hat{A} = \displaystyle{\frac{1}{N} \sum_{n=0}^{N-1} x[n]}\) | \(\hat{A} = \displaystyle{\frac{1}{2N} \sum_{n=0}^{N-1} x[n]}\) |
| \(E(\hat{A})\) | \( = A\) | \( \begin{align} &= A; if \; A=0 \\ & \neq A; if \; A \neq 0 \end{align} \) |
| Bias | Unbiased | Biased |
Testing the bias of an estimation in Matlab:
To test the bias of the above mentioned estimators in Matlab, the signal model: \(x[n]=A+w[n]\) is taken as a starting point. Here \(A\) is a constant DC value (say for example it takes a value of 1.5) and w[n] is a vector of random noise that follows standard normal distribution with mean=0 and variance=1.
Generate 5000 signal samples \(x[n]\) by setting \(A=1.5\) and adding it with \(w[n]\) generated using Matlab’s “randn” function.
N=5000; %Number of samples for the test A=1.5 ;%Actual DC value w = randn(1,N); %Standard Normal Distribution mean=0,variance=1 represents noise x = A + w ; %Received signal samples
Implement the above mentioned estimator functions and display their estimated values
estA1 = sum(x)/N;% Estimated DC component from x[n] using estimator 1 estA2 = sum(x)/(2*N); % Estimated DC component from x[n] using estimator 2 %Display estimated values disp([‘Estimator 1: ’ num2str(estA1) ]); disp([‘Estimator 2: ’ num2str(estA2) ]);
Sample Result :
Estimator 1: 1.5185 % Estimator 1’s result will near exact value of 1.5 as N grows larger
Estimator 2: 0.75923 % Estimator 2’s result is biased as it is far away from the actual DC value
The above result just prints the estimated value. Since the estimated parameter – \(\hat{A}\) is a constant \(E(\hat{A}) = \hat{A}\). In real world scenario, the parameter that is estimated, will be a random variable. In that case you have to print the “expectation” (mean) of the estimated value for comparison.
See Also