A.13  Rectangular Integration to Define Normalization Factors for Functions

In several situations a computer is used to obtain points that should represent a continuous function $f(x)$, $x\in ℝ$. Two examples of this situation are the estimation of probability density functions (PDF) via histograms and power spectral density (PSD) estimation via an FFT routine.

Instead of aiming at an analytical expression $\widehat{f}(x)$ to represent $f(x)$, the task consists in obtaining a set of points $\{\widehat{f}(x[n])\}$ calculated at the values $x[n]$, $n=0,\dots ,N-1$, which are a uniformly-sampled version of the abscissa $x$.

Often it is possible to first obtain a set of values $\{ĝ(x[n])\}$ in which the value $ĝ(x[n])$ is proportional to $f(x[n])$, i. e., $f(x[n])\propto ĝ(x[n])$. In this case, it is required to later determine a scaling factor $\kappa \in ℝ$ such that the final set of values to represent $f(x)$ is obtained via

Note that the goal is not necessarily to have $\widehat{f}(x)\approx f(x)$. There are situations in which the set of points $\{\widehat{f}(x[n])\}$ must obey a property. For example, when histograms are used to estimate probability mass functions, one desired property is that ${\sum <!--\; FUNCTION\; APPLICATION\; -->}_{x[n]}\widehat{f}(x[n])=1$. Alternatively, the goal may be to scale the histogram such that the two resulting curves (normalized histogram and probability density function) coincide. The values of $\kappa$ are different for these two possible cases of histogram normalization as discussed after recalling the rectangle method.

The rectangle method [ urlBMrec] is used for approximating a definite integral:

where $h=(b-a)∕N$ is the rectangle width and $x[n]=a+\mathit{nh}$.

The rectangle method can be used, for instance, to relate the continuous-time convolution in Eq. (3.2) with its discrete-time counterpart in Eq. (3.1). Assuming $T_s$ is the sampling interval used to obtain the discrete-time signals $x[n]$ and $h[n]$ from $x(t)$ and $h(t)$, respectively, the factor $T_s$ is required to better approximate the samples of $y(t)=x(t)\ast h(t)$ when using a discrete-time convolution:

Besides, rectangle integration is useful to calculate the scaling factor $\kappa$ in the two cases discussed in next section.

A.13.1  Two normalizations for the histogram

When the task is to estimate the PDF $f(x)$ of a continuous random variable, one can try using a discrete histogram $ĝ(x[n])$, which is obtained by drawing $M$ values from $f(x)$ and counting the number of values occurring at each of $B$ bins. Intuitively, for large $M$ and $B$, the curve (or the “envelope”) of the histogram resembles $f(x)$ but it is off by a normalization factor $\kappa$.

If $\kappa =1∕M$ is chosen, which is the most adopted option, one has $\widehat{f}(x[n])=(1∕M)ĝ(x[n])$ and, consequently ${\sum <!--\; FUNCTION\; APPLICATION\; -->}_n\widehat{f}(x[n])=1$. However, in this case, $\widehat{f}(x[n])$ may be far from $f(x)$ by a large scaling factor. This can be observed in the curves generated by the following code:

1M=1000; x=3*rand(1,M); %M random numbers from 0 to 3 
2B=100; [hatgx,N_x]=hist(x,B); %histogram with B bins 
3hatfx = hatgx/M; %normalize the histogram to sum up to 1 
4plot(N_x,hatfx,[-1,0,0,3,3,4],[0,0,1/3,1/3,0,0],'o-') 
5xlabel('random variable x'), ylabel('PDF f(x)') 
6legend('estimated','theoretical'); sum(hatfx)

The result of sum(hatfx) is equal to one, as specified, but the PDF of the simulated distribution $U(0,3)$ is 1/3 over its support and the superimposed estimated and theoretical graphs do not match. This discrepancy between the curves should be expected given that the normalized histogram hatfx was in fact an estimate of a probability mass function (PMF) of a discrete random variable, obtained by quantizing the original $x$. Another normalization factor $\kappa \ne 1∕M$ must be used if the goal is to have $f(x)\approx \widehat{f}(x[n])$.

To obtain $\kappa$ such that $f(x)\approx \widehat{f}(x[n])$, one can use the property that the integral of a PDF is one. Based on the rectangle method one can write

where $h=(b-a)∕B$. Because ${\sum <!--\; FUNCTION\; APPLICATION\; -->}_{n=0}^{N-1}ĝ(x[n])=M$, one obtains $\kappa =1∕(\mathit{hM})$, which is the original factor $1∕M$ divided by the bin width $h$. The function ak_normalize_histogram.m uses this approach. Using the same example of the previous code, the following commands for obtaining hatfx would lead to consistent theoretical and estimated curves:

1M=1000; x=3*rand(1,M); %M random numbers from 0 to 3 
2B=100; [hatgx,N_x]=hist(x,B); %histogram with B bins 
3h=3/B; %h is the bin width assuming the support is 3 
4hatfx = hatgx/(M*h); %PDF values via normalized histogram 
5plot(N_x,hatfx,[-1,0,0,3,3,4],[0,0,1/3,1/3,0,0],'o-') 
6xlabel('random variable x'), ylabel('PDF f(x)') 
7legend('estimated','theoretical'); sum(hatfx)

As expected, in contrast to the sum equal to one in the first code, in this case sum(hatfx)=1/h=33.3. Both histogram normalization factors, $\kappa =1∕M$ and $\kappa =1∕(\mathit{hM})$, are useful and the choice depends whether the application requires values from a PMF or PDF, respectively.

A.13.2  Two normalizations for power distribution using FFT

Another application that can be related to Eq. (A.21) is the use of FFT for estimating how the signal power is distributed over frequency. It is assumed here a finite-duration discrete-time signal $x[n]$ with $N$ non-zero samples.

The squared FFT magnitude $|\mathrm{\text{FFT}}\{x[n]\}{|}^2$ plays the role of the function $ĝ$ in Eq. (A.29). The choice $\kappa =1∕N^2$ leads to an estimate $\widehat{f}(\cdot )$ of the mean-square spectrum (MSS) $S_{\mathrm{\text{ms}}}[k]$ of Eq. (4.33), while $\kappa =1∕(N^2\mathrm{\Delta }f)$ corresponds to PSD $S(f)$ in Eq. (4.21), where $\mathrm{\Delta }f=\mathrm{\text{BW}}∕N$ and $\mathrm{\text{BW}}$ is given in Hz. As indicated in Table A.1, the two options for $\kappa$ have similarities with the ones for histogram normalization.

In both cases in Table A.1, when going from a discrete to a continuous function, the bin width ($h$ for histogram and $\mathrm{\Delta }f$ for the FFT), is used as normalization factor.