Mutual Information Functions versus Correlation Functions

Wentian Li

90-024

This paper studies one application of mutual information to symbolic sequences: the mutual information function $M(d)$. This function is compared with the more frequently used correlation function $1(d)$. An exact relation between $M(d)$ and $1(d)$ is derived for binary sequences. For sequences with more than two symbols, no such general relation exists; in particular, $1(d)=0$ may or may not lead to $M(d) = 0$. This linear, but not general, independence between symbols separated by a distance is studied for ternary sequences. Also included is the estimation of the finite-size effect on calculating mutual information. Finally, the concept of "symbolic noise" is discussed.