Bounds on Information Combining

Abstract

Abstract-When the same data sequence is transmitted over two independent channels, or when a data sequence is transmitted twice but independently over the same channel, the independent observations can be combined at the receiver side. From an information-theory point of view, the overall mutual information between the data sequence and the received sequences represents a combination of the mutual information of the two channels. This concept is termed information combining. In this paper, a lower bound and an upper bound on the combined information is presented, and it is proved that these bounds are tight. Furthermore, this principle is extended to the computation of extrinsic information on single code bits for a repetition code and for a single parity-check code of length three, respectively.For illustration of the concept and the bounds on information combining, two applications are considered. First, bounds on the information processing characteristic (IPC) of a parallel concatenated code are derived from its extrinsic information transfer (EXIT) chart. Second, bounds on the EXIT chart for an outer repetition code and for an outer single parity-check code of a serially concatenated coding scheme are computed.Index Terms-Extrinsic information, extrinsic information transfer (EXIT) chart, information combining, information processing characteristic (IPC), iterative decoding, mutual information, parallel concatenated codes.

Extremal Problems of Information Combining

“…As a special case, if all channels have the same distribution, the average extrinsic mutual information is (13) where the th moment is defined in (8).…”

“…Without loss of generality, one can focus on and ask the following question: can we find tight lower and upper bounds on the combined information , i.e., the extrinsic mutual information? Such an extremal problem was first considered in [12] for a repetition code. It was shown that is maximized (or minimized) when both and are BECs (or binary-symmetric channels (BSCs)) with prescribed mutual information values.…”

“…It was shown that is maximized (or minimized) when both and are BECs (or binary-symmetric channels (BSCs)) with prescribed mutual information values. In [13], the case of single parity-check codes was studied, and it was shown that BSCs achieve the upper bound, and BECs achieve the lower bound. In [14]- [17], the above results were extended to arbitrary codeword length for both repetition codes and single parity-check codes.…”

“…In [12]- [15], the extrinsic mutual information is optimized over all channels subject to their individual mutual information constraints. In [16], [17], the problems were generalized by optimizing the extrinsic mutual information with respect to a single channel.…”

“…Bounds obtained from the information combining problems are used to bound the VND and CND EXIT functions [13], [17]. In [17], the lower bounds on the extrinsic mutual information for the variable nodes and check nodes were combined to obtain the overall worst performance bound, which was used to find sufficient conditions on communication channels for successful decoding.…”