Optimal transmit filters for constrained complexity channel shortening detectors

We consider intersymbol interference channels with reduced-complexity, mutual information optimized, channel shortening detectors. For a given channel and receiver complexity, we optimize the transmit filter to use. The cost function we consider is the (Shannon) achievable information rate of the entire transceiver system. By functional analysis, we can establish a general form of the optimal transmit filter, which can then be optimized by standard numerical methods. As a side result, we also obtain an insight of the behaviour of the standard waterfilling algorithm for intersymbol interference channels.