Grasping Bessel-Thomson scope response

In the heydays of analogue scopes, you pretty much had only one choice for frequency response—Gaussian. That came about because of the natural way in which the behaviour of multiple components with similar responses (even if they are not individually Gaussian) aggregates up. There didn't seem to be a strong reason to tamper with that response shape by deliberately inserting a filter somewhere.

The world of DSP has brought with it, among other things, the ability to easily tailor a response to a specific shape that may not necessarily be Gaussian. One obvious choice that has emerged is the flat response—something that is a better approximation to the textbook "brickwall" and that was hard to get to using analogue circuitry. A flat response oscilloscope gives you better results when you want to accurately measure a signal's rise time. However, more generally speaking, a flat response may not necessarily be an optimal choice.

Many textbooks actually refer to the brickwall as the "ideal" filter response, but that's more than a bit misleading. If your primary object of interest is not the rise time, a flat response or even a brickwall is actually not a good thing. For example, certain measurements require there to be minimal or no overshoot; never mind if the oscilloscope's agility does not quite match up to the signal rise time.

A case in point is the use of the Eye Diagram, especially to measure a transmitter's extinction ratio. The lines in an ideal eye diagram would be sharp, but in the real world they are fuzzy. Technically, they are statistical histograms—little bell-shaped distributions in their own right. A measurement procedure needs to find the centre of each of these histograms to determine, for example, the extinction ratio. Reducing the fuzziness of the lines—i.e., sharpening the histogram—is crucial to getting a good measurement.

Eye diagrams are insensitive to small variations of the 3dB bandwidth, but sensitive to small variations in the shape of the frequency response of the receiver. A response that allows some overshoot or ringing will immediately broaden the histogram. Enter the Bessel filter, also called a Bessel-Thomson filter. These filters are pretty far removed from the brickwall shape—in fact, as the diagram below shows, they exhibit the most offensively gentle roll-off compared to Butterworth, Chebyshev, and so on, and you can quite gently slide off the passband without hurting your rear.

Adding insult to injury is the fact that the droop starts well before the specified cut-off frequency, so it loses the flatness battle with the Butterworth filter.

What sets the Bessel-Thomson filters apart is their almost constant group delay (almost linear phase), which means no overshoot. So the use of a fourth-order Bessel-Thomson response, unlike a flat response, keeps the histogram sharp. This allows better determination of the true mean of the histogram.