Amplitude precision is a simple specification that quantifies a channel’s amplitude error. The error is the difference between the nominal input amplitude and the channel’s measured amplitude.
If a signal at 1 V amplitude is considered to be 0 dB and the channel measures -0.05 dB, then the precision is within 0.05 dB. The amplitude precision depends on the front-end calibration, so it is possible to have the error in either direction, although it tends to be below the nominal amplitude in most cases.
Amplitude linearity is a way to specify the precision over a range of nominal input amplitudes. Precision typically depends on the input amplitude, and when just one number is provided for the amplitude precision specification, it is usually at 1 kHz and 1 V.
To fully capture the linearity specification, a graph could be provided, but this can be difficult to present in a format that is easy to interpret. Instead, a simple table is provided that displays a few different input ranges and the corresponding amplitude precision. The reason it is referred to as amplitude linearity is because the precision is not a straight line relative to the input amplitude.
| Amplitude Range (dBFS) | Amplitude Error |
|---|---|
| 0 to -80 | ±0.08 dB |
| -80 to -100 | ±0.25 dB |
| -100 to -120 | ±0.5 dB |
| -120 to -140 | ±1 dB |
The graph below shows what the specification would look like on a graph. The error has been exaggerated to make it easier to see the difference between nominal and measured. With the actual numbers, the difference would be small and difficult to see, which is another reason to present the specification in a table instead of a graph. Even with the difference exaggerated for visual effect, presenting the specification in a table is easier to quantify.
