The gain error is defined as the deviation of the last output step’s midpoint from the ideal straight line, after compensating for offset error.
After compensating for offset errors, applying an input voltage of 0 always give an output value of 0. However, gain errors cause the actual transfer function slope to deviate from the ideal slope. This gain error can be measured and compensated for by scaling the output values.
Run-time compensation often uses integer arithmetic, since floating point calculation takes too long to perform. Therefore, to get the best possible precision, the slope deviation should be measured as far from 0 as possible. The larger the values, the better precision you get. This is described in detail later in this document.
The example of a 3-bit ADC transfer functions with gain errors is shown in the Figure 1. The following description holds for both single-ended and differential modes.
To measure the gain error, the input value is increased from 0 until the last output step is reached. The scaling factor for gain compensation equals the ideal output value for the midpoint of the last step divided by the actual value of the step.
In the e Figure 1 (A), the output value saturates before the input voltage reaches its maximum. The vertical arrow shows the midpoint of the last output step. The ideal output value at this input voltage should be 5.5, and the scaling factor equals 5.5 divided by 7. In the e Figure 1 (B), the output value has only reached 6 when the input voltage is at its maximum. This results in a negative deviation for the actual transfer function. The ideal output value for the midpoint of the last step is 7.5 in this case. The scaling factor now equals 7.5 divided by 6. The measurement procedure is illustrated in the flowchart below.
