Illustrating the tradeoff between balance and calibration

using a fake example of anonymous Wikipedia editors

by Nate TeBlunthuis

I wrote here about the bias encoded into the ORES models deployed on Wikipedia for helping editors to monitor changes to the encyclopedia. There I showed that the models were unfair to newcomers and anonymous editors using two different notions of fairness: balance and calibration. I brought up the fact that there is an inherent tradeoff between these two quantified notions of fairness such that in non-trivial situations it is impossible to satisfy them both. Here, I'm going to illustrate this point with a simple simulation and show how a straightfoward approach to creating a balanced model from an imbalanced one results in a model which is not calibrated.

Let's say that whether an edit is damaging is a stochastic function of two observable variables: whether the editor is anonymous and X, which stands for everything else we can observe and include in our model. We'll say the linear probit model with these two variables is the true model.

Next I'll fit a model to the generated data and generate model predictions

The true model should be calibrated, but not balanced. Let's verify that is the case.

So we see that it is calibrated, but is it balanced?

Not even close! The model is super unbalanced. Non-damaging anonymous edits have almost the same average score as damaging non-anonymous edits!

Some people think that you can sovle algorithmic bias problems by using feature engineering and ignoring protected classes. There are some merits to this approach, but it doesn't help solve the balance vs calibration tradeoff. To illustrate this point, let's fit another model that only uses X and ignores anons.

The model is still imbalanced! But that did seem to make things a little bit better. Is the model still calibrated?

No it's really not calibrated now! So ignoring anons makes a choice about the tradeoff between balance and calibration, but it does so in an arbitrary way that depends on myriad factors including the correlation between anonymous editing and X.

A better approach to creating a balanced model comes from Hardt et al. (2016). Since the point where the ROC curves for the two protected classes intersect corresponds to choices of threshholds with equal false positve and negative rates, you can transform a good predictor to a worse predictor that is balanced by using different threshholds for different types of editors.

Plot the ROC curves.

So it looks like we can find balance with the FPR is around 0.2

Let's make new predictions and check balance and calibration. Note that now our threshhold for classifying an edit as damaging is much higher for anons than for non-anons.

Check that our new predictor is balanced

Using different threshholds for the different classes gives us a nearly balanced classifier!
The next question is if the balanced predictor is calibrated. What do you expect?

Nope! Not balanced. The predicted rate of vandalism for anons is lower than the true rate and for non-anons the predicted rate of vandalism is greater than the true rate. Finally, we can visualize the difference between calibration and balance. I'm going to do this using a sample of points, color them according to whether they are false positive, false negative, true positive, or true negative, and then show how predictions change between the calibrated and balanced predictors.

What can we see from this plot that balancing the model reduces the overall accuracy as it introduces more false classifications than true ones. Specifically, in order to correct a small handful of false positives for anons, we introduce even more false negatives. Similarly, to correct a handful of false negatives for non-anons, we accept even more false positives. Such sacrifices must be made to achieve balance.

Finally, we can use a plot to illustrate that balance means that, within the groups of damaging or non-damaging edits, the model predicts damage with equal probabilities for anonymous and non-anonymous edits.

How much accuracy did we lose by making our model balanced? Of course, this will depend on the particulars of how I simulated the data.

Accuracy is just one measure of model fitness, so let's also take a look at overall precision and recall.

Even though in this simulated data anons were three times as likely to make damaging edits compared to non-anons, balancing the model only costs 5 percentage points of accuracy. Moreover, the balanced model has better accuracy than the model that ignores that anons exist! And similarly, we see that balancing the model results in a substantial hit to precision and recall, but ignoring the very informative information that the editor is anonymous makes things even worse!

Choosing the point where the ROC curves for the two groups intersects is a good way to choose threshholds that will balance the model, but this comes at the cost of calibration. Removing the anon variable from the model is a way to compromise between balance and fairness, but potentially at the cost of accuracy (in this exercise, the cost in accuracy was quite high, but if we increase the rate of X enough it will not matter much).

However, total balance and total calibration can be thought of as boundaries that define the space of possible tradeoffs between these two different notions of fairness. Wikipedians might want to make a principled compromise between balance and calibration. One way to do this can be to choose different threshholds for anons and for non-anons that may not accomplish total balance, but that can preserve more calibration. There are also good approaches based on adding constraints (e.g. using KKT conditions) to the model that carefully penalize deviations from balance and calibration.