Fairness (machine learning)

Fairness in machine learning refers to the various attempts at correcting algorithmic bias in automated decision processes based on machine learning models. Decisions made by computers after a machine-learning process may be considered unfair if they were based on variables considered sensitive. Examples of these kinds of variable include gender, ethnicity, sexual orientation, disability and more. As it is the case with many ethical concepts, definitions of fairness and bias are always controversial. In general, fairness and bias are considered relevant when the decision process impacts people's lives. In machine learning, the problem of algorithmic bias is well known and well studied. Outcomes may be skewed by a range of factors and thus might be considered unfair with respect to certain groups or individuals. An example would be the way social media sites deliver personalized news to consumers.

Context

Research about fairness in machine learning is a relatively recent topic. In 2018, a majority of papers on the topic had been published in the preceding three years.[1] That same year, IBM introduced AI Fairness 360, a Python library with several algorithms to reduce software bias and increase its fairness[2][3] and Facebook made public their use of a tool, Fairness Flow, to detect bias in their AI. However, the source code of the tool is not accessible.[4] In 2019, Google published a set of tools in GitHub to study the effects of fairness in the long run.[5]

Controversies

The use of algorithmic decision making in the legal system has been a notable area of use under scrutiny. In 2014, then U.S. Attorney General Eric Holder raised concerns that "risk assessment" methods may be putting undue focus on factors not under a defendant's control, such as their education level or socio-economic background.[6] An investigative report published by ProPublica in 2016 claimed that the COMPAS software, widely used in US courts to predict recidivism, was racially biased. The report claimed that black defendants were almost twice as likely to be incorrectly labelled as higher risk than white defendants, while making the opposite mistake with white defendants.[7] Racial and gender bias has also been noted in image recognition algorithms. Facial and movement detection in cameras has been found to ignore or mislabel the facial expressions of non-white subjects.[8] In 2015, the automatic tagging feature in both Flickr and Google Photos was found to label black people with tags such as "animal" and "gorilla".[9] A 2016 international beauty contest judged by an AI algorithm was found to be biased towards individuals with lighter skin, likely due to bias in training data.[10] A study of three commercial gender classification algorithms in 2018 found that all three algorithms were generally most accurate when classifying light-skinned males and worst when classifying dark-skinned females.[11]

Group Fairness criteria

In classification problems, an algorithm learns a function to predict a discrete characteristic ${\textstyle Y}$, the target variable, from known characteristics ${\textstyle X}$. We model ${\textstyle A}$ as a discrete random variable which encodes some characteristics contained or implicitly encoded in ${\textstyle X}$ that we consider as sensitive characteristics (gender, ethnicity, sexual orientation, etc.). We finally denote by ${\textstyle R}$ the prediction of the classifier. Now let us define three main criteria to evaluate if a given classifier is fair, that is if its predictions are not influenced by some of these sensitive variables.[12]

Independence

We say the random variables ${\textstyle (R,A)}$ satisfy independence if the sensitive characteristics ${\textstyle A}$ are statistically independent of the prediction ${\textstyle R}$, and we write

${\displaystyle R\bot A.}$
We can also express this notion with the following formula:
${\displaystyle P(R=r\ |\ A=a)=P(R=r\ |\ A=b)\quad \forall r\in R\quad \forall a,b\in A}$
This means that the classification rate for each target classes is equal for people belonging to different groups with respect to sensitive characteristics ${\displaystyle A}$.

Yet another equivalent expression for independence can be given using the concept of mutual information between random variables, defined as

${\displaystyle I(X,Y)=H(X)+H(Y)-H(X,Y)}$
In this formula, ${\textstyle H(X)}$ is the entropy of the random variable ${\displaystyle X}$. Then ${\textstyle (R,A)}$ satisfy independence if ${\textstyle I(R,A)=0}$.

A possible relaxation of the independence definition include introducing a positive slack ${\textstyle \epsilon >0}$ and is given by the formula:

${\displaystyle P(R=r\ |\ A=a)\geq P(R=r\ |\ A=b)-\epsilon \quad \forall r\in R\quad \forall a,b\in A}$

Finally, another possible relaxation is to require ${\textstyle I(R,A)\leq \epsilon }$.

Separation

We say the random variables ${\textstyle (R,A,Y)}$ satisfy separation if the sensitive characteristics ${\textstyle A}$ are statistically independent of the prediction ${\textstyle R}$ given the target value ${\textstyle Y}$, and we write

${\displaystyle R\bot A\ |\ Y.}$
We can also express this notion with the following formula:
${\displaystyle P(R=r\ |\ Y=q,A=a)=P(R=r\ |\ Y=q,A=b)\quad \forall r\in R\quad q\in Y\quad \forall a,b\in A}$
This means that all the dependence of the decision ${\displaystyle R}$ on the sensitive attribute ${\displaystyle A}$ must be justified by the actual dependence of the true target variable ${\displaystyle Y}$.

Another equivalent expression, in the case of a binary target rate, is that the true positive rate and the false positive rate are equal (and therefore the false negative rate and the true negative rate are equal) for every value of the sensitive characteristics:

${\displaystyle P(R=1\ |\ Y=1,A=a)=P(R=1\ |\ Y=1,A=b)\quad \forall a,b\in A}$
${\displaystyle P(R=1\ |\ Y=0,A=a)=P(R=1\ |\ Y=0,A=b)\quad \forall a,b\in A}$

A possible relaxation of the given definitions is to allow the value for the difference between rates to be a positive number lower than a given slack ${\textstyle \epsilon >0}$, rather than equal to zero.

In some fields separation (separation coefficient) in a confusion matrix is a measure of the distance (at a given level of the probability score) between the predicted cumulative percent negative and predicted cumulative percent positive.

The greater this separation coefficient is at a given score value, the more effective the model is at differentiating between the set of positives and negatives at a particular probability cut-off. According to Mayes:[13] "It is often observed in the credit industry that the selection of validation measures depends on the modeling approach. For example, if modeling procedure is parametric or semi-parametric, the two-sample K-S test is often used. If the model is derived by heuristic or iterative search methods, the measure of model performance is usually divergence. A third option is the coefficient of separation...The coefficient of separation, compared to the other two methods, seems to be most reasonable as a measure for model performance because it reflects the separation pattern of a model."

Sufficiency

We say the random variables ${\textstyle (R,A,Y)}$ satisfy sufficiency if the sensitive characteristics ${\textstyle A}$ are statistically independent of the target value ${\textstyle Y}$ given the prediction ${\textstyle R}$, and we write

${\displaystyle Y\bot A\ |\ R.}$
We can also express this notion with the following formula:
${\displaystyle P(Y=q\ |\ R=r,A=a)=P(Y=q\ |\ R=r,A=b)\quad \forall q\in Y\quad r\in R\quad \forall a,b\in A}$
This means that the probability of actually being in each of the groups is equal for two individuals with different sensitive characteristics given that they were predicted to belong to the same group.

Relationships between definitions

Finally, we sum up some of the main results that relate the three definitions given above:

• Assuming ${\textstyle Y}$ is binary, if ${\textstyle A}$ and ${\textstyle Y}$ are not statistically independent, and ${\textstyle R}$ and ${\textstyle Y}$ are not statistically independent either, then independence and separation cannot both hold.
• If ${\textstyle (R,A,Y)}$ as a joint distribution has positive probability for all its possible values and ${\textstyle A}$ and ${\textstyle Y}$ are not statistically independent, then separation and sufficiency cannot both hold.

Mathematical formulation of group fairness definitions

Preliminary definitions

Most statistical measures of fairness rely on different metrics, so we will start by defining them. When working with a binary classifier, both the predicted and the actual classes can take two values: positive and negative. Now let us start explaining the different possible relations between predicted and actual outcome:[14]

Confusion matrix
• True positive (TP): The case where both the predicted and the actual outcome are in a positive class.
• True negative (TN): The case where both the predicted outcome and the actual outcome are assigned to the negative class.
• False positive (FP): A case predicted to befall into a positive class assigned in the actual outcome is to the negative one.
• False negative (FN): A case predicted to be in the negative class with an actual outcome is in the positive one.

These relations can be easily represented with a confusion matrix, a table that describes the accuracy of a classification model. In this matrix, columns and rows represent instances of the predicted and the actual cases, respectively.

By using these relations, we can define multiple metrics which can be later used to measure the fairness of an algorithm:

• Positive predicted value (PPV): the fraction of positive cases which were correctly predicted out of all the positive predictions. It is usually referred to as precision, and represents the probability of a correct positive prediction. It is given by the following formula:

${\displaystyle PPV=P(actual=+\ |\ prediction=+)={\frac {TP}{TP+FP}}}$

• False discovery rate (FDR): the fraction of positive predictions which were actually negative out of all the positive predictions. It represents the probability of an erroneous positive prediction, and it is given by the following formula:

${\displaystyle FDR=P(actual=-\ |\ prediction=+)={\frac {FP}{TP+FP}}}$

• Negative predicted value (NPV): the fraction of negative cases which were correctly predicted out of all the negative predictions. It represents the probability of a correct negative prediction, and it is given by the following formula:

${\displaystyle NPV=P(actual=-\ |\ prediction=-)={\frac {TN}{TN+FN}}}$

• False omission rate (FOR): the fraction of negative predictions which were actually positive out of all the negative predictions. It represents the probability of an erroneous negative prediction, and it is given by the following formula:

${\displaystyle FOR=P(actual=+\ |\ prediction=-)={\frac {FN}{TN+FN}}}$

• True positive rate (TPR): the fraction of positive cases which were correctly predicted out of all the positive cases. It is usually referred to as sensitivity or recall, and it represents the probability of the positive subjects to be classified correctly as such. It is given by the formula:

${\displaystyle TPR=P(prediction=+\ |\ actual=+)={\frac {TP}{TP+FN}}}$

• False negative rate (FNR): the fraction of positive cases which were incorrectly predicted to be negative out of all the positive cases. It represents the probability of the positive subjects to be classified incorrectly as negative ones, and it is given by the formula:

${\displaystyle FNR=P(prediction=-\ |\ actual=+)={\frac {FN}{TP+FN}}}$

• True negative rate (TNR): the fraction of negative cases which were correctly predicted out of all the negative cases. It represents the probability of the negative subjects to be classified correctly as such, and it is given by the formula:

${\displaystyle TNR=P(prediction=-\ |\ actual=-)={\frac {TN}{TN+FP}}}$

• False positive rate (FPR): the fraction of negative cases which were incorrectly predicted to be positive out of all the negative cases. It represents the probability of the negative subjects to be classified incorrectly as positive ones, and it is given by the formula:

${\displaystyle FPR=P(prediction=+\ |\ actual=-)={\frac {FP}{TN+FP}}}$

Relationship between fairness criteria as shown in Barocas et al.[12]

The following criteria can be understood as measures of the three general definitions given at the beginning of this section, namely Independence, Separation and Sufficiency. In the table[12] to the right, we can see the relationships between them.

To define these measures specifically, we will divide them into three big groups as done in Verma et al.:[14] definitions based on a predicted outcome, on predicted and actual outcomes, and definitions based on predicted probabilities and the actual outcome.

We will be working with a binary classifier and the following notation: ${\textstyle S}$ refers to the score given by the classifier, which is the probability of a certain subject to be in the positive or the negative class. ${\textstyle R}$ represents the final classification predicted by the algorithm, and its value is usually derived from ${\textstyle S}$, for example will be positive when ${\textstyle S}$ is above a certain threshold. ${\textstyle Y}$ represents the actual outcome, that is, the real classification of the individual and, finally, ${\textstyle A}$ denotes the sensitive attributes of the subjects.

Definitions based on predicted outcome

The definitions in this section focus on a predicted outcome ${\textstyle R}$ for various distributions of subjects. They are the simplest and most intuitive notions of fairness.

• Demographic parity, also referred to as statistical parity, acceptance rate parity and benchmarking. A classifier satisfies this definition if the subjects in the protected and unprotected groups have equal probability of being assigned to the positive predicted class. This is, if the following formula is satisfied:

${\displaystyle P(R=+\ |\ A=a)=P(R=+\ |\ A=b)\quad \forall a,b\in A}$

• Conditional statistical parity. Basically consists in the definition above, but restricted only to a subset of the instances. In mathematical notation this would be:

${\displaystyle P(R=+\ |\ L=l,A=a)=P(R=+\ |\ L=l,A=b)\quad \forall a,b\in A\quad \forall l\in L}$

Definitions based on predicted and actual outcomes

These definitions not only considers the predicted outcome ${\textstyle R}$ but also compare it to the actual outcome ${\textstyle Y}$.

• Predictive parity, also referred to as outcome test. A classifier satisfies this definition if the subjects in the protected and unprotected groups have equal PPV. This is, if the following formula is satisfied:

${\displaystyle P(Y=+\ |\ R=+,A=a)=P(Y=+\ |\ R=+,A=b)\quad \forall a,b\in A}$

Mathematically, if a classifier has equal PPV for both groups, it will also have equal FDR, satisfying the formula:

${\displaystyle P(Y=-\ |\ R=+,A=a)=P(Y=-\ |\ R=+,A=b)\quad \forall a,b\in A}$

• False positive error rate balance, also referred to as predictive equality. A classifier satisfies this definition if the subjects in the protected and unprotected groups have aqual FPR. This is, if the following formula is satisfied:

${\displaystyle P(R=+\ |\ Y=-,A=a)=P(R=+\ |\ Y=-,A=b)\quad \forall a,b\in A}$

Mathematically, if a classifier has equal FPR for both groups, it will also have equal TNR, satisfying the formula:

${\displaystyle P(R=-\ |\ Y=-,A=a)=P(R=-\ |\ Y=-,A=b)\quad \forall a,b\in A}$

• False negative error rate balance, also referred to as equal opportunity. A classifier satisfies this definition if the subjects in the protected and unprotected groups have equal FNR. This is, if the following formula is satisfied:

${\displaystyle P(R=-\ |\ Y=+,A=a)=P(R=-\ |\ Y=+,A=b)\quad \forall a,b\in A}$

Mathematically, if a classifier has equal FNR for both groups, it will also have equal TPR, satisfying the formula:

${\displaystyle P(R=+\ |\ Y=+,A=a)=P(R=+\ |\ Y=+,A=b)\quad \forall a,b\in A}$

• Equalized odds, also referred to as conditional procedure accuracy equality and disparate mistreatment. A classifier satisfies this definition if the subjects in the protected and unprotected groups have equal TPR and equal FPR, satisfying the formula:

${\displaystyle P(R=+\ |\ Y=y,A=a)=P(R=+\ |\ Y=y,A=b)\quad y\in \{+,-\}\quad \forall a,b\in A}$

• Conditional use accuracy equality. A classifier satisfies this definition if the subjects in the protected and unprotected groups have equal PPV and equal NPV, satisfying the formula:

${\displaystyle P(Y=y\ |\ R=y,A=a)=P(Y=y\ |\ R=y,A=b)\quad y\in \{+,-\}\quad \forall a,b\in A}$

• Overall accuracy equality. A classifier satisfies this definition if the subject in the protected and unprotected groups have equal prediction accuracy, that is, the probability of a subject from one class to be assigned to it. This is, if it satisfies the following formula:

${\displaystyle P(R=Y,A=a)=P(R=Y\ |\ A=b)\quad \forall a,b\in A}$

• Treatment equality. A classifier satisfies this definition if the subjects in the protected and unprotected groups have an equal ratio of FN and FP, satisfying the formula:

${\displaystyle {\frac {FN_{A=a}}{FP_{A=a}}}={\frac {FN_{A=b}}{FP_{A=b}}}}$

Definitions based on predicted probabilities and actual outcome

These definitions are based in the actual outcome ${\textstyle Y}$ and the predicted probability score ${\textstyle S}$.

• Test-fairness, also known as calibration or matching conditional frequencies. A classifier satisfies this definition if individuals with the same predicted probability score ${\textstyle S}$ have the same probability of being classified in the positive class when they belong to either the protected or the unprotected group:

${\displaystyle P(Y=+\ |\ S=s,A=a)=P(Y=+\ |\ S=s,A=b)\quad \forall s\in S\quad \forall a,b\in A}$

• Well-calibration is an extension of the previous definition. It states that when individuals inside or outside the protected group have the same predicted probability score ${\textstyle S}$ they must have the same probability of being classified in the positive class, and this probability must be equal to ${\textstyle S}$:

${\displaystyle P(Y=+\ |\ S=s,A=a)=P(Y=+\ |\ S=s,A=b)=s\quad \forall s\in S\quad \forall a,b\in A}$

• Balance for positive class. A classifier satisfies this definition if the subjects constituting the positive class from both protected and unprotected groups have equal average predicted probability score ${\textstyle S}$. This means that the expected value of probability score for the protected and unprotected groups with positive actual outcome ${\textstyle Y}$ is the same, satisfying the formula:

${\displaystyle E(S\ |\ Y=+,A=a)=E(S\ |\ Y=+,A=b)\quad \forall a,b\in A}$

• Balance for negative class. A classifier satisfies this definition if the subjects constituting the negative class from both protected and unprotected groups have equal average predicted probability score ${\textstyle S}$. This means that the expected value of probability score for the protected and unprotected groups with negative actual outcome ${\textstyle Y}$ is the same, satisfying the formula:

${\displaystyle E(S\ |\ Y=-,A=a)=E(S\ |\ Y=-,A=b)\quad \forall a,b\in A}$

Social welfare function

Some scholars have proposed defining algorithmic fairness in terms of a social welfare function. They argue that using a social welfare function enables an algorithm designer to consider fairness and predictive accuracy in terms of their benefits to the people affected by the algorithm. It also allows the designer to trade off efficiency and equity in a principled way.[15] Sendhil Mullainathan has stated that algorithm designers should use social welfare functions in order to recognize absolute gains for disadvantaged groups. For example, a study found that using a decision-making algorithm in pretrial detention rather than pure human judgment reduced the detention rates for Blacks, Hispanics, and racial minorities overall, even while keeping the crime rate constant.[16]

Individual Fairness criteria

An important distinction among fairness definitions is the one between group and individual notions.[17][18][14][19] Roughly speaking, while group fairness criteria compare quantities at a group level, typically identified by sensitive attributes (e.g. gender, ethnicity, age, etc...), individual criteria compare individuals. In words, individual fairness follow the principle that "similar individuals should receive similar treatments".

There is a very intuitive approach to fairness, which usually goes under the name of Fairness Through Unawareness (FTU), or Blindness, that prescribe not to explicitly employ sensitive features when making (automated) decisions. This is effectively a notion of individual fairness, since two individuals differing only for the values of their sensitive attributes would receive the same outcome.

However, in general, FTU is subject to several drawbacks, the main being that it does not take into account possible correlations between sensitive attributes and non-sensitive attributes employed in the decision-making process. For example, an agent with the (malignant) intention to discriminate on the basis of gender could introduce in the model a proxy variable for gender (i.e. a variable highly correlated with gender) and effectively using gender information while at the same time being compliant to the FTU prescription.

The problem of what variables correlated to sensitive ones are fairly employable by a model in the decision-making process is a crucial one, and is relevant for group concepts as well: independence metrics require a complete removal of sensitive information, while separation-based metrics allow for correlation, but only as far as the labeled target variable "justify" them.

The most general concept of individual fairness was introduced in the pioneer work by Dwork and collaborators in 2012[20] and can be thought of as a mathematical translation of the principle that the decision map taking features as input should be built such that it is able to "map similar individuals similarly", that is expressed as a Lipschitz condition on the model map. They call this approach Fairness Through Awareness (FTA), precisely as counterpoint to FTU, since they underline the importance of choosing the appropriate target-related distance metric in order to assess which individuals are similar in specific situations. Again, this problem is very related to the point raised above about what variables can be seen as "legitimate" in particular contexts.

Causality-based metrics

Causal fairness measures the frequency with which two nearly identical users or applications who differ only in a set of characteristics with respect to which resource allocation must be fair receive identical treatment.[21] An entire branch of the academic research on fairness metrics is devoted to leverage causal models to assess bias in machine learning models. This approach is usually justified by the fact that the same observational distribution of data may hide different causal relationships among the variables at play, possibly with different interpretations of whether the outcome are affected by some form of bias or not.[12]

Kusner et al.[22] propose to employ counterfactuals, and define a decision-making process counterfactually fair if, for any individual, the outcome does not change in the counterfactual scenario where the sensitive attributes are changed. The mathematical formulation reads:

${\displaystyle P(R_{A\leftarrow a}=1\ |\ A=a,X=x)=P(R_{A\leftarrow b}=1\ |\ A=a,X=x),\quad \forall a,b;}$

that is: taken a random individual with sensitive attribute ${\displaystyle A=a}$ and other features ${\displaystyle X=x}$ and the same individual if she had ${\displaystyle A=b}$, they should have same chance of being accepted. The symbol ${\displaystyle {\hat {R}}_{A\leftarrow a}}$ represents the counterfactual random variable ${\displaystyle R}$ in the scenario where the sensitive attribute ${\displaystyle A}$ is fixed to ${\displaystyle A=a}$. The conditioning on ${\displaystyle A=a,X=x}$ means that this requirement is at the individual level, in that we are conditioning on all the variables identifying a single observation.

Bias Mitigation strategies

Fairness can be applied to machine learning algorithms in three different ways: data preprocessing, optimization during software training, or post-processing results of the algorithm.

Preprocessing

Usually, the classifier is not the only problem; the dataset is also biased. The discrimination of a dataset ${\textstyle D}$ with respect to the group ${\textstyle A=a}$ can be defined as follows:

${\displaystyle disc_{A=a}(D)={\frac {|\{X\in D|X(A)\neq a,X(Y)=+\}|}{|\{X\in D|X(A)\neq a\}|}}-{\frac {|\{X\in D|X(A)=a,X(Y)=+\}|}{|\{X\in D|X(A)=a\}|}}}$

That is, an approximation to the difference between the probabilities of belonging in the positive class given that the subject has a protected characteristic different from ${\textstyle a}$ and equal to ${\textstyle a}$.

Algorithms correcting bias at preprocessing remove information about dataset variables which might result in unfair decisions, while trying to alter as little as possible. This is not as simple as just removing the sensitive variable, because other attributes can be correlated to the protected one.

A way to do this is to map each individual in the initial dataset to an intermediate representation in which it is impossible to identify whether it belongs to a particular protected group while maintaining as much information as possible. Then, the new representation of the data is adjusted to get the maximum accuracy in the algorithm.

This way, individuals are mapped into a new multivariable representation where the probability of any member of a protected group to be mapped to a certain value in the new representation is the same as the probability of an individual which doesn't belong to the protected group. Then, this representation is used to obtain the prediction for the individual, instead of the initial data. As the intermediate representation is constructed giving the same probability to individuals inside or outside the protected group, this attribute is hidden to the classificator.

An example is explained in Zemel et al.[23] where a multinomial random variable is used as an intermediate representation. In the process, the system is encouraged to preserve all information except that which can lead to biased decisions, and to obtain a prediction as accurate as possible.

On the one hand, this procedure has the advantage that the preprocessed data can be used for any machine learning task. Furthermore, the classifier does not need to be modified, as the correction is applied to the dataset before processing. On the other hand, the other methods obtain better results in accuracy and fairness.[24]

Reweighing

Reweighing is an example of a preprocessing algorithm. The idea is to assign a weight to each dataset point such that the weighted discrimination is 0 with respect to the designated group.[25]

If the dataset ${\textstyle D}$ was unbiased the sensitive variable ${\textstyle A}$ and the target variable ${\textstyle Y}$ would be statistically independent and the probability of the joint distribution would be the product of the probabilities as follows:

${\displaystyle P_{exp}(A=a\wedge Y=+)=P(A=a)\times P(Y=+)={\frac {|\{X\in D|X(A)=a\}|}{|D|}}\times {\frac {|\{X\in D|X(Y)=+\}|}{|D|}}}$

In reality, however, the dataset is not unbiased and the variables are not statistically independent so the observed probability is:

${\displaystyle P_{obs}(A=a\wedge Y=+)={\frac {|\{X\in D|X(A)=a\wedge X(Y)=+\}|}{|D|}}}$

To compensate for the bias, the software adds a weight, lower for favored objects and higher for unfavored objects. For each ${\textstyle X\in D}$ we get:

${\displaystyle W(X)={\frac {P_{exp}(A=X(A)\wedge Y=X(Y))}{P_{obs}(A=X(A)\wedge Y=X(Y))}}}$

When we have for each ${\textstyle X}$ a weight associated ${\textstyle W(X)}$ we compute the weighted discrimination with respect to group ${\textstyle A=a}$ as follows:

${\displaystyle disc_{A=a}(D)={\frac {\sum W(X)X\in \{X\in D|X(A)\neq a,X(Y)=+\}}{\sum W(X)X\in \{X\in D|X(A)\neq a\}}}-{\frac {\sum W(X)X\in \{X\in D|X(A)=a,X(Y)=+\}}{\sum W(X)X\in \{X\in D|X(A)=a\}}}}$

It can be shown that after reweighting this weighted discrimination is 0.

Inprocessing

Another approach is to correct the bias at training time. This can be done by adding constraints to the optimization objective of the algorithm.[26] These constraints force the algorithm to improve fairness, by keeping the same rates of certain measures for the protected group and the rest of individuals. For example, we can add to the objective of the algorithm the condition that the false positive rate is the same for individuals in the protected group and the ones outside the protected group.

The main measures used in this approach are false positive rate, false negative rate, and overall misclassification rate. It is possible to add just one or several of these constraints to the objective of the algorithm. Note that the equality of false negative rates implies the equality of true positive rates so this implies the equality of opportunity. After adding the restrictions to the problem it may turn intractable, so a relaxation on them may be needed.

This technique obtains good results in improving fairness while keeping high accuracy and lets the programmer choose the fairness measures to improve. However, each machine learning task may need a different method to be applied and the code in the classifier needs to be modified, which is not always possible.[24]

We train two classifiers at the same time through some gradient-based method (f.e.: gradient descent). The first one, the predictor tries to accomplish the task of predicting ${\textstyle Y}$, the target variable, given ${\textstyle X}$, the input, by modifying its weights ${\textstyle W}$ to minimize some loss function ${\textstyle L_{P}({\hat {y}},y)}$. The second one, the adversary tries to accomplish the task of predicting ${\textstyle A}$, the sensitive variable, given ${\textstyle {\hat {Y}}}$ by modifying its weights ${\textstyle U}$ to minimize some loss function ${\textstyle L_{A}({\hat {a}},a)}$.[27][28]

An important point here is that, in order to propagate correctly, ${\textstyle {\hat {Y}}}$ above must refer to the raw output of the classifier, not the discrete prediction; for example, with an artificial neural network and a classification problem, ${\textstyle {\hat {Y}}}$ could refer to the output of the softmax layer.

Then we update ${\textstyle U}$ to minimize ${\textstyle L_{A}}$ at each training step according to the gradient ${\textstyle \nabla _{U}L_{A}}$ and we modify ${\textstyle W}$ according to the expression:

${\displaystyle \nabla _{W}L_{P}-proj_{\nabla _{W}L_{A}}\nabla _{W}L_{P}-\alpha \nabla _{W}L_{A}}$
where $\alpha$ is a tuneable hyperparameter that can vary at each time step.

Graphic representation of the vectors used in adversarial debiasing as shown in Zhang et al.[27]

The intuitive idea is that we want the predictor to try to minimize ${\textstyle L_{P}}$ (therefore the term ${\textstyle \nabla _{W}L_{P}}$) while, at the same time, maximize ${\textstyle L_{A}}$ (therefore the term ${\textstyle -\alpha \nabla _{W}L_{A}}$), so that the adversary fails at predicting the sensitive variable from ${\textstyle {\hat {Y}}}$.

The term ${\textstyle -proj_{\nabla _{W}L_{A}}\nabla _{W}L_{P}}$ prevents the predictor from moving in a direction that helps the adversary decrease its loss function.

It can be shown that training a predictor classification model with this algorithm improves demographic parity with respect to training it without the adversary.

Postprocessing

The final method tries to correct the results of a classifier to achieve fairness. In this method, we have a classifier that returns a score for each individual and we need to do a binary prediction for them. High scores are likely to get a positive outcome, while low scores are likely to get a negative one, but we can adjust the threshold to determine when to answer yes as desired. Note that variations in the threshold value affect the trade-off between the rates for true positives and true negatives.

If the score function is fair in the sense that it is independent of the protected attribute, then any choice of the threshold will also be fair, but classifiers of this type tend to be biased, so a different threshold may be required for each protected group to achieve fairness.[29] A way to do this is plotting the true positive rate against the false negative rate at various threshold settings (this is called ROC curve) and find a threshold where the rates for the protected group and other individuals are equal.[29]

The advantages of postprocessing include that the technique can be applied after any classifiers, without modifying it, and has a good performance in fairness measures. The cons are the need to access to the protected attribute in test time and the lack of choice in the balance between accuracy and fairness.[24]

Reject Option based Classification

Given a classifier let ${\textstyle P(+|X)}$ be the probability computed by the classifiers as the probability that the instance ${\textstyle X}$ belongs to the positive class +. When ${\textstyle P(+|X)}$ is close to 1 or to 0, the instance ${\textstyle X}$ is specified with high degree of certainty to belong to class + or - respectively. However, when ${\textstyle P(+|X)}$ is closer to 0.5 the classification is more unclear.[30]

We say ${\textstyle X}$ is a "rejected instance" if ${\textstyle max(P(+|X),1-P(+|X))\leq \theta }$ with a certain ${\textstyle \theta }$ such that ${\textstyle 0.5<\theta <1}$.

The algorithm of "ROC" consists on classifying the non-rejected instances following the rule above and the rejected instances as follows: if the instance is an example of a deprived group (${\displaystyle X(A)=a}$) then label it as positive, otherwise, label it as negative.

We can optimize different measures of discrimination (link) as functions of ${\textstyle \theta }$ to find the optimal ${\textstyle \theta }$ for each problem and avoid becoming discriminatory against the privileged group.[30]

References

1. ^ Moritz Hardt, Berkeley. Retrieved 18 December 2019
2. ^ "IBM AI Fairness 360 open source toolkit adds new functionalities". Tech Republic.
3. ^ IBM AI Fairness 360. Retrieved 18 December 2019
4. ^ Fairness Flow el detector de sesgos de Facebook. Retrieved 28 December 2019
5. ^ ML-Fairness gym. Retrieved 18 December 2019
6. ^ "Attorney General Eric Holder Speaks at the National Association of Criminal Defense Lawyers 57th Annual Meeting and 13th State Criminal Justice Network Conference". www.justice.gov. 2014-08-01. Retrieved 2022-04-16.
7. ^ Mattu, Julia Angwin,Jeff Larson,Lauren Kirchner,Surya. "Machine Bias". ProPublica. Retrieved 2022-04-16.
8. ^ Rose, Adam (2010-01-22). Time. ISSN 0040-781X http://content.time.com/time/business/article/0,8599,1954643,00.html. Retrieved 2022-04-16. {{cite magazine}}: Missing or empty |title= (help)
9. ^ "Google says sorry for racist auto-tag in photo app". the Guardian. 2015-07-01. Retrieved 2022-04-16.
10. ^ "A beauty contest was judged by AI and the robots didn't like dark skin". the Guardian. 2016-09-08. Retrieved 2022-04-16.
11. ^ Buolamwini, Joy; Gebru, Timnit (February 2018). Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification (PDF). Conference on Fairness, Accountability and Transparency. New York, NY, USA. pp. 77–91.
12. ^ a b c d Solon Barocas; Moritz Hardt; Arvind Narayanan, Fairness and Machine Learning. Retrieved 15 December 2019.
13. ^ Mayes, Elizabeth (2001). Handbook of Credit Scoring. NY, NY, USA: Glenlake Publishing. p. 282. ISBN 0-8144-0619-X.
14. ^ a b c Verma, Sahil, and Julia Rubin. "Fairness definitions explained." In 2018 IEEE/ACM international workshop on software fairness (fairware), pp. 1-7. IEEE, 2018.
15. ^ Chen, Violet (Xinying); Hooker, J. N. (2021). "Welfare-based Fairness through Optimization". arXiv:2102.00311 [cs.AI].
16. ^ Mullainathan, Sendhil (June 19, 2018). Algorithmic Fairness and the Social Welfare Function. Keynote at the 19th ACM Conference on Economics and Computation (EC'18). YouTube. 48 minutes in. In other words, if you have a social welfare function where what you care about is harm, and you care about harm to the African Americans, there you go: 12 percent less African Americans in jail overnight.... Before we get into the minutiae of relative harm, the welfare function is defined in absolute harm, so we should actually calculate the absolute harm first.
17. ^ Mitchell, Shira, Eric Potash, Solon Barocas, Alexander D'Amour, and Kristian Lum. "Algorithmic fairness: Choices, assumptions, and definitions." Annual Review of Statistics and Its Application 8 (2021): 141-163.
18. ^ Castelnovo, Alessandro, Riccardo Crupi, Greta Greco, Daniele Regoli, Ilaria Giuseppina Penco, and Andrea Claudio Cosentini. "A clarification of the nuances in the fairness metrics landscape." Scientific Reports 12, no. 1 (2022): 1-21.
19. ^ Mehrabi, Ninareh, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. "A survey on bias and fairness in machine learning." ACM Computing Surveys (CSUR) 54, no. 6 (2021): 1-35.
20. ^ Dwork, Cynthia, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. "Fairness through awareness." In Proceedings of the 3rd innovations in theoretical computer science conference, pp. 214-226. 2012.
21. ^ Galhotra, Sainyam; Brun, Yuriy; Meliou, Alexandra (2017). Fairness Testing: Testing Software for Discrimination. Proceedings of the 11th Joint Meeting of the European Software Engineering Conference and ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE). pp. 498–510. arXiv:1709.03221. doi:10.1145/3106237.3106277. ISBN 9781450351058. S2CID 6324652.
22. ^ Kusner, M. J., Loftus, J., Russell, C., & Silva, R. (2017). Counterfactual fairness. Advances in neural information processing systems, 30.
23. ^ Richard Zemel; Yu (Ledell) Wu; Kevin Swersky; Toniann Pitassi; Cyntia Dwork, Learning Fair Representations. Retrieved 1 December 2019
24. ^ a b c Ziyuan Zhong, Tutorial on Fairness in Machine Learning. Retrieved 1 December 2019
25. ^ Faisal Kamiran; Toon Calders, Data preprocessing techniques for classification without discrimination. Retrieved 17 December 2019
26. ^ Muhammad Bilal Zafar; Isabel Valera; Manuel Gómez Rodríguez; Krishna P. Gummadi, Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment. Retrieved 1 December 2019
27. ^ a b Brian Hu Zhang; Blake Lemoine; Margaret Mitchell, Mitigating Unwanted Biases with Adversarial Learning. Retrieved 17 December 2019
28. ^ Joyce Xu, Algorithmic Solutions to Algorithmic Bias: A Technical Guide. Retrieved 17 December 2019
29. ^ a b Moritz Hardt; Eric Price; Nathan Srebro, Equality of Opportunity in Supervised Learning. Retrieved 1 December 2019
30. ^ a b Faisal Kamiran; Asim Karim; Xiangliang Zhang, Decision Theory for Discrimination-aware Classification. Retrieved 17 December 2019