Machine Learning/AI Bias
Deep learning algorithms are vastly used in AI based applications today. These algorithms find patterns in data, and are being used to make decisions that affect people’s lives.
But are these models completely fair and free of bias?
As humans, we have an understanding of fairness. It is our responsibility to ensure that our algorithms are fair.
What is fairness?
- Equal treatment
- Being impartial
Why is it important to not have bias?
- Bias can alter decisions
- An incorrect result could ruin somebody’s life
To understand the implications of bias in real life, let’s look at machine bias in the criminal justice system.
ProPublica, an independent, non-profit, investigative journalism news site published an article on Machine bias in 2016. It brought to light racial bias in the software used to predict future criminals.
One of the most important predictions law enforcement can make is the likelihood of a person to commit a crime again. It is called Recidivism Rate, ie, the likelihood of re-offending
When using AI in criminal justice (or in general), racial bias can become a huge issue. It can have devastating and unfair consequences. In order to avoid this, we have to ask if the predictions are fair.
COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) is an algorithm used in the US Judicial System to assess a criminal’s likelihood of becoming a recidivist, ie, re-offend.
This algorithm assigns a COMPAS score to the offenders to predict if they will re-offend. ProPublica found that this system predicts African-Americans to have a higher likelihood of recommitting a crime than other races.
Borden had committed a petty theft and had a minor misdemeanors on her record from when she was a juvenile. Prater was a more seasoned criminal, having served jail time before for armed robbery and also some other charges against him.
But according to COMPAS scores which is essentially a risk assessment, Borden was rated at higher risk and Prater was rated at low risk.
However, two years later, the computer algorithm was found to have made a wrong prediction. Borden had not committed any new crimes. Prater, on the other hand, was serving an 8 year sentence for robbery.
According to ProPublica, the COMPAS model prediction fails by incorrectly predicting higher recidivism for African-American defendants and lower recidivism for whites.
Northpointe (the company that created COMPAS ) did a technical analysis to counter ProPublica’s article on racial bias. They conducted a technical analysis using the same data that ProPublica had analyzed, to address the concerns raised by the ProPublica article on the fairness of the COMPAS Risk Scale for blacks and whites.
The table shows the AUC results for the General Recidivism Risk Scale (GRRS) and Violent Recidivism Risk Scale (VRRS) in a sample dataset. This sample data is a subset of a two-year data frame (general recidivism date frame for GRRS and violent recidivism data frame for VRRS) with filters applied as used by ProPublica.
GRRS Analysis: AUC for whites is 0.693 (0.670, 0.716). The AUC for blacks is 0.704 (0.686, 0.722). The test comparing the areas under the respective ROC curves for blacks and whites indicates the areas are not significantly different (p=0.438).
VRRS Analysis: AUC for whites is 0.683 (0.640, 0.726). The AUC for blacks is 0.708 (0.680, 0.737). The test comparing the areas under the respective ROC curves for blacks and whites indicates the areas are not significantly different (p=0.383).
Northpointe in their technical review and analysis supported by data, calculated predictive values for blacks and whites in the target population and concluded that their model is not biased.
How do we detect and mitigate bias in machine learning models?
Bias can creep in at any of the stages of model building — understanding/framing the problem, collecting the data or preparing the data.
Machine bias is a hard problem to fix, but one that the scientific community is working hard to solve. According to an article in MIT Technology Review, AI researchers are using a wide variety to approaches to mitigate the AI bias:
- Algorithms that help detect and mitigate hidden biases within training data. (eg: AI Fairness 360)
- Algorithms that mitigate the biases learned by the model regardless of the data quality
- Processes that hold companies accountable to the fairer outcomes. eg: Gender Shades
- Discussions that hash out the different definitions of fairness (eg: AI Fairness 360)
According to the European Union High Level Expert Group on Artificial Intelligence guidelines, we can reduce bias by making our machine learning models:
- Lawful — respecting all applicable laws and regulations
- Robust — both from a technical perspective while taking into account its social environment
- Ethical — respecting ethical principles and values
The good news is that we now know that AI bias is a real problem with major consequences and are taking steps in the right direction to find solutions.
Sources
- https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
- https://www.equivant.com/response-to-propublica-demonstrating-accuracy-equity-and-predictive-parity/
- http://go.volarisgroup.com/rs/430-MBX-989/images/ProPublica_Commentary_Final_070616.pdf
- http://www.northpointeinc.com/files/publications/Criminal-Justice-Behavior-COMPAS.pdf
- https://www.technologyreview.com/2019/02/04/137602/this-is-how-ai-bias-really-happensand-why-its-so-hard-to-fix/
- https://towardsdatascience.com/is-your-machine-learning-model-biased-94f9ee176b67
- https://www.toptal.com/artificial-intelligence/mitigating-ai-bias
Divya Sikka is a Student Ambassador in the Inspirit AI Student Ambassadors Program. Inspirit AI is a pre-collegiate enrichment program that exposes curious high school students globally to AI through live online classes. Learn more at https://www.inspiritai.com/.