FairLens

FairLens is a machine learning project focused on fairness in loan approval systems.

The main idea behind this project is to build a loan approval model and then investigate whether the model treats different groups of applicants fairly. If bias is identified, different bias mitigation techniques can be applied and the results can be compared.

The project is based on the ideas presented in the research paper "FairCompass: Operationalizing Fairness in Machine Learning" and adapts them to the specific domain of banking and loan approval.

Why FairLens?

Machine learning models can make loan approval faster and more consistent, but they can also learn biases that exist in historical data.

For example, a model might have different approval rates for different groups of applicants. Simply looking at the overall accuracy of the model may not reveal these problems.

FairLens focuses on looking beyond accuracy by analyzing:

• Model performance
• Approval rates between groups
• False positive and false negative rates
• Fairness metrics
• Intersectional subgroups
• Bias before and after mitigation
• Human review of suspicious or uncertain decisions

Project Workflow

Loan Dataset
     |
     v
Data Preprocessing
     |
     v
Train Loan Approval Model
     |
     v
Model Predictions
     |
     v
Fairness Audit
     |
     +--------------------+
     |                    |
     v                    v
No Significant Bias    Bias Detected
     |                    |
     |                    v
     |              Bias Mitigation
     |                    |
     |                    v
     |              Re-evaluate Model
     |                    |
     +---------+----------+
               |
               v
       Human-in-the-Loop
               |
               v
        Final Decision

Main Objectives

The main objectives of the project are:

1. Build a machine learning model for loan approval.
2. Measure the performance of the model.
3. Identify potential bias between different applicant groups.
4. Analyze intersectional subgroups.
5. Apply suitable bias mitigation techniques.
6. Compare the model before and after mitigation.
7. Provide fairness information that can help a human reviewer make a better-informed decision.

Fairness Analysis

FairLens can be used to compare model outcomes across different groups.

For example:

Group A
Approval Rate: 72%

Group B
Approval Rate: 51%

A difference like this does not automatically prove that the model is discriminatory. It indicates that the groups should be investigated further.

The project focuses on using fairness metrics and subgroup analysis to understand where these differences come from.

Fairness Metrics

Depending on the dataset and use case, the project can evaluate metrics such as:

• Demographic Parity
• Equal Opportunity
• Equalized Odds
• False Positive Rate
• False Negative Rate
• Selection / Approval Rate

The choice of metric is important because different fairness definitions can lead to different conclusions.

This follows an important idea from the FairCompass paper: fairness cannot always be reduced to a single metric.

Subgroup Analysis

One of the important parts of FairLens is looking at combinations of attributes instead of checking each attribute separately.

For example:

Gender
Gender + Income
Gender + Employment
Age + Income
Gender + Age + Income

This can help identify problems that may not be visible when looking at only one attribute.

For example, the overall difference between two groups might be small, while a particular intersectional subgroup may show a much larger difference.

Bias Mitigation

After identifying a potential fairness problem, FairLens can experiment with different mitigation approaches.

Pre-processing

Bias is addressed before the model is trained.

Examples include:

• Re-weighting
• Re-sampling
• Data transformation

In-processing

Fairness is considered during model training.

Examples include:

• Fairness constraints
• Fairness-aware optimization
• Adversarial debiasing

Post-processing

The trained model's predictions are adjusted to improve a selected fairness objective.

Examples include:

• Threshold adjustment
• Equalized-odds-based methods
• Reject-option approaches

The goal is not simply to maximize fairness. The project also compares predictive performance before and after mitigation.

Human-in-the-Loop

A major part of the project is the Human-in-the-Loop (HITL) approach.

The idea is not to make the machine learning model the final authority for every loan application.

Instead, the model provides a recommendation along with information that helps a human reviewer understand the decision.

A simplified process is:

Applicant
    |
    v
ML Model
    |
    v
Prediction + Confidence
    |
    v
Fairness Check
    |
    +-------------------+
    |                   |
    v                   v
Normal Case         Flagged Case
    |                   |
    |                   v
    |              Human Review
    |                   |
    +---------+---------+
              |
              v
       Final Decision

Cases can be flagged for review when the model has low confidence or when the fairness analysis identifies a potential issue.

The human reviewer can then examine the available information and make the final decision.

FairCompass Connection

FairLens is inspired by the FairCompass research paper.

FairCompass proposes a human-centered approach to fairness auditing and introduces an:

Exploration
     |
     v
Guidance
     |
     v
Informed Analysis
     |
     v
New Investigation

FairLens adapts this idea to loan approval.

The user can first explore the data and model results, use fairness information to investigate possible issues, and then use the findings to make a more informed analysis.

Model Evaluation

The project evaluates both normal machine learning performance and fairness.

Model Performance

• Accuracy
• Precision
• Recall
• F1 Score
• ROC-AUC

Fairness

• Approval rate differences
• Demographic parity
• Equal opportunity
• Equalized odds
• False positive rate
• False negative rate
• Subgroup differences

The final evaluation will compare the baseline model with the models produced after applying bias mitigation techniques.

Example Comparison

The final results can be presented in a table similar to:

Model	Accuracy	Precision	Recall	F1 Score	Fairness Gap
Baseline	-	-	-	-	-
Re-weighted	-	-	-	-	-
Fairness-aware	-	-	-	-	-
Post-processed	-	-	-	-	-

The values will be replaced with the actual experimental results.

Tech Stack

The project is being developed using Python and common machine learning tools.

• Python
• Pandas
• NumPy
• Scikit-learn
• Fairlearn
• IBM AI Fairness 360
• Matplotlib
• Plotly
• Jupyter Notebook
• VS Code
• Git
• GitHub

The exact technologies may change as development continues.

Project Structure

The planned project structure is:

FairLens/
│
├── data/
│   ├── raw/
│   └── processed/
│
├── notebooks/
│   ├── data_analysis.ipynb
│   ├── model_training.ipynb
│   └── fairness_analysis.ipynb
│
├── src/
│   ├── preprocessing/
│   ├── models/
│   ├── fairness/
│   ├── mitigation/
│   └── evaluation/
│
├── dashboard/
│   └── app.py
│
├── models/
│
├── reports/
│
├── requirements.txt
└── README.md

The planned development stages are:

• Prepare and clean the loan approval dataset
• Perform exploratory data analysis
• Train the baseline loan approval model
• Evaluate baseline performance
• Implement fairness auditing
• Perform subgroup analysis
• Implement bias mitigation
• Compare baseline and mitigated models
• Develop the fairness dashboard
• Implement the human-in-the-loop review workflow
• Generate fairness audit reports

Future Improvements

Some possible improvements for the project are:

• Better subgroup discovery
• More fairness metrics
• Explainable AI for individual predictions
• Counterfactual explanations
• Model drift monitoring
• Continuous fairness monitoring
• Human decision tracking
• Automated fairness reports
• Additional loan approval datasets
• Domain-specific fairness rules

Important Note

FairLens is an academic project and is intended for research and experimentation.

A fairness metric or machine learning prediction should not be treated as the only factor in a real-world lending decision. Loan approval systems require appropriate human oversight, domain knowledge, organizational policies, and compliance with applicable laws and regulations.

The paper proposes a human-centered approach to fairness auditing using visual analytics, subgroup discovery, fairness guidance, and a human-in-the-loop process.

Author

JAI AKASH K

Software Engineer.

---

If you find this project useful or have suggestions, feel free to open an issue or contribute to the project.