Continual Learning with MNIST

A Python-based machine learning pipeline designed to train, evaluate, and benchmark continual learning algorithms on the MNIST dataset. The project tracks model degradation, catastrophic forgetting, and mitigation strategies (such as experience replay).

📂 Project Structure

├── data/                  # Dataset directory
│   └── MNIST/raw/         # Raw MNIST binary files (.ubyte.gz)
├── results/               # Generated training metrics, matrices, and plots
├── config.py              # Global configurations and hyperparameters
├── dataset.py             # Data loading and preprocessing pipelines
├── model.py               # Core neural network architectures
├── metrics.py             # Performance tracking (accuracy, forgetting, etc.)
├── train_baseline.py      # Standard baseline training routine
├── sweep_knob.py          # Hyperparameter sweeping script
├── test_sample.py         # Unit testing and sampling validation
└── diagnose.py            # Diagnostic tools to evaluate model performance

🚀 Getting Started

1. Prerequisites

Ensure you have Python 3.8+ installed. Install the required dependencies:

pip install -r requirements.txt

(Note: Common requirements include torch, torchvision, numpy, matplotlib, and pandas)

2. Dataset Setup

The setup automatically downloads or parses the MNIST raw binaries inside the data/ directory. Ensure your raw data path matches:

data/MNIST/raw/train-images-idx3-ubyte

3. Running the Pipeline

• Train the Baseline Model:

  python train_baseline.py

• Run Hyperparameter Sweeps:

  python sweep_knob.py

• Run Diagnostics & Evaluation:

  python diagnose.py
  

📊 Results & Artifacts

All training runs automatically log data to the results/ directory, generating:

• Confusion Matrices: baseline_matrix.csv and replay_matrix.csv to evaluate performance across sequential tasks.
• Performance Plots: Visual graphs such as baseline_task1_decay.png and mitigation_plot.png tracking catastrophic forgetting mitigation.

📝 License

This project is licensed under the MIT License.