Undergraduate research at Swinburne University of Technology (Undergraduate Research Program) on fine-tuning transformer models for biomedical / clinical question answering.
Fine-tunes BERT on medical QA datasets (CPGQA + SQuAD) to answer questions grounded in clinical practice guidelines, and studies which fine-tuning strategies preserve accuracy while cutting training cost.
- +65% medical-QA accuracy from fine-tuning BERT on curated clinical datasets.
- Evaluated LoRA, layer freezing, and hyperparameter sweeps — identified a configuration that cut training compute ~30% with no measurable quality loss.
- Results and reproducible code published in the Swinburne research repository.
- Base model — BERT (transformer encoder) fine-tuned for extractive QA.
- Data — CPGQA (clinical practice-guideline QA) with SQuAD for transfer.
- Efficiency — parameter-efficient tuning (LoRA) + selective layer freezing.
- Evaluation — hyperparameter suites to map the quality vs. compute trade-off.
| Path | Description |
|---|---|
BERT_Finetuned-CPGQA-SQuAD/ |
Fine-tuning experiments and trained model artifacts |
Colab Notebooks/ |
Training and evaluation notebooks |
Fortnightly Reports/ |
Research progress reports |
Website/ |
Project demo site |
Python · PyTorch · Hugging Face Transformers · BERT · LoRA · SQuAD · Jupyter