A deep learning system for crowd density detection and behavioural anomaly detection in surveillance footage.
CrowdWatch combines two models into a single web dashboard: a YOLOv8 detector that counts people and generates crowd density heatmaps, and a CNN-LSTM classifier that flags anomalous crowd behaviour. When crowd density or anomaly risk crosses a threshold, the system triggers an alert.
This project was developed as a Final Year Project for the Bachelor of Information Systems (Hons.) in Intelligent Systems Engineering at Universiti Teknologi MARA (UiTM).
- Crowd density detection using YOLOv8m, with people count and bounding box output
- Density heatmap generation showing where people are concentrated in the frame
- Behavioural anomaly detection using a CNN-LSTM model with a ResNet-18 backbone (binary: normal vs anomalous)
- Three input modes: single image analysis, video file analysis, and live webcam feed
- Live analytics dashboard with people count over time and current risk level
- Three alert channels: audible tone, on-screen red banner, and Telegram bot notification
Trained on the CrowdHuman dataset (15,000 training images, 4,370 validation images).
| Metric | Score |
|---|---|
| Precision | 86.1% |
| Recall | 76.5% |
| F1-Score | 81.0% |
| mAP@0.5 | 83.7% |
| Inference speed | 181.8 FPS |
Trained on the ShanghaiTech Campus dataset. 238 labelled clips were split 80:20 into 190 training clips and 48 validation clips.
| Metric | Score |
|---|---|
| Accuracy | 87.5% |
| AUC-ROC | 93.41% |
| F1-Score | 76.92% |
Confusion matrix on the 48 validation clips: TN=32, FP=3, FN=3, TP=10.
The model performs binary classification only. It labels a clip as normal or anomalous but does not identify the specific type of anomaly.
| Component | Technology |
|---|---|
| Language | Python 3.11.9 |
| Deep learning | PyTorch (nightly, CUDA 12.8) |
| Detection | Ultralytics YOLOv8 8.4.43 |
| Video processing | OpenCV 4.13.0 |
| Web framework | Flask 3.1 |
| Frontend | HTML, CSS, JavaScript, Chart.js |
| Streaming | MJPEG |
| Alerts | Telegram Bot API |
All training and testing was done locally.
| Component | Specification |
|---|---|
| Laptop | Lenovo 83LY |
| CPU | Intel Core i7-14700HX |
| RAM | 32GB |
| GPU | NVIDIA RTX 5060 Laptop GPU (8GB VRAM) |
| OS | Windows 11 Home 64-bit |
git clone https://github.com/YOURUSERNAME/CrowdWatch.git
cd CrowdWatchpython -m venv venv
venv\Scripts\activateThe RTX 50-series uses the Blackwell architecture (sm_120), which is not supported by the current stable PyTorch release. The nightly build with CUDA 12.8 is required:
pip install --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/cu128If you are on an older GPU, the standard stable install will work fine instead.
pip install -r requirements.txtThe trained weights are not included in this repository because of file size limits. Download them from the Releases page and place them in a weights/ folder in the project root:
weights/
├── yolov8m_crowdhuman.pt
└── cnn_lstm_resnet18.pth
Create a .env file in the project root:
TELEGRAM_BOT_TOKEN=your_token_here
TELEGRAM_CHAT_ID=your_chat_id_here
If you skip this step, the dashboard still works. Only the Telegram alerts will be disabled.
Start the Flask dashboard:
python scripts/app_full.pyThen open http://localhost:5000 in your browser.
From the dashboard you can upload an image, upload a video, or start the webcam feed. Results appear with the detection overlay, heatmap, people count, and behaviour classification.
The datasets are not included in this repository. To retrain the models from scratch, download them first:
- CrowdHuman from Hugging Face
- ShanghaiTech Campus from Kaggle
Then run the preparation scripts, which convert annotations and generate the train/validation splits:
python scripts/prepare_crowdhuman.py
python scripts/prepare_shanghaitech.pyTo retrain:
python scripts/train_yolo.py
python scripts/train_cnn_lstm.pyTo evaluate:
python scripts/evaluate_cnn_lstm.py
python scripts/calculate_metrics.pyCrowdWatch/
├── scripts/
│ ├── app_full.py Main Flask application
│ ├── prepare_crowdhuman.py CrowdHuman annotation conversion
│ ├── prepare_shanghaitech.py ShanghaiTech clip preparation and splitting
│ ├── train_yolo.py YOLOv8 training
│ ├── train_cnn_lstm.py CNN-LSTM training
│ ├── evaluate_cnn_lstm.py CNN-LSTM evaluation
│ ├── calculate_metrics.py Metric calculation
│ ├── generate_heatmaps.py Heatmap generation
│ ├── generate_samples.py Sample output generation
│ ├── telegram_notif.py Telegram alert handler
│ ├── templates/ HTML templates
│ └── static/ CSS, JavaScript, alert audio
├── requirements.txt
├── .gitignore
└── README.md
- The anomaly detection model is trained on clip-level labels using an OR rule, where a clip is marked anomalous if any single frame within it is anomalous. This is a coarse form of supervision.
- The validation set contains only 13 anomalous clips, so the anomaly-class metrics are based on a small sample.
- The ShanghaiTech test set of 199 clips could not be used because it does not ship with ground-truth label files.
- The CNN-LSTM is trained on whole frames rather than person-cropped regions, so background context influences the classification.
- CrowdHuman is largely eye-level and street-perspective footage, so detection accuracy drops on densely packed crowds filmed from a high or overhead angle.
- End-to-end processing latency has not been formally profiled. The FPS figures above describe raw model inference speed only.
- Multi-class anomaly detection to identify the specific type of anomalous behaviour rather than a binary flag
- Formal experimental baseline comparison against alternative detection and classification architectures
- Training on person-cropped regions instead of whole frames
- Frame-level rather than clip-level supervision
Supervised by Prof. Dr. Marina Yusoff, Universiti Teknologi MARA.
Datasets used: CrowdHuman and ShanghaiTech Campus.