Autonomous Perception

A robotics perception engineering project documenting the transition from traditional 2D computer vision to ROS2-based autonomous perception systems.

Built using CARLA, ROS2, OpenCV, and YOLOv8, the project progresses through a series of engineering phases covering simulator integration, perception pipelines, modular ROS2 architectures, system benchmarking, and performance optimization.

---

Technology Stack

Category	Technologies
Simulation	CARLA 0.9.15
Robotics Middleware	ROS2 Humble
Computer Vision	OpenCV
Object Detection & Segmentation	YOLOv8
Programming Language	Python
Communication	CycloneDDS
Model Optimization	ONNX Runtime, TensorRT
Environment	Windows 11 + WSL2 Ubuntu 22.04

---

System Architecture

CARLA Simulator
        │
        ▼
ROS2 Communication Layer
        │
        ▼
Perception Pipeline
        │
        ▼
Hardware, Network & System Profiling
        │
        ▼
Replay-Based Benchmarking
        │
        ▼
Quantization & Pipeline Optimization

---

Completed Phases

✅ Phase 1 — CARLA + ROS2 Integration

Validated end-to-end communication between CARLA and ROS2 using RGB camera streaming and OpenCV visualization.

📁 View Phase 1

---

✅ Phase 2 — Real-Time Object Detection

Integrated YOLOv8 into the ROS2 perception pipeline for real-time object detection and visualization.

📁 View Phase 2

---

✅ Phase 3 — Modular ROS2 Architecture

Converted standalone scripts into a reusable ROS2 package architecture with launch files, configuration management, and automated deployment workflows.

📁 View Phase 3

---

✅ Phase 4 — Offline Replay & Benchmarking

Developed a deterministic dataset generation and offline replay framework for robotics perception benchmarking. Validated synchronized recording, replay-based perception evaluation, and resource utilization analysis across online and offline perception pipelines.

📁 View Phase 4

✅ Phase 5 — ROS2 Transport Performance Investigation

Investigated DDS transport behavior, throughput limitations, QoS configurations, and publisher scalability for perception workloads. Validated replay-based benchmarking, large-payload transport, and multi-threaded publishing architectures.

📁 View Phase 5

✅ Phase 6 — Model Compression & Perception Optimization

Optimized the perception pipeline through TensorRT acceleration, prediction consistency analysis, pipeline profiling, and architecture optimization. Benchmarked FP32, FP16, and INT8 inference while identifying rendering bottlenecks and improving end-to-end perception throughput.

📁 View Phase 6

---

Demonstrations

Phase 1 — CARLA + ROS2 Integration	Phase 2 — Real-Time Object Detection
End-to-end CARLA → ROS2 → OpenCV pipeline validation	Real-time object detection using YOLOv8 and ROS2

Phase 4 — Offline Replay & Benchmarking	Phase 5 — ROS2 Transport Performance Investigation
Deterministic offline replay and perception benchmarking	Multi-threaded DDS throughput scaling

Phase 6 — Model Compression & Perception Optimization

TensorRT optimization, pipeline profiling, and end-to-end perception acceleration.

---

Key Performance Results

Engineering Milestone	Result
Offline Replay (Phase 4)	CPU utilization reduced from ~75% → ~30%
Offline Replay (Phase 4)	GPU utilization reduced from ~60% → ~30%
ROS2 Transport (Phase 5)	Live CARLA publisher: ~3 FPS
ROS2 Transport (Phase 5)	Multi-threaded DDS publisher: ~15 FPS
Model Optimization (Phase 6)	TensorRT FP32 → INT8: ~3.5 FPS → ~26 FPS
Overall Improvement	~7.4× end-to-end perception throughput

Additional Validation

• TensorRT FP16 preserved high prediction consistency (94.75% detection agreement, 97.77% class agreement, 0.976 mean Box IoU) compared with the FP32 baseline.

---

Future Work

Planned areas of development include:

• Sensor synchronization
• Multi-camera perception
• 3D perception workflows
• Sensor fusion
• Distance Estimation
• Bird's Eye View (BEV)
• Autonomous systems engineering workflows

---

Project Journal

Detailed development notes, planning, experiments, and progress tracking are maintained separately:

• roadmap/project_journal.md