DIMS • ORTHO Viewer

Interactive dashboard for ORTHO game session data — trajectory visualization, kinematics processing, and game analysis.

Public code-only mirror in the DIMS-network, forked from bots-viewer. It ships no datasets (no ortho.db, no session/timeseries data) and the multi-agent simulation module has been excluded. Build ortho.db from your own data via import_data.sh (see Quick Start).

What's Inside

1. SQLite Database (ortho.db) with 104 ORTHO game sessions (CNK‑ORTHO / Michal Weiss data)

• 1,049 game tracks, 292k trajectory points
• Sessions metadata: team name, date, participant ages, companionship, duration
• Track‑level data: difficulty level, completion status, mistakes, timing
• Processed kinematics: vx, vy, speed, acceleration per point (computed lazily)

2. Dash 4.x Web App (app.py) – dark‑theme DIMS‑style dashboard:

• Sidebar filters: team name, date range, age X/Y sliders, companionship
• Session table: click to select a game; "Processed" column shows kinematics status
• Lazy processing: first time you select a session → vx, vy, speed, acceleration computed and stored → instant on revisit
• Four tabs:
  1. Session Info – metadata, participant details, performance stats
  2. Tracks Timeline – horizontal Gantt bars (green=completed, red=mistake, orange=active)
  3. Trajectory – X,Y scatter plot; single‑track view colors points by speed (Plasma colorscale)
  4. Kinematics – four‑row subplot: speed | Vx | Vy | acceleration over time (all tracks overlaid)

4. Data‑layer utilities:

• ortho_db.py – OrthoDatabase class (CRUD + kinematics processing API)
• importer.py – JSON importer for ORTHO‑Data.json (Sessions_List / Track_Results format)
• schema.sql – full SQLite schema (sessions, game_tracks, trajectories, kinematics, session_stats, …)

Quick Start

1. Import Data (First time setup)

cd /home/m11/codes/DIMS_ORTHO_VIEWER

# Option A: Using the bash script (recommended)
./import_data.sh

# Option B: Using Python script directly
python import_data.py --db ortho.db

# Option C: Using the original importer
python importer.py --db ortho.db --json logs/ORTHO-Data.json

2. Run the Dashboard (requires conda environment dims with Dash, Plotly, Pandas, NumPy):

./run.sh               # defaults to port 8050
# or
python3 app.py --db ortho.db --port 8050

3. Open Browser: http://localhost:8050

4. Use the Dashboard:

• Filter sessions with the left‑sidebar controls
• Click a row in the session table to select a game
• First selection triggers kinematics computation (≈1‑2 sec)
• Navigate tabs to see trajectory, speed, acceleration plots

Data Pipeline

ORTHO‑Data.json (CNK‑ORTHO logs)
    ↓ (importer.py)
SQLite ortho.db (raw trajectories)
    ↓ (on first session select)
Kinematics table (vx, vy, speed, acceleration)
    ↓ (Dash callbacks)
Interactive visualizations

Database Architecture

Core Tables

Table	Description	Key Fields
sessions	Per‑session metadata	session_id, date, team_name, age_x, age_y, companionship_x, companionship_y
game_tracks	Level attempts	track_key, session_id, level_seq, difficulty_level, completed, mistake
trajectories	Raw X,Y positions	traj_id, session_id, track_key, point_x, point_y, timestamp_ms
kinematics	Computed velocities	session_id, track_key, timestamp_ms, vx, vy, speed, acceleration
session_stats	Cached aggregates	session_id, total_levels_attempted, total_mistakes, avg_speed

Advanced Analysis Tables

Table	Purpose	Analysis Type
rqa_results	Recurrence Quantification Analysis	Nonlinear dynamics
crqa_results	Cross‑Recurrence Quantification Analysis	Coordination between variables
crosswavelet_results	Wavelet coherence analysis	Time‑frequency coordination

Analysis Capabilities

The system includes comprehensive analysis tools in analysis.py:

1. Recurrence Quantification Analysis (RQA)

   • Recurrence rate, determinism, laminarity, divergence, entropy
   • Automatic threshold selection for target recurrence rates
   • Sparse matrix storage for efficiency

2. Cross‑Recurrence Quantification Analysis (cRQA)

   • Coordination analysis between different variables (e.g., vx vs vy)
   • Measures of synchronous behavior in dyadic interaction

3. Cross‑Wavelet Analysis

   • Time‑frequency coherence between signals
   • Phase synchronization analysis
   • Significance testing against surrogate data
   • Requires pycwt package (optional)

Key Database Features

1. Lazy Processing: Kinematics computed only when first accessed, then cached
2. Foreign Key Constraints: Ensures data integrity across tables
3. Comprehensive Indexing: Optimized for common query patterns
4. View Abstraction: Pre‑defined views for common analytical queries
5. Schema Evolution: Migration system for adding new analysis tables

Detailed DIMS Dashboard (docs/)

Overview

The docs/ directory contains a client-side DIMS dashboard for in-depth visualization of ORTHO games with eye-tracking data. This is a separate module that provides:

• Multi-perspective video visualization (wide, parent, child camera angles)
• Eye-tracking data integration with gaze visualization
• Advanced analyses: RQA (Recurrence Quantification Analysis) for gaze data
• Cross-wavelet analysis for velocity components
• Trajectory visualization with game path overlays
• Time-series synchronization across multiple modalities

Accessing the Detailed Dashboard

From the main ORTHO Explorer app:

1. Click the "🚀 Launch Detailed Dashboard" button in the sidebar
2. The dashboard opens in a new browser tab
3. If a session is selected, it will be passed to the detailed dashboard

Dashboard Features

Feature	Description
Video Visualization	Multiple camera perspectives with synchronized playback
Eye-Tracking	Gaze data visualization and RQA analysis
Trajectory Overlay	Game path visualization on video frames
Time-Series Plots	Synchronized plots of velocity, gaze, and other metrics
RQA Analysis	Recurrence Quantification Analysis for nonlinear dynamics
Cross-Wavelet	Time-frequency coherence analysis between signals

File Structure

docs/
├── index.html                    # Main dashboard HTML
├── config.json                   # Dashboard configuration
├── css/reset.css                 # Styles
├── js/app.js                     # Main application logic
├── js/video-component.js         # Video player component
├── optional_step_RQA.py          # RQA analysis script
├── optional_step_crosswavelet.py # Cross-wavelet analysis script
├── run_all_steps.sh             # Batch processing script
└── assets/                       # Data assets
    ├── videos/                   # Video files
    ├── images/                   # Game path images
    ├── timeseries/               # CSV time-series data
    └── transcripts/              # Video transcripts

Running Standalone

cd /home/m11/codes/DIMS_ORTHO_VIEWER/docs

# Open in browser
open index.html  # macOS
xdg-open index.html  # Linux
start index.html  # Windows

# Or run via Python HTTP server
python3 -m http.server 8000
# Then open: http://localhost:8000

Project Structure

DIMS_ORTHO_VIEWER/
├── app.py                    # Main Dash dashboard application
├── ortho_db.py              # Core database class (CRUD + kinematics)
├── ortho_db_new.py          # Extended database with advanced analyses
├── analysis.py              # RQA, cRQA, cross‑wavelet analysis functions
├── importer.py              # JSON → SQLite data importer
├── import_data.py           # Simple data import script (Python)
├── import_data.sh           # Data import script (Bash)
├── schema.sql               # Complete database schema (15+ tables)
├── ortho.db                 # Pre‑populated database (104 sessions) - NOT in git
├── run.sh                   # Application launcher script
├── requirements.txt         # Python dependencies
├── module.yaml              # DIMS plugin metadata
├── DATA_STRUCTURE.md        # Detailed data format documentation
├── IMPLEMENTATION_PLAN.md   # Project architecture and roadmap
├── README.md                # This file
├── .gitignore               # Git ignore rules (excludes *.db, logs_*.csv, etc.)
├── logs/                    # ORTHO data files (tracked in git)
│   ├── ORTHO-Data.json     # Complete JSON data
│   ├── ORTHO-Data.csv      # Complete CSV data  
│   ├── ORTHO-Data-Light.csv # Lightweight CSV
│   └── ORTHO-DataFrame.json # DataFrame JSON
├── docs/                    # Detailed DIMS dashboard with eye-tracking
│   ├── index.html          # Main dashboard HTML
│   ├── config.json         # Dashboard configuration
│   ├── ReadMe.MD           # Dashboard documentation
│   ├── css/reset.css       # Styles
│   ├── js/app.js           # Main application logic
│   ├── js/video-component.js # Video player component
│   ├── optional_step_RQA.py # RQA analysis script
│   ├── optional_step_crosswavelet.py # Cross-wavelet analysis
│   ├── run_all_steps.sh    # Batch processing script
│   └── assets/             # Data assets (videos, images, timeseries)
└── agents/                  # Multi‑agent simulation system
    ├── ortho_two_agents.py      # Main simulation with live reward editor
    ├── ortho_agent.py           # Abstract base agent + Q‑learning
    ├── ortho_engine.py          # Game engine and physics
    ├── ortho_renderer.py        # Visualization layer
    ├── ortho_runner.py          # Experiment runner
    ├── ortho_data_analysis.py   # Post‑experiment analysis
    ├── ortho_unsupervised_agent.py  # Self‑supervised variant
    ├── visualize_ortho_path.py  # Path visualization
    ├── README.md                # Agent system documentation
    ├── games_config.json        # Game scenario configurations
    └── Paths/                   # Pre‑defined maze paths (JSON)
        ├── path1_spiral.json
        ├── path2_s_curve.json
        └── ...

Development

Python Environment

conda activate dims
pip install dash plotly pandas numpy  # For dashboard
pip install pygame numpy              # For agent simulations

Extending the System

1. New Analysis Modules: Extend OrthoDatabase class and add new tabs in app.py
2. New Agent Strategies: Inherit from BaseAgent in agents/ortho_agent.py
3. New Visualization Types: Add callback functions in app.py with new plot layouts
4. Database Schema Updates: Modify schema.sql and implement migrations in ortho_db.py

Data Sources

• Original ORTHO logs: /home/m11/codes/DIMS_CORE/ortho/ORTHO‑logs‑Herman/
• CNK‑ORTHO dataset: 104 sessions, 1,049 tracks, 292k trajectory points
• No gaze data included (Herman logs lack eye‑tracking recordings)

Notes

• Participant Labels: X_axis and Y_axis are arbitrary labels, NOT fixed roles (parent/child)
• Coordinate System: Integer grid positions (0‑1800 pixels), velocity in "units per second"
• Date Handling: Stored as YYYYMMDD strings; filters convert to YYYY‑MM‑DD
• Lazy Processing: Kinematics computed only when first accessed, then cached
• Performance: ~1‑2 seconds for initial kinematics computation per session

Troubleshooting

Common Issues

1. Database Connection Errors

   # Check if database file exists
   ls -la ortho.db
   
   # Recreate from schema if needed
   rm ortho.db
   sqlite3 ortho.db < schema.sql

2. Missing Dependencies

   # Install all requirements
   pip install -r requirements.txt
   
   # For agent simulations
   pip install pygame numpy

3. Port Already in Use

   # Use a different port
   python app.py --port 8051
   
   # Find and kill process using port 8050
   lsof -ti:8050 | xargs kill -9

4. Agent Simulation Issues

   • Ensure pygame is installed: pip install pygame
   • Check Python version (requires 3.8+)
   • Verify display is available for PyGame (or use headless mode)

Data Import

To import new ORTHO session data:

python -c "
from importer import import_json_to_db
import_json_to_db('ORTHO-Data.json', 'ortho.db')
"

Research Context

This project supports research on:

• Interpersonal coordination in dyadic gameplay
• Embodied cognition through tabletop interaction
• Movement synchrony analysis using RQA and cross‑wavelet methods
• Multi‑agent coordination emergence in split‑control tasks
• Mathematics education through embodied design (ORTHO project)

Based on the ORTHO research:

Potega vel Żabik, K., Abrahamson, D., & Iłowiecka‑Tańska, I. (2024).
It Takes Two to OЯTHO: A Tabletop Action‑Based Embodied Design for the Cartesian System.
Digital Experiences in Mathematics Education, 10, 189–201.

Future Work

Planned Features

1. Gaze Data Integration: Incorporate eye‑tracking data from available sessions
2. Real‑time Analysis: Stream processing for live data visualization
3. Machine Learning: Predictive models for coordination success
4. Extended Agent Models: Deep reinforcement learning agents
5. Multi‑modal Analysis: Combine movement, gaze, and audio data

Integration Opportunities

• DIMS Core: Full integration with DIMS dashboard ecosystem
• ELAN Export: Direct export to ELAN annotation format
• MATLAB/EEGLAB: Compatibility with neuroscience analysis pipelines
• Open Science: Data sharing formats for reproducibility

Screenshot

---

Repo: github.com/mikub97/DIMS_ORTHO_VIEWER (private)
Local path: /home/m11/codes/DIMS_ORTHO_VIEWER/
Live: http://localhost:8050 (when running)