AniFlow

Turn manhwa & webtoon chapters into narrated recap videos — fully automated.

AniFlow downloads manhwa/webtoon chapters, detects and crops panels using YOLO, extracts speech-bubble text, generates AI narration, converts it to speech, and renders video clips — all through a browser-based UI.

✨ Features

🌐 Web Scraper — download chapters from manhuaus, manhwatop, MangaDex, and more
🔍 YOLO Panel Detection — custom-trained models detect panels and speech bubbles
🧹 Text Removal — automatically clean speech bubbles from art panels
🤖 AI Narration — local (Ollama) or cloud (Gemini / Claude) narration
🎙️ Multi-TTS — Edge TTS (free), Kokoro (local), or ElevenLabs (cloud)
🎬 Video Rendering — Ken Burns animation, blur backgrounds, BGM, watermarks
👥 Character Casting — import from MyAnimeList or scan faces from chapters
📦 Batch Mode — process dozens of chapters unattended

Two Pipelines

	Dialogue Recap	Narrated Recap
Panels	Original panels with speech bubbles	Clean art panels (bubbles removed)
Audio	Extracted dialogue read aloud	AI-written YouTube-style recap narration
Best for	Letting the original dialogue shine	YouTube story recaps

📂 See it in action! Check the example_output/ folder for sample clips from both modes.

🎥 Video Tutorials

Tutorial	Link
💬 Dialogue Recap — Single Chapter	Watch on YouTube
💬 Dialogue Recap — Batch Mode (50+ Chapters)	Watch on YouTube
🎙️ Narrated Recap — Automatic Pipeline	Watch on YouTube
🎙️ Narrated Recap — Batch Mode (Full Series)	Watch on YouTube

📋 Requirements

Dependency	Purpose	Install
Python 3.10 – 3.12	Runtime	python.org
FFmpeg	Video/audio processing	`brew install ffmpeg` (macOS) · `sudo apt install ffmpeg` (Linux) · ffmpeg.org (Windows)
9router / Ollama	Local AI narration & text extraction	9router / ollama.com
~10 GB disk	Python env + model weights	—

GPU recommended but not required. Apple Silicon Macs and NVIDIA GPUs will significantly speed up panel detection and Kokoro TTS.

🚀 Quick Start

Option A: One-Command Setup (Recommended)

git clone https://github.com/aashish254/Aniflow.git
cd AniFlow
chmod +x setup.sh
./setup.sh

The setup script will check dependencies, create a virtual environment, install packages, pull Ollama models, and start the app.

Option B: Manual Setup

# 1. Clone the repository
git clone https://github.com/aashish254/Aniflow.git
cd AniFlow

# 2. Create a virtual environment (Python 3.12 recommended)
py -3.12 -m venv venv             # Windows
# python3 -m venv venv            # macOS / Linux

# 3. Activate virtual environment
# Windows (PowerShell):
.\venv\Scripts\Activate.ps1
# macOS / Linux:
# source venv/bin/activate

# 4. Install Python dependencies (~10-20 min, torch is large)
pip install --upgrade pip
pip install -r requirements.txt

# 5. Configure 9router (Alternative to Ollama)
# If you use 9router, set these environment variables before running:
# Windows (PowerShell):
# $env:USE_NINE_ROUTER="1"
# $env:NINE_ROUTER_HOST="http://127.0.0.1:20128"
# $env:VISION_MODEL="Free-Tier"
# $env:WRITER_MODEL="Free-Tier"
# $env:TEXT_WRITER_MODEL="Free-Tier"

# 6. Pull Ollama models (Skip if using 9router)
# ollama pull qwen2.5vl:3b
# ollama pull qwen2.5:14b

# 7. Run the app
python app.py

# 8. Open UI in browser:
# http://localhost:8080

🇮🇩 Panduan Bahasa Indonesia

📋 Persyaratan

Dependensi	Fungsi	Instalasi
Python 3.10 – 3.12	Runtime	python.org
FFmpeg	Pemrosesan video/audio	`brew install ffmpeg` (macOS) · `sudo apt install ffmpeg` (Linux) · ffmpeg.org (Windows)
9router / Ollama	AI lokal untuk narasi & ekstraksi teks	9router / ollama.com
~10 GB disk	Python env + model weights	—

GPU disarankan tapi tidak wajib. Apple Silicon Mac dan GPU NVIDIA akan sangat mempercepat deteksi panel dan Kokoro TTS.

🚀 Cara Menjalankan Aplikasi (Windows & 9router)

Langkah 1: Pastikan 9router Aktif

Pastikan 9router sudah berjalan di latar belakang (port default: http://127.0.0.1:20128).

Langkah 2: Jalankan di PowerShell

# 1. Masuk folder proyek
cd D:\Aniflow

# 2. Aktifkan virtual environment
.\venv\Scripts\Activate.ps1

# 3. Set environment variables untuk 9router
$env:USE_NINE_ROUTER="1"
$env:NINE_ROUTER_HOST="http://127.0.0.1:20128"
$env:VISION_MODEL="Free-Tier"
$env:WRITER_MODEL="Free-Tier"
$env:TEXT_WRITER_MODEL="Free-Tier"

# (Opsional) Jika 9router butuh API Key
# $env:NINE_ROUTER_API_KEY="API_KEY_ANDA"

# 4. Jalankan server AniFlow
python app.py

Langkah 3: Buka Browser

Akses antarmuka web di:

http://localhost:8080

⚙️ Configuration

setup.bat


Or follow the manual steps above using `venv\Scripts\activate` instead.

---

## 🎬 How to Use

### Narrated Recap (Recommended)

1. **Source** — Paste a manhwa URL → **Analyze** → pick chapters → **Download**
   (or point to a local folder of images)
2. **Cast** *(optional)* — Import characters from MyAnimeList, or scan chapter faces
   and assign names. The narrator uses real names ("Jin-Woo…") instead of "the dark-haired guy."
3. **Stitch** — Stitch → Detect (bubble panels) → Crop → Stitch → Detect (art panels) → Crop.
   Boxes are editable — drag, resize, or draw new ones.
4. **Review** — Check cropped panels; delete junk or re-crop.
5. **Extract** — AI reads speech-bubble text.
6. **Narrate** — Choose your AI backend:
   - 🦙 **Ollama (local)** — free & offline. `qwen2.5vl:7b` (vision) or `qwen2.5:14b` (text-only).
   - ✨ **Gemini API** — paste multiple API keys (auto-rotation on quota). Free keys at [aistudio.google.com](https://aistudio.google.com).
   - 🤖 **Claude API** — single key.
7. **Review Text** — Edit any narration line or re-narrate individual panels.
8. **Audio** — TTS with Edge (free), Kokoro (local), or ElevenLabs.
9. **Video** — Renders clips with Ken Burns animation, blur backgrounds, BGM.
10. **Review Clips / Sort Clips** — Preview, download, or batch-merge.

### Dialogue Recap

Same flow minus the cast/narrate steps — extracted dialogue becomes the audio,
panels keep their original speech bubbles.

### Batch Mode

Download chapters 1–68, enable **Batch Mode**, hit **Run Full Batch** — every step
runs across all chapters automatically. Progress is streamed live.

---

## 📁 Project Structure

AniFlow/ ├── app.py # Flask web app + API routes ├── config.py # All configuration & path settings ├── requirements.txt # Python dependencies ├── setup.sh / setup.bat # One-command setup scripts ├── start.sh # Quick-start script (macOS/Linux) │ ├── pipeline/ # Core processing modules │ ├── web_scraper.py # Chapter downloading (multi-site) │ ├── stitcher.py # Image stitching │ ├── panel_extractor_yolo.py # YOLO panel detection │ ├── cropper.py # Panel cropping │ ├── text_extractor.py # OCR / text extraction │ ├── text_remover.py # Speech bubble cleaning │ ├── narrator.py # Local AI narration (Ollama) │ ├── gemini_narrator.py # Cloud narration (Gemini API) │ ├── scene_narrator.py # Scene-level narration │ ├── tts_engine.py # Edge TTS engine │ ├── kokoro_tts.py # Kokoro local TTS │ ├── elevenlabs_tts.py # ElevenLabs cloud TTS │ ├── compositor.py # Video rendering │ ├── orchestrator.py # Pipeline orchestration │ ├── cast_system.py # Character management │ ├── character_builder.py # Face detection & grouping │ ├── mal_character_importer.py # MAL character import │ ├── script_rewriter.py # Narration style rewriter │ ├── capcut_director.py # CapCut export │ └── panel_filter.py # Panel quality filtering │ ├── models/ # YOLO model weights (committed) │ ├── manhwa_panel_cropped.pt # Art panel detector │ └── panel_with_bubble_text.pt # Bubble panel detector │ ├── templates/ # HTML templates ├── static/ # CSS, JS, logos ├── assets/ # BGM, watermarks, intros, outros └── tests/ # Test & diagnostic scripts


### Generated Folders (gitignored)

| Folder | Contents |
|---|---|
| `manhwa_download/` | Downloaded chapter images + per-chapter state |
| `manhwa_stitch/` | Stitched full-page strips |
| `manhwa_detect/` | Detection metadata (bounding boxes) |
| `manhwa_crop/` | Cropped individual panels |
| `manhwa_text/` | Extracted dialogue JSON |
| `manhwa_audio/` | Generated TTS audio files |
| `manhwa_clips/` | Final rendered video clips |
| `output/` | Legacy output directory |
| `~/RECAP/Cast Data/` | Your character casts (faces + profiles) |

All generated data is **regenerable** — delete any folder and re-run the pipeline.

---

## ⚙️ Configuration

### API Keys (Optional)

Copy `.env.example` to `.env` and fill in keys only for services you want to use:

```bash
cp .env.example .env

Key	Service	Required?
`GEMINI_API_KEY`	Google Gemini narration	Only if using Gemini
`ANTHROPIC_API_KEY`	Claude narration	Only if using Claude
`ELEVENLABS_API_KEY`	ElevenLabs TTS	Only if using ElevenLabs
`MAL_CLIENT_ID`	MyAnimeList import	Only if importing characters from MAL

Ollama + Edge TTS work completely offline — no API keys needed for the core pipeline.

Ollama Models

Model	Role	Size	Required?
`qwen2.5vl:3b`	Vision parser (reads panels)	~2 GB	✅ Required
`qwen2.5:14b`	Text-only narrator	~9 GB	📌 Recommended
`qwen2.5vl:7b`	Vision narrator (sees panels + cast)	~5 GB	Optional

🔧 Troubleshooting

Problem	Solution
"Address already in use"	Another instance is running: `lsof -ti :8080 \| xargs kill -9`
Ollama dot is red	Start Ollama and pull models: `ollama pull qwen2.5vl:3b`
Downloads hang	Protected site — the app auto-switches to headless browser (~40s). MangaDex URLs are most reliable.
Bubbles not removed	EasyOCR inpainter handles clear bubbles; complex ones can be fixed with the crop tool in Review.
`ModuleNotFoundError`	Make sure your venv is activated: `source venv/bin/activate`
Torch install fails	Try: `pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu`

🙏 Acknowledgments & Credits

AniFlow is built on the shoulders of many incredible open-source projects. Huge thanks to:

AI & Machine Learning

Ollama — Local LLM runtime for AI narration and text extraction
Qwen 2.5 VL by Alibaba Cloud — Vision-language model for panel understanding
Ultralytics YOLOv8 — Object detection framework for panel detection
Magi v2 by Ragav Sachdeva — Manga/comic page understanding and speaker-aware transcription
EasyOCR by JaidedAI — Text detection for speech bubble cleaning
Manga OCR by kha-white — OCR tuned for manga/manhwa text
Google Gemini API — Cloud AI narration alternative
Anthropic Claude API — Cloud AI narration alternative

Text-to-Speech

Edge TTS by rany2 — Free Microsoft Edge TTS (default engine)
Kokoro TTS by hexgrad — High-quality local text-to-speech
ElevenLabs — Premium cloud TTS

Video & Image Processing

MoviePy by Zulko — Python video editing
FFmpeg — The backbone of all media processing
OpenCV — Computer vision operations
Pillow — Image manipulation

Web Scraping

Beautiful Soup by Leonard Richardson — HTML parsing
curl_cffi — Cloudflare-friendly HTTP with TLS fingerprinting
CloudScraper by VeNoMouS — Cloudflare challenge bypass
DrissionPage by g1879 — Headless browser fallback for protected sites

Web Framework

Flask — Python web framework
Flask-SocketIO — Real-time progress streaming

Other

lbpcascade_animeface by nagadomi — Anime/manga face detection cascade
PyTorch — Deep learning framework
Hugging Face Transformers — Model loading for Magi v2
MyAnimeList — Character data source for the cast system

📜 License

This project is licensed under the MIT License.

🤝 Contributing

Contributions are welcome! See CONTRIBUTING.md for guidelines.

Made with ❤️ for the manhwa community

Navigation Menu

Repository files navigation

AniFlow

✨ Features

Two Pipelines

🎥 Video Tutorials

📋 Requirements

🚀 Quick Start

Option A: One-Command Setup (Recommended)

Option B: Manual Setup

🇮🇩 Panduan Bahasa Indonesia

📋 Persyaratan

🚀 Cara Menjalankan Aplikasi (Windows & 9router)

Langkah 1: Pastikan 9router Aktif

Langkah 2: Jalankan di PowerShell

Langkah 3: Buka Browser

⚙️ Configuration

Ollama Models

🔧 Troubleshooting

🙏 Acknowledgments & Credits

AI & Machine Learning

Text-to-Speech

Video & Image Processing

Web Scraping

Web Framework

Other

📜 License

🤝 Contributing

About

Releases

Packages

Contributors

Languages

Name		Name	Last commit message	Last commit date
.github/ISSUE_TEMPLATE		.github/ISSUE_TEMPLATE
assets		assets
casts		casts
example_output		example_output
input/raw_manhwa		input/raw_manhwa
models		models
pipeline		pipeline
projects		projects
static		static
temp		temp
templates		templates
tests		tests
.env.example		.env.example
.gitignore		.gitignore
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
README.md		README.md
SECURITY.md		SECURITY.md
app.py		app.py
config.py		config.py
requirements.txt		requirements.txt
setup.bat		setup.bat
setup.sh		setup.sh
start.sh		start.sh

Latest commit

History

Folders and files

Repository files navigation

AniFlow

✨ Features

Two Pipelines

🎥 Video Tutorials

📋 Requirements

🚀 Quick Start

Option A: One-Command Setup (Recommended)

Option B: Manual Setup

🇮🇩 Panduan Bahasa Indonesia

📋 Persyaratan

🚀 Cara Menjalankan Aplikasi (Windows & 9router)

Langkah 1: Pastikan 9router Aktif

Langkah 2: Jalankan di PowerShell

Langkah 3: Buka Browser

⚙️ Configuration

Ollama Models

🔧 Troubleshooting

🙏 Acknowledgments & Credits

AI & Machine Learning

Text-to-Speech

Video & Image Processing

Web Scraping

Web Framework

Other

📜 License

🤝 Contributing

About

Topics

Resources

Contributing

Security policy

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages