Skip to content

Latest commit

 

History

History
322 lines (262 loc) · 14.9 KB

File metadata and controls

322 lines (262 loc) · 14.9 KB

TMDB-Import

English 简体中文 DeepWiki

A powerful Python tool for extracting TV series metadata from 40+ streaming platforms and automatically importing it into The Movie Database (TMDB). Supports metadata extraction, image processing, and TMDB import with browser automation.

This script uses the Playwright automation framework and supports Chrome/Chromium browsers. Playwright automatically downloads and manages browsers, eliminating the need for manual driver installation.

Table of Contents

Installation

Install Dependencies

Full Installation (All Features)

pip install -r requirements.txt
playwright install chromium

Minimal Installation (Core Features Only)

pip install python-dateutil

This minimal installation supports most extractors that don't require browser automation.

Optional Dependencies

Dependencies are loaded on-demand. The program will show helpful error messages when a feature requires a missing package:

  • Browser Automation (for sites like Youku, IQIYI that require JavaScript rendering)

    pip install playwright
    playwright install chromium
  • Chinese Variant Conversion (Simplified ↔ Traditional)

    pip install opencc-python-reimplemented

    Enable in config.ini by setting chinese_convert = zh-CN (or zh-TW, zh-HK)

  • Image Processing (for backdrop/poster cropping and format conversion)

    pip install Pillow bordercrop

Individual Package Installation

pip install playwright
pip install python-dateutil
pip install Pillow
pip install bordercrop
pip install opencc-python-reimplemented
playwright install chromium

Usage

Command Line Options

  • -h, --help: Show help information
  • -V, --version: Show version information
  • -d, --debug: Enable debug logging (default is INFO level)
  • --headless: Run browser in headless mode (default is GUI mode)

Basic Usage

python -m tmdb_import [options] "URL"

Browser Mode Description

  • GUI Mode (default): Browser window is visible, suitable for debugging and scenarios requiring manual interaction
  • Headless Mode (--headless): Browser runs in the background without displaying a window, suitable for automation scripts and server environments
  • Headless mode can improve performance and reduce resource usage, especially suitable for batch processing tasks

Extract Episode Data

python -m tmdb_import "http://www.***.com/video.html"
python -m tmdb_import -d "http://www.***.com/video.html"  # Enable debug logging
python -m tmdb_import --headless "http://www.***.com/video.html"  # Run in headless mode
python -m tmdb_import -d --headless "http://www.***.com/video.html"  # Debug + headless mode
python -m tmdb_import "http://www.***.com/video.html" --debug  # Options can also be placed after URL
  • Extract episode data through web links, including title, plot description, duration, release time (mostly current platform time), and background image links.
  • Output files:
    • metadata.json: Full structured metadata (show-level fields, season-level fields, episode list by season)
    • import.csv: Flattened episode list for TMDB import

Import Data to TMDB

python -m tmdb_import "https://www.themoviedb.org/tv/{tv_id}/season/{season_number}?language={language}"
# Example: python -m tmdb_import "https://www.themoviedb.org/tv/203646/season/1?language=zh-CN"
# Enable debug: python -m tmdb_import -d "https://www.themoviedb.org/tv/203646/season/1?language=zh-CN"
# Headless mode: python -m tmdb_import --headless "https://www.themoviedb.org/tv/203646/season/1?language=zh-CN"
# Combined options: python -m tmdb_import -d --headless "https://www.themoviedb.org/tv/203646/season/1?language=zh-CN"
  • Import data from import.csv in the directory to TMDB. When uploading backdrop images, automatically crop black borders and adapt to the aspect ratio required by TMDB. On first run, manual login is required (or set tmdb_username and tmdb_password in config.ini for automatic login). See Configuration for more options.

Image Processing

python -m tmdb_import backdrop "https://www.***.com/image.jpg"
python -m tmdb_import --headless backdrop "https://www.***.com/image.jpg"  # Process backdrop in headless mode
  • Crop backdrop images to fit TMDB requirements
python -m tmdb_import poster "https://www.***.com/image.jpg"
python -m tmdb_import --headless poster "https://www.***.com/image.jpg"  # Process poster in headless mode
  • Crop poster images to fit TMDB requirements
python -m tmdb_import fitsize width*height "https://www.***.com/image.jpg"
python -m tmdb_import --headless fitsize 1920*1080 "https://www.***.com/image.jpg"  # Crop in headless mode
  • Crop images according to specified width and height

Using as a Python Library

TMDB-Import can also be used as a library in your Python projects:

# Add to sys.path (if not installed via pip)
import sys
sys.path.insert(0, r'/path/to/TMDB-Import')

# Import directly (package uses underscore now)
from tmdb_import import extract_from_url, save_metadata_json, create_csv

# Extract metadata
metadata = extract_from_url("https://tver.jp/series/...")

# Access metadata
print(metadata.title)       # Show title
print(metadata.overview)    # Show overview
print(metadata.poster)      # Poster URL
print(metadata.language)    # Language code (e.g., 'ja-JP')

# Access episodes
for season in metadata.seasons:
    for episode_num, episode in season.episodes.items():
        print(f"E{episode_num}: {episode.name}")

# Optional: Save files manually
save_metadata_json("output.json", metadata)
for season in metadata.seasons:
    if season.episodes:
        create_csv("output.csv", season.episodes)
        break

Available Functions:

  • extract_from_url(url, language="zh-CN"): Extract and process metadata from URL
  • save_metadata_json(filename, metadata): Save metadata to JSON file
  • create_csv(filename, episodes_dict): Save episodes to CSV file
  • import_to_tmdb(url, username, password): Import data to TMDB

Metadata Structure:

  • Metadata: Show-level data (title, overview, poster, backdrop, logo, language, seasons)
  • Season: Season-level data (season_number, name, overview, poster, episodes)
  • Episode: Episode data (episode_number, name, air_date, runtime, overview, backdrop)

Configuration

The config.ini file in the working directory controls the behaviour of the script. All settings are placed under the [DEFAULT] section.

Key Default Description
encoding utf-8-sig CSV file encoding (e.g. utf-8, utf-8-sig, gbk)
save_user_profile true Persist the browser session under the Browser/ folder so you stay logged in between runs
tmdb_username (empty) TMDB account username for automatic login
tmdb_password (empty) TMDB account password for automatic login
backdrop_forced_upload false When true, upload a backdrop image even if one already exists on TMDB
backdrop_vote_after_upload false When true, automatically cast a thumbs-up vote on the newly uploaded backdrop
filter_words (empty) Comma-separated words; episodes whose titles contain any of these words are filtered out. Remaining episodes are automatically renumbered while preserving gaps (e.g., if episodes 1,2,3,4,5,6,10 exist and episodes 2,4 are filtered, result will be 1,2,3,8 to preserve the gap after episode 6)
rename_csv_on_import false When true, rename import.csv to import_{tmdb_id}_s{season}_{language}.csv before importing
delete_csv_after_import false When true, delete the CSV file after a successful import
chinese_convert (empty) Convert Chinese text variant after extraction. Leave empty to disable. Options: zh-CN (Simplified), zh-TW (Taiwan Traditional), zh-HK (Hong Kong Traditional). Only applied when the source language is Chinese (zh-*).

Example config.ini:

[DEFAULT]
encoding = utf-8-sig
save_user_profile = true
tmdb_username = your_username
tmdb_password = your_password
backdrop_forced_upload = false
backdrop_vote_after_upload = false
filter_words = 番外,加更
rename_csv_on_import = false
delete_csv_after_import = false
chinese_convert = 

Test Environment

Windows 11, Chrome/Chromium, Python 3, and Visual Studio Code.

Supported Platforms

Website Title Plot Duration Release Date Backdrop Default Language
anidb x x Follow site
apple zh-CN
asahi ja-JP
bilibili x zh-CN
biliintl x x Follow site
crunchyroll Follow site
cctv x zh-CN
fod x ja-JP
hbomax x Follow site
iqiyi x x x zh-CN
ixigua zh-CN
kktv x zh-TW
kocowa Follow site
linetv x x zh-TW
litv x x x zh-TW
mgtv x x zh-CN
migu x x zh-CN
mytvsuper zh-HK
myvideo x x zh-TW
netflix x x x Follow site
nhk x ja-JP
paravi x ja-JP
primevideo Follow site
ptsplus x zh-TW
qq x zh-CN
sohu zh-CN
tvdb x x Follow site
tvbanywhere zh-HK
tver ja-JP
viki x x en-US
viu zh-CN
yahoo x x Follow site
wavve ko-KR
youku zh-CN
youtube Follow site

Troubleshooting

Browser Issues

  • Error: "Playwright not found"

    • Solution: Run pip install playwright and playwright install chromium
    • Verify with: playwright install chromium
  • Headless mode crashes

    • Check your OS and Chromium version compatibility
    • Try GUI mode first: Remove --headless flag
    • Update Playwright: pip install --upgrade playwright
  • "Timeout" or "Page not loading"

    • Increase timeout in code or use --debug for detailed logs
    • Check your internet connection
    • The target site may have changed its structure

Metadata Extraction

  • Missing data (empty fields)

    • Check the "Supported Platforms" table for feature availability
    • Use --debug flag to see what data was extracted
    • The platform may not provide that information
  • Chinese conversion not working

    • Verify installation: pip install opencc-python-reimplemented
    • Ensure config.ini has correct chinese_convert value
    • Only works when source language is Chinese (zh-*)

TMDB Import

  • Login failure

    • Verify username/password in config.ini or enter manually
    • Check TMDB account is not locked
    • Enable 2FA if required by account settings
  • Image upload errors

    • Verify image format (JPEG, PNG supported)
    • Check image size meets TMDB requirements
    • backdrop_forced_upload = true to replace existing images

Image Processing

  • Image cropping produces blank results

    • Verify image URL is accessible
    • Try different image (may be corrupted)
    • Check image format compatibility
  • Black border detection not working

    • Increase border detection threshold in config
    • Some images may not have clear borders to detect

FAQ

Q: Do I need a TMDB account?

  • A: Only for the import function. Extraction works without an account.

Q: Can I use this without Playwright?

  • A: Yes, with minimal installation. Extraction-only sites don't need browser automation.

Q: Does this support TV shows with multiple languages?

  • A: Yes. Use the language parameter in the TMDB URL to specify the target language.

Q: How do I automate batch imports?

  • A: Use config.ini for automation settings and create a Python script that calls extract_from_url() in a loop.

Q: What if a site is not in the supported list?

  • A: You can create a custom extractor or request support on GitHub.

Q: Is there a GUI?

  • A: This is a command-line tool. The browser displays a GUI window in normal mode.

Q: How long does extraction typically take?

  • A: 5-30 seconds depending on site complexity and network speed. Enable --headless for faster performance.

Q: Can I contribute new platform support?

  • A: Yes! Contributions are welcome. Create a new extractor in tmdb_import/extractors/.

License

MIT License - See LICENSE file for details.