📜 Papyrus-scripts

A Python library for working with Papyrus, a large-scale curated dataset of bioactivity data aimed at machine learning applications. It handles downloading, versioning, and caching the dataset, and provides a fluent API to filter, search, and export exactly the subset you need — without ever loading the full dataset into memory.

✨ Features

• 🗂️ Versioned dataset access — download and cache specific Papyrus releases from Zenodo or 4TU, with automatic integrity checks.
• 🔗 Fluent filtering API — chain quality, source, organism, protein-class and molecular filters over a lazy Polars pipeline; only what you keep gets materialized.
• 🧬 Proteins, structures & descriptors — retrieve matching UniProt targets, 2D/3D compound structures, and precomputed molecular/protein descriptors in one call.
• 🔍 Similarity & substructure search — build an indexed FPSim2/RDKit search database over the dataset, with CPU, GPU (CUDA), or auto-fallback search engines.
• 🧠 DNN-ready — train PyTorch-based QSAR/PCM models on curated subsets, with y-scrambling and repeated seeds built in.
• 💻 CLI included — download, convert and clean up dataset files without writing any Python.
• 🔄 Format-friendly — transparent LZMA ↔ Gzip conversion and Parquet caching for tools that don't handle .xz.

📦 Installation

pip install papyrus-scripts

⚠️ Troubleshooting pip installation

If you see Defaulting to user installation because normal site-packages is not writeable followed by import errors, reinstall with:

pip uninstall -y papyrus-scripts
python -m pip install papyrus-scripts

Optional extras enable additional functionality:

Extra	Enables
papyrus-scripts[subsim]	CPU similarity & substructure search (tables, FPSim2)
papyrus-scripts[gpu]	GPU-accelerated similarity search (cupy-cuda12x[ctk], bundles the CUDA 12.x toolkit)
papyrus-scripts[simgpu]	CPU + GPU-accelerated similarity search (subsim + gpu)
papyrus-scripts[dnn]	DNN model training (torch, skorch)
papyrus-scripts[all]	Everything above

Note: openbabel (needed only for FP2/FP3/FP4 fingerprints) must be installed via conda-forge, not pip, when used alongside RDKit/FPSim2/cupy in the same environment: conda install -c conda-forge openbabel.

Note: the gpu extra pins cupy-cuda12x[ctk], whose ctk extra pulls in the CUDA 12.x toolkit as a pip package — no separate toolkit install needed, only an NVIDIA driver compatible with CUDA 12.x.

🛠️ Requirements

• Python 3.11+
• RDKit

💡 Usage

Quickstart: the object-oriented API

The recommended way to interact with the dataset. It downloads and caches data automatically as needed.

from papyrus_scripts import PapyrusDataset

dataset = PapyrusDataset(version='2024.09.2', plusplus=True)

filtered = (dataset
            .keep_source(['chembl', 'sharma'])
            .keep_quality('high'))

df = filtered.to_dataframe()
proteins = filtered.proteins().to_dataframe()

Downloading data: the CLI

# Download Papyrus++ bioactivities & targets for the latest version
papyrus download -V latest

# Download the full (all-quality) dataset with all precomputed descriptors for a specific revision
papyrus download -V 2022.11.3 --more -d all

# Download Papyrus++ data & compound structures for two versions
papyrus download -V 2022.11.3 -V 2022.04.2 -S

papyrus download --help

By default, data is downloaded to pystow's home directory; override it with -o.

Legacy functional API

from papyrus_scripts import (read_papyrus, read_protein_set,
                              keep_quality, keep_source, consume_chunks)

chunks = read_papyrus(version='2024.09.2', plusplus=True, chunksize=1_000_000)
proteins = read_protein_set(version='2024.09.2')

filtered = keep_quality(keep_source(chunks, source=['chembl', 'sharma']), min_quality='high')
df = consume_chunks(filtered)

Similarity & substructure search

from papyrus_scripts.subsim_search import FPSubSim2

fpss = FPSubSim2()
fpss.create_from_papyrus(version='2024.09.2', njobs=-1)  # builds a search database using all CPU cores

# cuda=False (default, CPU) | True (GPU, raises if unavailable) | 'auto' (GPU with CPU fallback)
engine = fpss.get_similarity_lib(cuda='auto')
hits = engine.similarity('CCO', threshold=0.7)

sub_lib = fpss.get_substructure_lib()
matches = sub_lib.substructure('c1ccccc1')

📊 Dataset versions

Papyrus releases are hosted on Zenodo:

Papyrus version	Revisions	Legacy alias	ChEMBL version	Zenodo
2022.04	2	05.4	29	link
2022.08	3	05.5	30	link
2022.11	4	05.6	31	link
2024.09	2	05.7	34	link

Note: for machine learning use cases, we recommend the datasets without stereochemistry (files marked "2D" and/or "without_stereochemistry").

⚙️ Advanced utilities

Compression conversion

Data is distributed as LZMA-compressed files (.xz), which some tools (e.g. Pipeline Pilot) don't support. Convert to Gzip (or back) without manually decompressing:

papyrus convert -v latest

Removing downloaded data

papyrus clean --remove_root

from papyrus_scripts import remove_papyrus

remove_papyrus(papyrus_root=True)

📚 Learn more

• simple_examples.ipynb
• matchRCSB.ipynb — matching Papyrus data against the Protein Data Bank
• advanced_querying.ipynb
• To reproduce the models, extraction pipeline and visualizations from the original publication, see Papyrus-modelling.

🖋️ Citation

If you use papyrus-scripts or the Papyrus dataset in your research, please cite:

@article{Bequignon2023Papyrus,
  title={Papyrus - A large scale curated dataset aimed at bioactivity predictions},
  author={B{\'e}quignon, Olivier J.M. and Bongers, Bart J. and Jespers, Willem and IJzerman, Adriaan P. and van de Water, Bob and van Westen, Gerard J.P.},
  journal={Journal of Cheminformatics},
  volume={15},
  number={3},
  year={2023},
  doi={10.1186/s13321-022-00672-x}
}

📄 License

This project is licensed under the MIT License.