Add draft JOSS paper submission by samueljackson92 · Pull Request #364 · ukaea/toktagger

samueljackson92 · 2026-08-10T08:30:57Z

This PR is to track development of a draft JOSS submission. All are welcome to help edit.

Also add Niraj!

wk9874 · 2026-08-17T10:33:55Z

+
+# Summary
+
+Tokamak fusion experiments such as MAST, MAST-U and JET produce large volumes of multi-modal diagnostic data with every experimental shot, including scalar time-series signals (e.g., plasma current, density, D-alpha emission), camera video, and signal derived spectrograms. Understanding and controlling plasma behaviour depends on identifying physical events within this data, such as edge-localised modes (ELMs), transitions between low- and high-confinement regimes (L-mode/H-mode), disruptions, and other magnetohydrodynamic phenomena. Historically, these events have been identified manually by physicists on a shot-by-shot basis. As experiments scale in number and diagnostic bandwidth, manual labelling no longer scales, motivating the use of machine learning models trained on human-labelled examples to automate detection of these events. Building such models first requires a way to efficiently create, manage, and iterate on labelled datasets drawn directly from tokamak data systems. `TokTagger` is an open-source, web-based annotation platform that addresses this need, combining an interactive labelling interface for time-series, image, and video data with an extensible Python backend that connects directly to existing fusion data-access systems and machine learning workflows.


Should substantiate 'large volumes of data' with a statistic, either here or in the statement of need section. Could use:

JET produced 105,929 pulses with more than 10GB of raw data per pulse [1], and MAST produced 30,471 shots with 7GB per shot [2]

[1]: Vega, J., et al. "New developments at JET in diagnostics, real-time control, data acquisition and information retrieval with potential application to ITER." Fusion Engineering and Design 84.12 (2009): 2136-2144.
[2]: Jackson, Samuel, et al. "An open data service for supporting research in machine learning on tokamak data." IEEE Transactions on Plasma Science (2025).

wk9874 · 2026-08-19T15:58:40Z

+
+# Summary
+
+Tokamak fusion experiments such as MAST, MAST-U and JET produce large volumes of multi-modal diagnostic data with every experimental shot, including scalar time-series signals (e.g., plasma current, density, D-alpha emission), camera video, and signal derived spectrograms. Understanding and controlling plasma behaviour depends on identifying physical events within this data, such as edge-localised modes (ELMs), transitions between low- and high-confinement regimes (L-mode/H-mode), disruptions, and other magnetohydrodynamic phenomena. Historically, these events have been identified manually by physicists on a shot-by-shot basis. As experiments scale in number and diagnostic bandwidth, manual labelling no longer scales, motivating the use of machine learning models trained on human-labelled examples to automate detection of these events. Building such models first requires a way to efficiently create, manage, and iterate on labelled datasets drawn directly from tokamak data systems. `TokTagger` is an open-source, web-based annotation platform that addresses this need, combining an interactive labelling interface for time-series, image, and video data with an extensible Python backend that connects directly to existing fusion data-access systems and machine learning workflows.


combining an interactive labelling interface for time-series, image, and video data

Should read

combining an interactive labelling interface for time-series, 2D profile (or spectrogram?), and video data

wk9874 · 2026-08-19T16:02:55Z

+
+Tokamak fusion experiments such as MAST, MAST-U and JET produce large volumes of multi-modal diagnostic data with every experimental shot, including scalar time-series signals (e.g., plasma current, density, D-alpha emission), camera video, and signal derived spectrograms. Understanding and controlling plasma behaviour depends on identifying physical events within this data, such as edge-localised modes (ELMs), transitions between low- and high-confinement regimes (L-mode/H-mode), disruptions, and other magnetohydrodynamic phenomena. Historically, these events have been identified manually by physicists on a shot-by-shot basis. As experiments scale in number and diagnostic bandwidth, manual labelling no longer scales, motivating the use of machine learning models trained on human-labelled examples to automate detection of these events. Building such models first requires a way to efficiently create, manage, and iterate on labelled datasets drawn directly from tokamak data systems. `TokTagger` is an open-source, web-based annotation platform that addresses this need, combining an interactive labelling interface for time-series, image, and video data with an extensible Python backend that connects directly to existing fusion data-access systems and machine learning workflows.
+
+![The time-series labelling interface, showing multiple synchronised diagnostic signals with time-region annotations.\label{fig:timeseries}](time-series.png)


Could use this image instead:

Maybe looks a bit more engaging / dynamic?

Possibly too much detail for this stage of the paper, but you could mention that the Peak Detection tool was used to automatically draw the green and red lines, with green lines human validated as ELMs. Purple line manually added (not sure what that is, could be measurement error, could be a UFO / Transient Impurity Event causing big spike in radiated light?)

wk9874 · 2026-08-19T16:06:37Z

+
+# Statement of need
+
+Labelling tokamak diagnostic data for machine learning presents challenges that are not well served by general-purpose annotation software. Diagnostic signals are multi-variate and must be viewed and labelled in relation to one another (e.g., correlating a plasma current, density, and soft X-ray emission to identify an IRE), rather than as independent, single-channel time series. Data must also be sourced directly from experiment-specific data systems, such as UDA [@Muir:2015] for MAST and MAST-U, SAL [@sal] for JET, to support inter-shot analysis, rather than from static exported files. Without dedicated tooling, researchers typically resort to ad hoc scripts, spreadsheets, or one-off notebooks to track labels for a given study. These approaches do not scale across users or experiments, are not easily reproducible, and provide no direct route from a labelled dataset to a trained model.


Data must also be sourced directly from experiment-specific data systems, such as UDA [@muir:2015] for MAST and MAST-U, SAL [@sal] for JET, to support inter-shot analysis, rather than from static exported files.

This doesn't read well to me, maybe need to replace the second comma with an and/or?

Data must also be sourced directly from experiment-specific data systems, such as UDA [@Muir:2015] for MAST and MAST-U or [@sal] for JET, to support inter-shot analysis, rather than from static exported files.

wk9874 · 2026-08-19T16:07:16Z

+
+Labelling tokamak diagnostic data for machine learning presents challenges that are not well served by general-purpose annotation software. Diagnostic signals are multi-variate and must be viewed and labelled in relation to one another (e.g., correlating a plasma current, density, and soft X-ray emission to identify an IRE), rather than as independent, single-channel time series. Data must also be sourced directly from experiment-specific data systems, such as UDA [@Muir:2015] for MAST and MAST-U, SAL [@sal] for JET, to support inter-shot analysis, rather than from static exported files. Without dedicated tooling, researchers typically resort to ad hoc scripts, spreadsheets, or one-off notebooks to track labels for a given study. These approaches do not scale across users or experiments, are not easily reproducible, and provide no direct route from a labelled dataset to a trained model.
+
+`TokTagger` was designed to close this gap between labelling and modelling. It provides a project-based workflow in which a physicist defines a labelling task (time-series or video annotation), attaches one of several tokamak-aware data loaders, and then browses and labels samples through a web UI. Semi-automated "annotators" (e.g., peak, change-point, jump, and outlier detection) suggest candidate labels to speed up manual work, and machine learning models can be trained directly on the growing set of human annotations and used to automate the labelling process. `TokTagger` is intended to be used by both diagnostic physicists building one-off labelled datasets and by machine learning researchers building larger, actively-maintained training sets for automated event detection across MAST-U and JET.


(time-series or video annotation)

mention 2D profile

trained directly on the growing set of human annotations

Maybe 'human validated annotations' since you just mentioned the automated annotators doing the bulk of the labelling?

for automated event detection across MAST-U and JET.

This implies it is limited to just these two machines, maybe rephrase? 'tokamaks such as MAST-U or JET'?

wk9874 · 2026-08-19T16:20:06Z

+
+- **Data loaders** (`toktagger.api.core.data_loaders`) define how samples are retrieved from a data source and converted into one of `TokTagger`'s standard data schemas (time series, multi-variate time series, spectrogram, or image). Built-in loaders cover UDA, SAL, FAIR-MAST, local tabular files, and local image/Numpy array files. New loaders are registered with a single `@LoaderRegistry.register(...)` decorator.
+
+- **Annotators** provide algorithmic pre-labelling for time series: peak detection, outlier detection (Mean Absolute Deviation or Isolation Forest [@scikit-learn]), change-point detection (PELT or a Hidden Markov Model, via `ruptures` [@Truong:2020] and `hmmlearn`), and jump detection. These surface candidate annotations that a human reviewer can accept, adjust, or reject.


These surface candidate annotations

That feels like a weird phrasing - These produce / suggest candidate annotations?

wk9874 · 2026-08-19T16:21:08Z

+
+- **Models** (`@ModelRegistry.register(...)`) wrap arbitrary machine learning estimators (scikit-learn, PyTorch, or otherwise) behind a common `train`/`predict`/`save`/`load` interface, together with helpers for train/validation/test splitting and progress reporting back to the UI. Model training and inference are scheduled with Ray [@Moritz:2018], which allows CPU- and GPU-bound tasks to be distributed across worker nodes and enables the same code to run unmodified on a laptop or a shared cluster.
+
+Annotation data is represented with a small set of Pydantic schemas — time points, time regions, bounding boxes, polygons, and their per-frame video equivalents — that are shared between the UI, the database layer, and the model API, so that a prediction returned by a model is structurally identical to an annotation created by a human. Projects track a query strategy (sequential, random, or uncertainty-based) that determines which sample is served next, allowing model predictions to directly steer labelling effort towards the most informative samples.


Extremely long first sentence, consider splitting into two

... per-frame video equivalents. These are shared...

Projects track a query strategy

Again sounds a bit weird, rephrase? (not sure how - projects use / require / employ?)

wk9874 · 2026-08-19T16:23:28Z

+
+Annotation data is represented with a small set of Pydantic schemas — time points, time regions, bounding boxes, polygons, and their per-frame video equivalents — that are shared between the UI, the database layer, and the model API, so that a prediction returned by a model is structurally identical to an annotation created by a human. Projects track a query strategy (sequential, random, or uncertainty-based) that determines which sample is served next, allowing model predictions to directly steer labelling effort towards the most informative samples.
+
+\autoref{fig:timeseries} shows the time-series labelling interface, in which multiple synchronised diagnostic signals are displayed with shared time axes and can be labelled with time-point and time-region annotations, either manually or with the assistance of the automated annotators described above. An equivalent frame-by-frame interface, supporting bounding box, polygon, and point annotations with propagation between frames, is provided for camera and video data. \autoref{fig:architecture} summarises the overall system architecture connecting the UI, API, database, and model workers.


Should also mention spectrogram here

Can we have figures of the other view(s)?

with propagation between frames

with optional propagation?

wk9874 · 2026-08-19T16:24:39Z

+
+\autoref{fig:timeseries} shows the time-series labelling interface, in which multiple synchronised diagnostic signals are displayed with shared time axes and can be labelled with time-point and time-region annotations, either manually or with the assistance of the automated annotators described above. An equivalent frame-by-frame interface, supporting bounding box, polygon, and point annotations with propagation between frames, is provided for camera and video data. \autoref{fig:architecture} summarises the overall system architecture connecting the UI, API, database, and model workers.
+
+The project is tested with `pytest`, covering the core annotation and data loader logic, the REST API, and the MongoDB integration, together with end-to-end browser tests written with Playwright that exercise the UI directly. Continuous integration runs the test suite and `ruff` linting on every change.


Playwright should be in backticks

wk9874 · 2026-08-19T16:25:38Z

+
+# Research impact statement
+
+`TokTagger` was initially developed by the UK Atomic Energy Authority (UKAEA) and the Science and Technology Facilities Council (STFC) to support the creation of labelled datasets from MAST, MAST-U, and JET diagnostic data for machine learning research. Since its first public release (v0.1.0), the project has grown through contributions from a team of developers across UKAEA beyond its original authors, with new data loaders, annotation types, and model-training functionality added in each subsequent release. It is used internally at UKAEA to curate labelled datasets in support of machine learning models for tasks such as automated ELM and disruption detection from MAST-U and JET diagnostics, and its direct integration with UDA, SAL, and FAIR-MAST is intended to make it straightforward for other groups working with these data systems to build similar labelled datasets for their own machine learning workflows. As a young, actively-developed project, `TokTagger`'s broader adoption and citation record are still emerging; this paper accompanies its release as reusable, general-purpose infrastructure for the wider fusion data science community.


for tasks such as automated ELM and disruption detection

Could mention UFO detection as an example video interface use case?

samueljackson92 added 2 commits August 10, 2026 09:13

Add JOSS draft

9157422

Add readme

4ca7079

samueljackson92 added the documentation Improvements or additions to documentation label Aug 10, 2026

praksharma reviewed Aug 10, 2026

View reviewed changes

Comment thread joss/paper.md

Update Affiliations

121eb80

Also add Niraj!

wk9874 requested changes Aug 19, 2026

View reviewed changes


		# Summary

		Tokamak fusion experiments such as MAST, MAST-U and JET produce large volumes of multi-modal diagnostic data with every experimental shot, including scalar time-series signals (e.g., plasma current, density, D-alpha emission), camera video, and signal derived spectrograms. Understanding and controlling plasma behaviour depends on identifying physical events within this data, such as edge-localised modes (ELMs), transitions between low- and high-confinement regimes (L-mode/H-mode), disruptions, and other magnetohydrodynamic phenomena. Historically, these events have been identified manually by physicists on a shot-by-shot basis. As experiments scale in number and diagnostic bandwidth, manual labelling no longer scales, motivating the use of machine learning models trained on human-labelled examples to automate detection of these events. Building such models first requires a way to efficiently create, manage, and iterate on labelled datasets drawn directly from tokamak data systems. `TokTagger` is an open-source, web-based annotation platform that addresses this need, combining an interactive labelling interface for time-series, image, and video data with an extensible Python backend that connects directly to existing fusion data-access systems and machine learning workflows.


		Tokamak fusion experiments such as MAST, MAST-U and JET produce large volumes of multi-modal diagnostic data with every experimental shot, including scalar time-series signals (e.g., plasma current, density, D-alpha emission), camera video, and signal derived spectrograms. Understanding and controlling plasma behaviour depends on identifying physical events within this data, such as edge-localised modes (ELMs), transitions between low- and high-confinement regimes (L-mode/H-mode), disruptions, and other magnetohydrodynamic phenomena. Historically, these events have been identified manually by physicists on a shot-by-shot basis. As experiments scale in number and diagnostic bandwidth, manual labelling no longer scales, motivating the use of machine learning models trained on human-labelled examples to automate detection of these events. Building such models first requires a way to efficiently create, manage, and iterate on labelled datasets drawn directly from tokamak data systems. `TokTagger` is an open-source, web-based annotation platform that addresses this need, combining an interactive labelling interface for time-series, image, and video data with an extensible Python backend that connects directly to existing fusion data-access systems and machine learning workflows.

		![The time-series labelling interface, showing multiple synchronised diagnostic signals with time-region annotations.\label{fig:timeseries}](time-series.png)


		# Statement of need

		Labelling tokamak diagnostic data for machine learning presents challenges that are not well served by general-purpose annotation software. Diagnostic signals are multi-variate and must be viewed and labelled in relation to one another (e.g., correlating a plasma current, density, and soft X-ray emission to identify an IRE), rather than as independent, single-channel time series. Data must also be sourced directly from experiment-specific data systems, such as UDA [@Muir:2015] for MAST and MAST-U, SAL [@sal] for JET, to support inter-shot analysis, rather than from static exported files. Without dedicated tooling, researchers typically resort to ad hoc scripts, spreadsheets, or one-off notebooks to track labels for a given study. These approaches do not scale across users or experiments, are not easily reproducible, and provide no direct route from a labelled dataset to a trained model.


		Labelling tokamak diagnostic data for machine learning presents challenges that are not well served by general-purpose annotation software. Diagnostic signals are multi-variate and must be viewed and labelled in relation to one another (e.g., correlating a plasma current, density, and soft X-ray emission to identify an IRE), rather than as independent, single-channel time series. Data must also be sourced directly from experiment-specific data systems, such as UDA [@Muir:2015] for MAST and MAST-U, SAL [@sal] for JET, to support inter-shot analysis, rather than from static exported files. Without dedicated tooling, researchers typically resort to ad hoc scripts, spreadsheets, or one-off notebooks to track labels for a given study. These approaches do not scale across users or experiments, are not easily reproducible, and provide no direct route from a labelled dataset to a trained model.

		`TokTagger` was designed to close this gap between labelling and modelling. It provides a project-based workflow in which a physicist defines a labelling task (time-series or video annotation), attaches one of several tokamak-aware data loaders, and then browses and labels samples through a web UI. Semi-automated "annotators" (e.g., peak, change-point, jump, and outlier detection) suggest candidate labels to speed up manual work, and machine learning models can be trained directly on the growing set of human annotations and used to automate the labelling process. `TokTagger` is intended to be used by both diagnostic physicists building one-off labelled datasets and by machine learning researchers building larger, actively-maintained training sets for automated event detection across MAST-U and JET.


		- Data loaders (`toktagger.api.core.data_loaders`) define how samples are retrieved from a data source and converted into one of `TokTagger`'s standard data schemas (time series, multi-variate time series, spectrogram, or image). Built-in loaders cover UDA, SAL, FAIR-MAST, local tabular files, and local image/Numpy array files. New loaders are registered with a single `@LoaderRegistry.register(...)` decorator.

		- Annotators provide algorithmic pre-labelling for time series: peak detection, outlier detection (Mean Absolute Deviation or Isolation Forest [@scikit-learn]), change-point detection (PELT or a Hidden Markov Model, via `ruptures` [@Truong:2020] and `hmmlearn`), and jump detection. These surface candidate annotations that a human reviewer can accept, adjust, or reject.


		- Models (`@ModelRegistry.register(...)`) wrap arbitrary machine learning estimators (scikit-learn, PyTorch, or otherwise) behind a common `train`/`predict`/`save`/`load` interface, together with helpers for train/validation/test splitting and progress reporting back to the UI. Model training and inference are scheduled with Ray [@Moritz:2018], which allows CPU- and GPU-bound tasks to be distributed across worker nodes and enables the same code to run unmodified on a laptop or a shared cluster.

		Annotation data is represented with a small set of Pydantic schemas — time points, time regions, bounding boxes, polygons, and their per-frame video equivalents — that are shared between the UI, the database layer, and the model API, so that a prediction returned by a model is structurally identical to an annotation created by a human. Projects track a query strategy (sequential, random, or uncertainty-based) that determines which sample is served next, allowing model predictions to directly steer labelling effort towards the most informative samples.


		Annotation data is represented with a small set of Pydantic schemas — time points, time regions, bounding boxes, polygons, and their per-frame video equivalents — that are shared between the UI, the database layer, and the model API, so that a prediction returned by a model is structurally identical to an annotation created by a human. Projects track a query strategy (sequential, random, or uncertainty-based) that determines which sample is served next, allowing model predictions to directly steer labelling effort towards the most informative samples.

		\autoref{fig:timeseries} shows the time-series labelling interface, in which multiple synchronised diagnostic signals are displayed with shared time axes and can be labelled with time-point and time-region annotations, either manually or with the assistance of the automated annotators described above. An equivalent frame-by-frame interface, supporting bounding box, polygon, and point annotations with propagation between frames, is provided for camera and video data. \autoref{fig:architecture} summarises the overall system architecture connecting the UI, API, database, and model workers.


		\autoref{fig:timeseries} shows the time-series labelling interface, in which multiple synchronised diagnostic signals are displayed with shared time axes and can be labelled with time-point and time-region annotations, either manually or with the assistance of the automated annotators described above. An equivalent frame-by-frame interface, supporting bounding box, polygon, and point annotations with propagation between frames, is provided for camera and video data. \autoref{fig:architecture} summarises the overall system architecture connecting the UI, API, database, and model workers.

		The project is tested with `pytest`, covering the core annotation and data loader logic, the REST API, and the MongoDB integration, together with end-to-end browser tests written with Playwright that exercise the UI directly. Continuous integration runs the test suite and `ruff` linting on every change.


		# Research impact statement

		`TokTagger` was initially developed by the UK Atomic Energy Authority (UKAEA) and the Science and Technology Facilities Council (STFC) to support the creation of labelled datasets from MAST, MAST-U, and JET diagnostic data for machine learning research. Since its first public release (v0.1.0), the project has grown through contributions from a team of developers across UKAEA beyond its original authors, with new data loaders, annotation types, and model-training functionality added in each subsequent release. It is used internally at UKAEA to curate labelled datasets in support of machine learning models for tasks such as automated ELM and disruption detection from MAST-U and JET diagnostics, and its direct integration with UDA, SAL, and FAIR-MAST is intended to make it straightforward for other groups working with these data systems to build similar labelled datasets for their own machine learning workflows. As a young, actively-developed project, `TokTagger`'s broader adoption and citation record are still emerging; this paper accompanies its release as reusable, general-purpose infrastructure for the wider fusion data science community.

Uh oh!

Conversation

samueljackson92 commented Aug 10, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wk9874 Aug 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

wk9874 Aug 19, 2026 •

edited

Loading