detectability

To calculate the probability that species will be recorded on a randomly selected survey day in a grid cell, given that the species is confirmed present in that cell

Detectability of species from opportunistic occurrence data

This repository contains a Python workflow to estimate recording detectability of species from opportunistic occurrence records aggregated into EEA reference grid cells. Detectability is calculated as the probability that a species is observed on a day when a grid cell is surveyed, conditional on the species being confirmed present in that cell. The workflow also produces taxonomic summaries (family/order) and exports files compatible with downstream visualisation in R / metacoder.

Overview of the method

For each species × grid cell combination, the script:

1. Defines survey days
   A survey day is any unique date in a grid cell where at least one occurrence record is present (any taxon).
   This approximates the number of independent sampling occasions in the cell.

2. Applies a “confirmed presence” filter
   To reduce spurious or transient occurrences, a species is treated as present in a grid cell only if it has:

   • ≥ 3 distinct years with records, and
   • a span of ≥ 10 years between earliest and latest record in the cell.

3. Calculates detection probability

   [ p_{s,c} = \frac{#\text{days species observed in cell}}{#\text{survey days in cell}} ]

4. Optionally applies shrinkage (stabilisation)

   A Beta prior is used to stabilise estimates in low-effort cells:

   [ p^*{s,c} = \frac{\text{detection_days}{s,c} + \alpha}{\text{survey_days}_c + \alpha + \beta} ]

   with default values: α = 1, β = 9 (prior mean ≈ 0.10).

5. Links GBIF taxonomy
   The script queries the GBIF Species API to retrieve kingdom–family classification for each speciesKey and caches results locally.

6. Summarises detectability Detectability is summarised at:

   • species level (across occupied cells)
   • family level (across species)
   • order level (across species)

Input data format

The script expects a tab-delimited file (TSV) with at minimum:

column	description
eeacellcode	EEA grid cell code (e.g. 5kmE3920N3100)
specieskey	GBIF speciesKey identifier
species	species name
yearmonthday	date as YYYY-MM-DD

Example row: