ONVIF Agentic Service

[ocampana-videotec]

Initial draft pull request to gather feedback on the AI specs being written right now.

Current status

• AI.xml is been written
• Proposal to add OAuth 2.1 in Security.xml

Tentative scheduling

• First complete draft circulated with ONVIF members and test tool vendor by the F2F meeting in Prague
• First hackathon at the plugfest in Mexico City

Topics to be included

• Transport (MCP)
• Security (OAuth 2.1)
• Minimum set of data and tools common to all the AI-native devices (potential candidate: Axis' proposal of observations)

---

Work items generated during this discussion

Work item	Assigned WG
Event Plane Delivery Model over MCP	WG Enh
Support for OAuth 2.1 in the Security Service #824	WG Enh
Observation and Assertion information model	WG AI
Space, Topology, and Coordinate Systems information model	WG AI
InformationItem and InformationElement information model	WG AI
SOSA/SSN interoperability and information model	WG AI

---

I expect this PR to be used as a shared notepad about the specs begin developed. Once ready, this will be closed a new a formal PR will be created.

[sujithhanwha]

Image is missing.

[willysagefalk]

PR specifies a persistent HTTP GET /mcp carrying newline-delimited notifications. MCP 2026-07-28 explicitly removed the GET stream. Requests use POST; long-lived notifications use the SSE response to subscriptions/listen. Independent notifications/message events cannot simply be placed on a permanent GET stream. Official MCP Streamable HTTP specification.

[willysagefalk]

server/discover is incorrectly mandatory as the first call.

A server must support it, but a modern client may call any operation directly and handle UnsupportedProtocolVersionError. The PR currently says the client shall call it first.

[ocampana-videotec]

server/discover is incorrectly mandatory as the first call.

A server must support it, but a modern client may call any operation directly and handle UnsupportedProtocolVersionError. The PR currently says the client shall call it first.

@willysagefalk Do you have a reference for that?

[willysagefalk]

@ocampana-videotec Yes. The normative reference is the MCP 2026-07-28 specification, Discovery — When to Call:

https://modelcontextprotocol.io/specification/2026-07-28/server/discover#when-to-call

It states that calling server/discover is optional for clients: a client may invoke any RPC directly and handle UnsupportedProtocolVersionError if the requested version is unsupported. The requirement is that servers MUST implement server/discover.

The same section only recommends calling it first for a dual-era client over stdio, as a backward-compatibility probe. Since the ONVIF proposal uses HTTP POST, that stdio-specific recommendation does not apply.

I therefore suggest replacing:

The ONVIF Client shall call server/discover as the first request...

with something along these lines:

The ONVIF Device shall support server/discover. An ONVIF Client may call server/discover before other requests to retrieve the supported protocol versions and capabilities.

[ocampana-videotec]

@ocampana-videotec Yes. The normative reference is the MCP 2026-07-28 specification, Discovery — When to Call:

Corrected.

[ocampana-videotec]

@willysagefalk , following our conversation, I just added chapter 9 about metadata.

As you can see it defines abstract concepts and for Observations and Provenance it proposes two specialization. SOSA/SSN was included to accommodate @DammnAI 's feedback in https://github.com/onvif/wg_AI/pull/9 , while the JLDS-based provenance is my proposal based on an already existing specification.

If any member wants to model an Observation, they can simply propose a similar derivation.

[willysagefalk]

Thanks for adding the metadata chapter. I suggest that we now use this PR as the working integration point and complete the model here. I reviewed head a7d7d490.

The following changes are needed before the draft can become an implementable specification.

Blocking implementation issues

1. Replace the persistent GET /mcp data channel

   The draft still describes notifications/message over a permanent GET /mcp connection. In MCP 2026-07-28, each Streamable HTTP message is sent by POST and a long-lived notification stream is opened by a subscriptions/listen request; that operation replaced the former HTTP GET stream.

   Please define the metadata delivery flow using subscriptions/listen, including the requested/acknowledged notification filter, subscription identifier, cancellation/reconnection behavior, and the exact notification type. If ONVIF needs notifications that carry complete observation payloads rather than standard resource-change notifications, this must be defined as an explicit, negotiated ONVIF extension with its own schema.

   References:

   • https://modelcontextprotocol.io/specification/2026-07-28/basic/transports
   • https://modelcontextprotocol.io/specification/2026-07-28/basic/patterns/subscriptions

2. Separate JSON-LD context from validation-schema identification

   The text currently reuses the LinkML schema id as the JSON-LD @context. These artifacts have different functions: @context maps JSON terms to IRIs, while a schema identifier selects the LinkML/JSON Schema used for structural validation.

   Please define separate, versioned identifiers, for example @context, schema, and, if needed, semanticContract. Retrieval should be through authenticated MCP resources or a trusted ONVIF registry, with immutable versions and integrity binding rather than fetching an arbitrary URI supplied by a payload.

   Reference: https://www.w3.org/TR/json-ld11/

3. Use a named Data Integrity cryptosuite exactly as specified

   The current proofValue description defines H(canonical(unsecuredDocument)) || H(canonical(proofConfig)). For ecdsa-rdfc-2019, W3C defines the reverse order: proofConfigHash || transformedDocumentHash.

   Please select a concrete cryptosuite and reference its complete transformation, hashing, serialization, verification-method, and proof-value rules rather than constructing a generic algorithm from the Security Baseline. Add at least one signing and verification test vector.

   Reference: https://www.w3.org/TR/vc-di-ecdsa/#hashing-ecdsa-rdfc-2019

4. Repair yaml/ai.yaml

   The file currently identifies itself as the Recording model:

   id: https://w3id.org/onvif/schema/recording
   name: onvif_recording_model

   It also uses tai:Capabilities without declaring tai. Please give the AI model its own namespace/name, declare every prefix, and validate imports and references. This should be covered by schema-generation/validation CI.

Information-model changes requiring WG agreement

5. Do not use Observation as both the evidence concept and the top-level payload container

   The draft defines Observation as “the top-level abstract container for every piece of information.” That makes an observation simultaneously an acquisition of evidence, a transport envelope, and—when AI inference is involved—a claim about the world. These roles have different identity, provenance, confidence, and lifecycle.

   I suggest defining the core boundary explicitly:

   • InformationEnvelope: serialization container for one or more information entities.
   • Observation: acquisition of evidence, with source/procedure, phenomenon time, result time, and observation location.
   • Statement: a typed proposition about a subject.
   • Assertion: an agent's attributed claim that a Statement is valid, with assertion time, confidence, provenance/authenticity, and supporting observations.
   • PhysicalObject: stable reference to the real-world subject.

   This matters for RE-ID: detections/tracks are observations; “Track A has the same identity as Track B” is a statement; the RE-ID service's confidence-bearing output is an assertion supported by those observations.

   SOSA can remain a concrete observation specialization or mapping, while the ONVIF model also supports inference, fusion, verification, and human observations without forcing all of them into a sensor-only pattern.

6. Replace SOSAObservation.hasResult: Any with testable result types

   Any prevents generated clients, schema validation, and the ONVIF test tool from determining what is conformant. Define a typed Result hierarchy/union. A direct measurement may use a typed value result; a derived conclusion should reference a typed Assertion/Statement structure.

7. Separate observation location from source coverage

   Where an observation applies is not the same as the area a source can observe. Please model these separately and use structured references for coordinate frame, geometry/spatial region, units, transforms, and topology. Avoid unconstrained zone or coordinate strings.

8. Add end-to-end normative examples and conformance points

   Please include at least:

   • one direct sensor observation;
   • one derived AI assertion;
   • one RE-ID assertion supported by two observations;
   • one signed JSON-LD example;
   • one subscriptions/listen delivery sequence.

   For each SHALL/MUST, identify the responsible role, observable pass/fail behavior, capability discovery, unsupported behavior/fault, and expected test-tool coverage.

One editorial cleanup: the PR description should describe the minimum information model in vendor-neutral ONVIF terms.

The abstract-class and specialization direction is useful. The main work now is to make the boundaries explicit and ensure the transport, JSON-LD, schema, and signature mechanisms can be implemented and tested consistently.

[ocampana-videotec]

Sorry @willysagefalk but we need to find a more efficient way to tackle the document, we cannot just copy&paste tons of lines from an LLM. For the future, we need IMHO to be shorter a more focused.

Point 1: I will study it, it looks correct.

Point 2: I have no idea right now. It's generated by the linkml tools...

Point 3: I intentionally got rid of ecdsa-rdfc-2019 in order to point our specs to the ONVIF Security Baseline.

Point 4: The yaml file is broken, I know it very weak, there is even a note in the AI.xml that that part will have to be removed and be put in the ai.yaml. We are simply too far now to allocate time to it.

Point 5: what is the problem here? beyond a style issue, I really see no problem for the observation to be to top level container.

Point 6: how is it defined in SOSA? We do not want to diverge. If this is not good enough, then we have a reason to ditch SOSA

Point 7: this looks like point 5. What is the use case we are trying to solve?

Point 8: I agree and in fact we are starting to work on Annex A. The problem is that before creating an example we must clarify how it will work.

Therefore, the immediately actionable items are: 1, 3 and 6.

[willysagefalk]

I suggest extending the model with a neutral Assertion type alongside Observation, together with an ObjectObservation specialization:

InformationEntity
├── Observation
│ ├── ObjectObservation
│ └── SOSAObservation
└── Assertion
├── Statement
└── Claim

ObjectObservation represents an observation concerning one persistent PhysicalObject. It groups the Claims produced within that observation context and may include observed positions or other object-related information.

Assertion is not an Observation. It expresses a proposition, relationship or conclusion and can reference Observations as supporting evidence. Statement and Claim are concrete specializations of Assertion.

For example, in RE-ID:

ObjectObservation A
└── contains Claims about Object/Track A

ObjectObservation B
└── contains Claims about Object/Track B

Re-identification Claim
├── proposition: A and B represent the same PhysicalObject
├── supportedBy: ObjectObservation A, ObjectObservation B
├── producedBy: RE-ID service
└── confidence: 0.82

SOSAObservation remains available for SOSA interoperability, but it does not need to be the foundation of the ONVIF information model. ObjectObservation and SOSAObservation describe different aspects, so a sensor-based ObjectObservation may also be mapped to SOSA.

[ocampana-videotec]

@willysagefalk let's start clarifying the definition of the abstract classes involved in metadata streaming. Once we clarify them, we can specialize stuff like ObjectObservation, PhysicaObject and so on.

My definitions:

• Observation: is the top-level abstract container for every piece of information emitted by an ONVIF AI Device
• Assertion: I do not have a definition yet
• Claim: I do not have a definition yet
• Statement: I do not have a definition yet

I do have a few ideas I am working for for statement, but I do not have a clear understanding of the difference between Claim and Assertion. My idea is that, if we are talking about abstract classes, assertions, claims and statements are the same thing.

If that's the case we could have

Observation
+- timing
+- location
+- provenance
+- assertion (or how we are going to call it)

To my eyes, InformationEntity is an empy container, it's just a waste of bandwidth. We do not need to wrap a rich container (Observation) with an empty conatiner (InformationEntitiy)

From assertion we could derive a statement, and for the statement use the idea of the predicate we were discussing at the Budapest meeting.

[willysagefalk]

InformationEntity is not intended as a serialized wrapper around an Observation, so it adds no bandwidth. It is only the neutral abstract superclass that allows both Observations and Assertions to be top-level information types.

An Observation represents an observation/perception context and may contain Assertions produced during that observation. An Assertion represents a proposition or conclusion and may reference one or more Observations as evidence. For example, a RE-ID Assertion that two observations concern the same physical object cannot naturally belong to only one of those observations.

I agree that we should first settle these two abstract definitions. We can then clarify the precise distinction between Statement and Claim and add specializations such as ObjectObservation.

[ocampana-videotec]

I am still having difficulties to understand clearly the difference between an Assertion and an Observation

I am good with "An Observation represents an observation/perception context and may contain Assertions produced during that observation". But then I see a logical problem in "An Assertion represents a proposition or conclusion and $\color{red}{\text{may}}$ reference one or more Observations as evidence".

Why may? Observations may exist without Assertions, but in my understanding Assertions cannot exist outside of an Observation. It is a must, not a may in my opinion.

Assertions without observation are, de facto, Observations.

Do you agree on this?

[willysagefalk]

The foundation should define only AuthenticityProof. The WG shall evaluate at least an embedded W3C Data Integrity proof and a manifest-based proof. Data Integrity is suitable for independently protected JSON-LD items; a manifest-based profile is likely better for batched envelopes, extraction, provenance chains and multiple producers.
My architectural preference for ONVIF would be:
DataIntegrityProof available for individual items;
ManifestProof for envelopes and streaming batches;

[ocampana-videotec]

My architectural preference for ONVIF would be: DataIntegrityProof available for individual items; ManifestProof for envelopes and streaming batches;

I disagree on this, I see no value in having two mechanisms of the same thing,

Verifiable Credential Data Integrity already exists and does not need to be invented from scratch. The manifest approach is chatter, not exsisting and most important, it is not clear what problem solves that Verifiable Credential Data Integrity does not.

Anyway, as discussed in today's telco, I will open a work item in video enhancement. WG AI does not have the compenecy in cryptography that we need to discuss this

[willysagefalk]

Proposed AI information-model work items

Based on the discussion, I propose dividing the information-model work into the following four work items. The common time concepts are included in the same work item as objects, observations, and assertions.

1. Objects, Observations, Assertions, and Time

Define the core model for representing perceived objects, observations, and conclusions derived from observations.

Scope

• Define the relationships between Object, Observation, and Assertion.
• Define observation specializations such as ObjectObservation, while allowing other specializations such as SpaceObservation.
• Define assertions as standalone propositions, including statements and claims.
• Define how assertions reference supporting observations without creating circular graphs.
• Define results, confidence, and uncertainty.
• Define the common time model, including:
  • observation or phenomenon time;
  • result time;
  • information production time;
  • validity intervals;
  • temporal uncertainty;
  • temporal ordering of observations and derived assertions.

Expected output

• LinkML definitions and JSON-LD representations.
• Definitions of the temporal properties and their semantics.
• Examples covering object detection, dwell, and re-identification.
• Validation rules preventing circular dependency graphs.

This work item does not define spatial topology or authenticity mechanisms.

---

2. Space, Topology, and Coordinate Systems

Define how physical and logical spaces are represented and how objects, observations, and assertions refer to them.

Scope

• Define ObservedSpace, spatial regions, sites, buildings, floors, rooms, zones, and other relevant spatial entities.
• Define containment, adjacency, connectivity, and other topology relationships.
• Define coordinate systems and coordinate frames, including:
  • units;
  • origin;
  • axis conventions;
  • parent frames;
  • transformations between frames.
• Define spatial geometry and spatial uncertainty.
• Define how objects, observations, and assertions reference spaces and coordinate frames.
• Align the model with existing ONVIF topology, geometry, and spatial concepts.

Expected output

• LinkML definitions for spaces, topology, geometry, and coordinate frames.
• JSON-LD examples using local device coordinates and site-level coordinates.
• Mapping guidance for relevant existing ONVIF structures.

Temporal semantics are defined by Work Item 1 and should not be independently redefined here.

---

3. Information Envelopes, Provenance, and Authenticity

Define how information items are packaged, attributed, transported, and protected.

Scope

• Define InformationItem and InformationEnvelope.
• Support both individual information items and envelopes containing multiple items.
• Define identifiers, producer information, creation metadata, and provenance.
• Define how provenance applies to observations and assertions.
• Define references to procedures, analytics, rules, models, versions, inputs, and parameters.
• Define an extensible authenticity model capable of supporting signature-based or manifest-based approaches.
• Define what an authenticity declaration protects and how verification results are represented.
• Align the security aspects with the ONVIF Security Baseline.
• Preserve a possible future path to C2PA interoperability.

Expected output

• LinkML definitions for information items, envelopes, provenance, and authenticity metadata.
• Examples of individually protected items and protected batches.
• Clearly defined producer, consumer, recorder, and verifier roles.
• A technology-neutral model that does not prematurely mandate a particular signing format.

This work item should distinguish provenance—where information came from—from authenticity—how its integrity and origin can be verified.

---

4. SOSA/SSN and External Semantic Interoperability

Define how the ONVIF information model interoperates with SOSA/SSN and other relevant semantic models without making those models mandatory for basic ONVIF implementations.

Scope

• Map ONVIF observations, procedures, results, features of interest, and time concepts to SOSA/SSN.
• Identify concepts that map directly, concepts requiring specialization, and concepts without suitable SOSA equivalents.
• Define the JSON-LD contexts or mapping rules required for semantic interoperability.
• Demonstrate how ONVIF-native information can be consumed as SOSA-compatible linked data.
• Evaluate alignment with SAREF, BRICK, and other models where justified by concrete use cases.
• Document semantic loss or ambiguity in conversions between representations.

Expected output

• A normative or informative SOSA interoperability mapping, as decided by the Working Group.
• JSON-LD context definitions.
• Examples showing ONVIF-native and SOSA-aligned representations of the same information.
• Validation of the mappings using representative observation and assertion use cases.

SOSA alignment should remain an interoperability layer. It should not prevent ONVIF from defining the concepts required by physical-security systems.

[ocampana-videotec]

Following the call of WG AI on Aug 12th, this draft PR was broken down into several parts, as listed in the top comment.

This PR is still open, to accommodate @bsriramprasad 's feedback in #818 and to revert the changes on the Security service. Once these items are done, I think that we can let circulate the first draft at the F2F meeting in Prague,

[dstafx]

To make the information item more useful we should instead link to a process. Every information item is created by some process. That process have have different types of provenance information such as things like version, dependencies, inputs etc.

Separately from this lineage information can be added to link this information item to other information items that it was derived_from.

[ocampana-videotec]

In my proposal, Provenance is abstract to allow modelling different "authors" of statements: for example it can be a process, a person or a device. I am open to deriving a ProcessProvenance

[dstafx]

Can't we extend by subclassing or sending separate information items? Generic extension fields give no clear migration path to the standard.

[ocampana-videotec]

The extensible attribute for proprietary extensions is here just to automatically inherit it in real classes. Just to avoid having to repeat it in every class (and have the risk of forgetting it)

[HansBusch]

IQSIGHT high level comments

We suggest to first align on the overall structure before looking at details.

Scope

This section defines some acronyms. It should contain what the spec covers.
Not sure whether this is complete:

• Architectural overview
• Definition of MCP device interface
• Security considerations
• Governance of interface development

Acronyms

• ONVIF - As this spec is also meant to be ONVIF: do you mean classical ONVIF services?
• ONVIF Device - Sounds more like classical ONVIF.
• ONVIF Client - Do we need to coin this. Sounds like the ONVIF 1.0 NVC mistake.
• MCP Metadata Streaming - MCP uses term metadata differently. Propose renaming to Event Streaming.
• WSDL-MCP Bridge - Defined but never used.

Overview and Architecture

Suggest to have one or two informative sections describing the overall concept

Requirements

As other ONVIF specs the specification should focus on device requirements.
Additionally the governance section may contain organizational requirements.

Section 8 Service Discovery and Data Exchange

Avoid requiring devices to support two schema representations

Looks like section 8.3.2 requires schema in LinkML and 8.3.4 in JSON schema.

Prefer using only JSON on device level for maximum interoperability.
Using linkml for specification and governance is fine.

Section 8.4.2 and 8.4.3

We really like the analytics functionality. Making these functions mandatory for every
device severely restricts applicability of this specification.

Additionally the parameters of these functions should have some attention.
Suggest to move these functions to a normative annex and align parameters with existing analytics specification

Classical ONVIF APIs

Shouldn't we include here the proposal for mapping ONVIF functions to MCP?

Metadata Event Delivery

This section should be named Event or Message Delivery.

9 Authentication and Authorization

Distinguish between MCP request authorization and message signing.
To our understanding requests are authorized with JWT tokens.

10 Metadata

This section defines events not metadata.

10.5.1.2 Proof fields

Hard to imagine how to maintain access to public keys of large distributed systems.
Shouldn't this use some hierarchical CA infrastructure or JWT tokens from an authorization server?

[willysagefalk]

I agree that we should align the scope, architecture and terminology before continuing with the detailed schemas and message flows.

A few proposed conclusions:

• Scope and terminology:

The specification should clearly state that the Agentic Service complements existing ONVIF services. We should not redefine ONVIF Device or ONVIF Client as MCP-specific concepts. In the MCP sections, we can instead use the established terms MCP Server, MCP Client and MCP Host. “Metadata Event Delivery” should be renamed to “Event Delivery” or “Message Delivery”. The unused WSDL-MCP Bridge term should either be given a clear architectural role or removed.

• Overview and requirements:

The current background and architecture sections can be simplified into one or two informative sections showing how the Agentic Service relates to existing ONVIF services. Normative requirements should primarily apply to the service implementation. Client requirements should only be included where they are necessary for interoperability.

• Schema representations:

I agree that a device should not be required to expose both LinkML and JSON Schema. LinkML can remain the normative source model used by ONVIF, while JSON Schema, MCP inputSchema and the JSON-LD context are generated runtime artifacts exposed by the service. The schema identifier and JSON-LD context should remain separate.

• Analytics functions:

I agree that the current three mandatory Skills are too restrictive. Generic MCP discovery should form the baseline. Analytics-specific tools should be conditional capabilities mandatory only when advertised and should align with the existing ONVIF analytics specifications.

• Existing ONVIF APIs:

This is an important part of the architecture. We should show two separate implementation paths: devices that expose the Agentic Service natively, and an ONVIF-to-MCP gateway that communicates with existing products through their current ONVIF APIs while exposing their capabilities as MCP tools and resources. This allows existing ONVIF products to participate without firmware changes.
The specification should define the common mapping principles—including capability discovery, operation and parameter mapping, errors and authorization—while preserving the semantics of the existing ONVIF services. This also gives the WSDL-MCP Bridge a clear role in the architecture rather than leaving it as an unused definition.

• Authorization and signing:

These are separate concerns. JWT bearer tokens authorize individual MCP requests. InformationItem or InformationEnvelope signatures provide durable integrity and producer authenticity after information has been forwarded or stored. An access JWT cannot replace such a signature.

• Chapter 10:

I suggest renaming this chapter to Information Model. Observation, Assertion, InformationItem, provenance and envelopes are information objects. They may be delivered as events, but they may also be retrieved, stored or referenced by other information objects.

[ocampana-videotec]

@HansBusch and @willysagefalk we have a problem here. In the Model Context Protocol (MCP), the inputSchema field in the response to the tools/list call must strictly be a JSON Schema. It is not possible to send LinkML definitions directly within the protocol. The MCP specification mandates JSON Schema because it is the standard natively understood by almost all Large Language Models (LLMs) for function calling, guaranteeing universal interoperability between any MCP server and client.

Therefore either we accept both LinkML and JSON Schema, or we must check whether we can replace LinkML with JSON Schema.

[willysagefalk]

@ocampana-videotec Yes, that’s what I meant: use LinkML to maintain the ONVIF model and generate JSON Schema for MCP. Devices would only need to expose JSON Schema, so they wouldn’t have to support both.

[ocampana-videotec]

@ocampana-videotec Yes, that’s what I meant: use LinkML to maintain the ONVIF model and generate JSON Schema for MCP. Devices would only need to expose JSON Schema, so they wouldn’t have to support both.

let's double check we are understanding the same thing: you want to use LinkML just to generate JSON Schema and then all the description of services and data exchanged between clients and devices will be with JSON Schema.

Is this understanding correct?

[willysagefalk]

Yes, that’s the idea. We maintain the ONVIF model in LinkML and generate the JSON Schema used by devices and clients. They exchange JSON data, with JSON-LD context where needed, without having to handle LinkML.

[ocampana-videotec]

Closing as agreed during the Prague F2F meeting. The development will continue in the AI/spec branch until a new PR will be created.