feat(llmobs): standardize agent manifest across integrations

[mz1119]

Description

Standardizes the agent_manifest emitted on agent spans across CrewAI, Google ADK, LangGraph, OpenAI Agents and Claude Agent SDK, so every integration reports the same keys (AgentManifest) in the same shapes, and instructions, tools and model are populated wherever the framework declares them.

Shared helpers in _integrations/agent_manifest.py (build_agent_manifest, instruction_fields, normalize_tool, filter_model_settings, config_value, as_str) generalize what pydantic_ai and the manual path already did: section-isolated building, an allowlist for model settings, a JSON-native result, and callables recorded by name rather than called or str()-ed. pydantic_ai moves onto them with no output change.

Per integration:

• OpenAI Agents: model_settings is allowlisted (it previously shipped extra_headers/extra_body). Hosted tools ship only declared config (no MCP headers/URL, no client objects). Callable instructions and stored prompts go to extra_instructions. Adds capabilities (MCP servers), data_contracts (output_type) and agent_settings.
• Google ADK: string models are no longer dropped. Callable instructions no longer break agentless JSON encoding. description maps to handoff_description. Adds system_prompts, model_settings (from generate_content_config), tool parameters, data_contracts, handoffs (sub_agents), guardrails (before-model/tool callbacks) and agent_settings. Removes model_configuration (it was pydantic's own config) and session_management (per-run values).
• CrewAI: instructions is goal plus backstory. Tool parameters come from args_schema. max_iter and code-execution settings move to agent_settings (fixes code_execution_permissions being overwritten). CrewAI's injected stop words are not reported.
• LangGraph: create_react_agent(model="gpt-4o") no longer raises on the provider:model split. The cached manifest is no longer mutated by a run's config. recursion_limit moves to agent_settings, dependencies (run input keys) is removed, and tools are normalized.
• Claude Agent SDK: adds instructions (system_prompt, including presets), capabilities (MCP servers, without per-run status), handoffs (agents), guardrails and agent_settings. Removes dependencies/max_iterations.

Every key web-ui reads (model_provider, handoff_description, CrewAI's {allow_delegation} handoffs) is kept.

Supersedes #19533 (callable ADK instructions now ship by name in extra_instructions; str(fn) would change on every deploy).

Testing

• Updated each integration's expected manifests; all six suites pass on the latest library version locally.
• Added tests/llmobs/test_agent_manifest_integrations.py: every integration emits only schema keys, round-trips through JSON, and is identical across rebuilds. Also covers callable instructions (never invoked), ADK string models, LangGraph model strings without a colon, cache isolation, and secret-bearing OpenAI settings and tools.

Risks

Manifest keys change shape for existing integrations (listed above). Keys web-ui reads are unchanged; removed keys are not read anywhere in web-ui. Older library versions in the matrix were not run locally.

Additional Notes

CrewAI instructions use the interpolated goal and backstory (as before), so they change when kickoff inputs change. The pre-interpolation values live on private attributes that CrewAI does not preserve reliably across crew copies.

🤖 Generated with Claude Code

[cit-pr-commenter-54b7da]

Codeowners resolved as

Resolved from the full PR diff against main using the target branch CODEOWNERS file.
CODEOWNERS team requests not listed below are not required by the current file set.

.claude/skills/llmobs-integrations/SKILL.md                             @DataDog/python-guild
ddtrace/llmobs/_integrations/agent_manifest.py                          @DataDog/ml-observability
ddtrace/llmobs/_integrations/claude_agent_sdk.py                        @DataDog/ml-observability
ddtrace/llmobs/_integrations/crewai.py                                  @DataDog/ml-observability
ddtrace/llmobs/_integrations/google_adk.py                              @DataDog/ml-observability
ddtrace/llmobs/_integrations/langgraph.py                               @DataDog/ml-observability
ddtrace/llmobs/_integrations/openai_agents.py                           @DataDog/ml-observability
ddtrace/llmobs/_integrations/pydantic_ai.py                             @DataDog/ml-observability
ddtrace/llmobs/types.py                                                 @DataDog/ml-observability
releasenotes/notes/llmobs-standardize-agent-manifest-df86615c800061b2.yaml  @DataDog/apm-python
tests/contrib/claude_agent_sdk/test_claude_agent_sdk_llmobs.py          @DataDog/ml-observability
tests/contrib/claude_agent_sdk/utils.py                                 @DataDog/ml-observability
tests/contrib/crewai/test_crewai_llmobs.py                              @DataDog/ml-observability
tests/contrib/google_adk/test_google_adk_llmobs.py                      @DataDog/ml-observability
tests/contrib/langgraph/test_langgraph_llmobs.py                        @DataDog/ml-observability
tests/contrib/openai_agents/test_openai_agents_llmobs.py                @DataDog/ml-observability
tests/llmobs/test_agent_manifest_integrations.py                        @DataDog/ml-observability

[cit-pr-commenter-54b7da]

Dependency direction analysis

📈 Existing violations got worse

3 pre-existing violation(s) increased in severity (e.g. their target became more depended-on, or got pulled into an import cycle), though the edge itself isn't new:

ddtrace.contrib.internal.openai._realtime -×-> ddtrace.llmobs.types  (contrib -> product:llmobs, score=38, +2 vs base)
ddtrace.contrib.internal.vllm.extractors -×-> ddtrace.llmobs.types  (contrib -> product:llmobs, score=38, +2 vs base)
ddtrace.contrib.internal.claude_agent_sdk._streaming -×-> ddtrace.llmobs.types  (contrib -> product:llmobs, score=38, +2 vs base)

⚠️ Existing dependency direction violations

There are 201 dependency direction violations that already exist on the base branch and have not been changed by this PR.

Show existing violations (showing 5 of 201 highest severity)

ddtrace.internal.tracemethods -×-> ddtrace.trace  (internal-core -> product:tracing, score=132)
ddtrace.internal.ci_visibility.filters -×-> ddtrace.trace  (product:ci_visibility -> product:tracing, score=130)
ddtrace.llmobs._integrations.vllm -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=130)
ddtrace.debugging._exception.replay -×-> ddtrace.trace  (product:debugging -> product:tracing, score=130)
ddtrace.llmobs._integrations.claude_agent_sdk -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=130)

To see all violations, download the layers-base.json and layers-pr.json artifacts from this CI job and run:

uv run --script scripts/import-analysis/layers.py compare layers-base.json layers-pr.json

[cit-pr-commenter-54b7da]

Circular import analysis

⚠️ Existing circular imports

There are 1 circular imports that already exist on the base branch and have not been changed by this PR.

ddtrace.errortracking._handled_exceptions.bytecode_injector -> ddtrace.errortracking._handled_exceptions.callbacks -> ddtrace.errortracking._handled_exceptions.collector -> ddtrace.errortracking._handled_exceptions.bytecode_reporting -> ddtrace.errortracking._handled_exceptions.bytecode_injector

[datadog-prod-us1-6]

 

✨ Unblock PR with BitsAI

❌ Errors

Your PR has failed checks. Please review the issues below and take necessary action before merging.

🚦 1 Pipeline job failed

DataDog/apm-reliability/dd-trace-py | prechecks — 🔧 Needs a code fix, caused by this PR

View more details · View in GitLab

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 08b38cb | Docs | View more details | Give us feedback!}