Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -303,6 +303,18 @@ jobs:
- run: npm test
- name: npm audit (Runtime OpenClaw dependencies)
run: node ../../ci/npm-audit-bulk.mjs package-lock.json
- name: Install LangGraph TypeScript locked dependencies
working-directory: runtimes/langgraph-ts
run: npm ci
- name: Typecheck LangGraph TypeScript runtime
working-directory: runtimes/langgraph-ts
run: npm run typecheck
- name: Build LangGraph TypeScript runtime
working-directory: runtimes/langgraph-ts
run: npm run build
- name: Test LangGraph TypeScript telemetry exports
working-directory: runtimes/langgraph-ts
run: npm test

mesh-plugin-build:
name: Mesh Plugin Build & Test
Expand Down
61 changes: 61 additions & 0 deletions docs/security-audits/2026-09-15-langgraph-telemetry-sdk.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
<!-- Copyright (c) Microsoft Corporation.
Licensed under the MIT License. -->

# LangGraph TypeScript telemetry - bounded delegated source review

Source: `8c659f891e5bbde756fb9c7fd355e5699b9be9eb`.
Base: `4ec8c5a87d6efdb2262ad48b6cdfedcbea892988`.

Status: **Scoped source-approved; actual hosted execution remains required.**
This record does not approve deployment or waive any current-head check.

## Scope and evidence

The actual candidate image build, ACR run `chh0`, failed TypeScript compilation:
the source imported the removed `Resource` constructor and called the removed
`NodeTracerProvider.addSpanProcessor` method. The committed dependency manifest
and lockfile already select OpenTelemetry SDK 2.8.0.

The repair uses `resourceFromAttributes` and the tracer provider's constructor
`spanProcessors` option. Resource attributes, service-version fallback, endpoint
precedence, registration, metrics readers, best-effort failure handling and
already-initialized behavior remain unchanged. No dependency version, identity
scope, egress permission or telemetry destination is changed.

The Docker builder and production dependency stages now use the existing
committed lockfile through `npm ci`, rather than resolving floating dependencies.
The removed Vitest `basic` reporter was separately reproduced using the verified
4.1.8 runner; the normal default reporter replaces it.

Four regressions use actual SDK providers and a loopback HTTP collector. They
require real trace and metric exports, resource attributes, endpoint precedence
and preservation of the first initialization. A caught initialization error or
missing provider is a test failure, not a successful optional-telemetry result.
Both provider shutdown and collector cleanup are exercised.

The existing required `Runtime OpenClaw Build & Test` status retains its name
and existing steps, with explicit locked LangGraph install, typecheck, build and
test steps added. This closes the missing runtime-build coverage; it does not
replace or skip another required check.

## Review and qualification limits

Implementation was performed in the separate `h100-beta-image-build` AI context;
the parent integration context reviewed the complete six-file change and the
unchanged lockfile. This is bounded source review, not exhaustive assurance of
the entire telemetry SDK or a second human review.

The maintainer's publication-review delegation is recorded in
[comment 5615522306](https://github.com/Azure/kars/pull/551#issuecomment-5615522306).
The author attestation is exercised under that delegation, not a claim of
personal code review by the maintainer.

Local syntax, workflow/lockfile-preservation and whitespace checks passed.
The locked SDK 2.8.0 dependency graph is not available locally; cached 1.30.1 was
not substituted. Local build/typecheck and real exporter execution are therefore
**not claimed**. All 31 current branch requirements and actual hosted runtime/
container qualification remain mandatory before merge or H100 installation.
No old image may be relabeled as a build of the repaired source.

Signed-off-by: pallakatos (author source attestation through explicit maintainer-delegated AI review, not a claim of personal code review) <191481949+pallakatos@users.noreply.github.com>
Signed-off-by: GitHub Copilot (independent-context delegated AI source review, not a second human) <223556219+Copilot@users.noreply.github.com>
11 changes: 10 additions & 1 deletion runtimes/langgraph-ts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,17 @@ await bootstrap();

```bash
npm ci
npm run typecheck
npm run build
npm test
```

The sandbox image (`sandbox-images/langgraph-ts/`) installs the
already-built `dist/` output.
already-built `dist/` output. Its builder and production dependency stages both
use the committed lockfile with `npm ci`.

Telemetry initialization targets the locked OpenTelemetry 2.x SDK APIs. The
regressions use real SDK providers and a local HTTP collector to verify trace
and metric exports, resource attributes, endpoint precedence and idempotent
initialization. CI runs these checks under the existing required runtime build
job; a best-effort initialization warning is not accepted as working telemetry.
2 changes: 1 addition & 1 deletion runtimes/langgraph-ts/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
"scripts": {
"build": "tsc -p .",
"typecheck": "tsc -p . --noEmit",
"test": "vitest run --reporter=basic",
"test": "vitest run",
"lint": "tsc -p . --noEmit"
},
"engines": {
Expand Down
14 changes: 8 additions & 6 deletions runtimes/langgraph-ts/src/otel.ts
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ export async function initTelemetry(
| undefined;
try {
api = await import('@opentelemetry/api');
const { Resource } = await import('@opentelemetry/resources');
const { resourceFromAttributes } = await import('@opentelemetry/resources');
const {
ATTR_SERVICE_NAME,
ATTR_SERVICE_VERSION,
Expand Down Expand Up @@ -83,18 +83,20 @@ export async function initTelemetry(
DEFAULT_OTLP_METRICS_ENDPOINT,
);

const resource = new Resource({
const resource = resourceFromAttributes({
[ATTR_SERVICE_NAME]: opts.serviceName,
[ATTR_SERVICE_VERSION]: opts.serviceVersion ?? '0.1.0',
'service.namespace': 'kars',
'kars.runtime.kind': 'LangGraph',
'kars.runtime.language': 'typescript',
});

const tracerProvider = new NodeTracerProvider({ resource });
tracerProvider.addSpanProcessor(
new BatchSpanProcessor(new OTLPTraceExporter({ url: tracesUrl })),
);
const tracerProvider = new NodeTracerProvider({
resource,
spanProcessors: [
new BatchSpanProcessor(new OTLPTraceExporter({ url: tracesUrl })),
],
});
tracerProvider.register();

const meterProvider = new MeterProvider({
Expand Down
236 changes: 236 additions & 0 deletions runtimes/langgraph-ts/tests/otel.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,236 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.

import { createServer, type Server } from 'node:http';
import {
context,
metrics,
propagation,
ProxyTracerProvider,
trace,
} from '@opentelemetry/api';
import { MeterProvider } from '@opentelemetry/sdk-metrics';
import { NodeTracerProvider } from '@opentelemetry/sdk-trace-node';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { _resetForTests, initTelemetry } from '../src/otel';

interface ExportRequest {
path: string | undefined;
body: unknown;
}

let server: Server | undefined;
let tracerProvider: NodeTracerProvider | undefined;
let meterProvider: MeterProvider | undefined;
const requests: ExportRequest[] = [];
const collectorErrors: unknown[] = [];

async function startCollector(port: number): Promise<string> {
server = createServer((request, response) => {
const chunks: Buffer[] = [];
request.on('data', (chunk: Buffer) => chunks.push(chunk));
request.on('error', (error) => collectorErrors.push(error));
request.on('end', () => {
try {
const body: unknown = JSON.parse(Buffer.concat(chunks).toString('utf8'));
requests.push({ path: request.url, body });
response.writeHead(200, { 'content-type': 'application/json' });
response.end('{}');
} catch (error) {
collectorErrors.push(error);
response.writeHead(400).end();
}
});
});
const collector = server;
await new Promise<void>((resolve, reject) => {
collector.once('error', reject);
collector.listen(port, '127.0.0.1', resolve);
});
const address = collector.address();
if (!address || typeof address === 'string') {
throw new Error('OTLP test collector has no TCP address');
}
return `http://127.0.0.1:${address.port}`;
}

beforeEach(() => {
requests.length = 0;
collectorErrors.length = 0;
for (const key of [
'OTEL_EXPORTER_OTLP_ENDPOINT',
'OTEL_EXPORTER_OTLP_TRACES_ENDPOINT',
'OTEL_EXPORTER_OTLP_METRICS_ENDPOINT',
'OTEL_EXPORTER_OTLP_HEADERS',
'OTEL_EXPORTER_OTLP_TRACES_HEADERS',
'OTEL_EXPORTER_OTLP_METRICS_HEADERS',
]) {
vi.stubEnv(key, '');
}
vi.stubEnv('OTEL_TRACES_SAMPLER', 'always_on');
vi.stubEnv('OTEL_METRIC_EXPORT_INTERVAL', '60000');
vi.spyOn(console, 'warn').mockImplementation(() => {});
vi.spyOn(console, 'error').mockImplementation(() => {});
});

afterEach(async () => {
try {
await Promise.all([
tracerProvider?.shutdown(),
meterProvider?.shutdown(),
]);
} finally {
tracerProvider = undefined;
meterProvider = undefined;
trace.disable();
metrics.disable();
context.disable();
propagation.disable();
_resetForTests();
vi.unstubAllEnvs();
vi.restoreAllMocks();
if (server?.listening) {
const collector = server;
collector.closeAllConnections();
await new Promise<void>((resolve, reject) => {
collector.close((error) => error ? reject(error) : resolve());
});
}
server = undefined;
}
});

describe('OpenTelemetry 2.x initialization and OTLP export', () => {
it.each([
{
name: 'explicit endpoints override signal-specific and general environment',
port: 0,
explicit: true,
signalEnv: true,
generalEnv: true,
tracesPath: '/explicit-traces',
metricsPath: '/explicit-metrics',
serviceVersion: '2.3.4',
},
{
name: 'signal-specific environment overrides the general endpoint',
port: 0,
explicit: false,
signalEnv: true,
generalEnv: true,
tracesPath: '/signal-traces',
metricsPath: '/signal-metrics',
serviceVersion: '2.3.4',
},
{
name: 'general endpoint remains verbatim for traces; metrics use their default',
port: 8443,
explicit: false,
signalEnv: false,
generalEnv: true,
tracesPath: '/general',
metricsPath: '/v1/metrics',
serviceVersion: undefined,
},
{
name: 'empty environment falls back to the router loopback endpoints',
port: 8443,
explicit: false,
signalEnv: false,
generalEnv: false,
tracesPath: '/v1/traces',
metricsPath: '/v1/metrics',
serviceVersion: undefined,
},
])('$name', async (testCase) => {
const endpoint = await startCollector(testCase.port);
if (testCase.generalEnv) {
vi.stubEnv('OTEL_EXPORTER_OTLP_ENDPOINT', `${endpoint}/general`);
}
if (testCase.signalEnv) {
vi.stubEnv('OTEL_EXPORTER_OTLP_TRACES_ENDPOINT', `${endpoint}/signal-traces`);
vi.stubEnv('OTEL_EXPORTER_OTLP_METRICS_ENDPOINT', `${endpoint}/signal-metrics`);
}
await initTelemetry({
serviceName: 'langgraph-export-regression',
serviceVersion: testCase.serviceVersion,
tracesEndpoint: testCase.explicit ? `${endpoint}/explicit-traces` : undefined,
metricsEndpoint: testCase.explicit ? `${endpoint}/explicit-metrics` : undefined,
});

const globalTracerProvider = trace.getTracerProvider();
expect(globalTracerProvider).toBeInstanceOf(ProxyTracerProvider);
if (!(globalTracerProvider instanceof ProxyTracerProvider)) {
throw new Error('initTelemetry did not register a tracer provider');
}
const delegate = globalTracerProvider.getDelegate();
if (!(delegate instanceof NodeTracerProvider)) {
throw new Error('initTelemetry did not initialize the real NodeTracerProvider');
}
tracerProvider = delegate;
const globalMeterProvider = metrics.getMeterProvider();
if (!(globalMeterProvider instanceof MeterProvider)) {
throw new Error('initTelemetry did not initialize the real MeterProvider');
}
meterProvider = globalMeterProvider;
expect(console.warn).not.toHaveBeenCalled();

await initTelemetry({
serviceName: 'must-not-replace-first-initialization',
serviceVersion: '9.9.9',
tracesEndpoint: `${endpoint}/must-not-export-traces`,
metricsEndpoint: `${endpoint}/must-not-export-metrics`,
});
expect(trace.getTracerProvider()).toBe(globalTracerProvider);
expect(globalTracerProvider.getDelegate()).toBe(tracerProvider);
expect(metrics.getMeterProvider()).toBe(meterProvider);
expect(console.error).toHaveBeenCalledTimes(1);

trace.getTracer('langgraph-regression').startSpan('real-sdk-span').end();
metrics.getMeter('langgraph-regression').createCounter('real_sdk_counter').add(3);
await tracerProvider.forceFlush();
await meterProvider.forceFlush();

expect(collectorErrors).toEqual([]);
expect(requests.map((request) => request.path).sort()).toEqual(
[testCase.tracesPath, testCase.metricsPath].sort(),
);
const attributes = expect.arrayContaining([
{ key: 'service.name', value: { stringValue: 'langgraph-export-regression' } },
{ key: 'service.version', value: { stringValue: testCase.serviceVersion ?? '0.1.0' } },
{ key: 'service.namespace', value: { stringValue: 'kars' } },
{ key: 'kars.runtime.kind', value: { stringValue: 'LangGraph' } },
{ key: 'kars.runtime.language', value: { stringValue: 'typescript' } },
]);
expect(requests.find((request) => request.path === testCase.tracesPath)?.body)
.toMatchObject({
resourceSpans: [{
resource: { attributes },
scopeSpans: expect.arrayContaining([expect.objectContaining({
scope: expect.objectContaining({ name: 'langgraph-regression' }),
spans: expect.arrayContaining([
expect.objectContaining({ name: 'real-sdk-span' }),
]),
})]),
}],
});
expect(requests.find((request) => request.path === testCase.metricsPath)?.body)
.toMatchObject({
resourceMetrics: [{
resource: { attributes },
scopeMetrics: expect.arrayContaining([expect.objectContaining({
scope: expect.objectContaining({ name: 'langgraph-regression' }),
metrics: expect.arrayContaining([expect.objectContaining({
name: 'real_sdk_counter',
sum: expect.objectContaining({
dataPoints: expect.arrayContaining([
expect.objectContaining({ asDouble: 3 }),
]),
}),
})]),
})]),
}],
});
expect(console.warn).not.toHaveBeenCalled();
});
});
Loading
Loading