Skip to content

cws: add DNS response IPs and CNAMEs to activity dumps - #515

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 3 commits into
DataDog:masterfrom
homoeconomics:daniel.zhou/CWS-6595-dns-response-ips-cnames
Sep 3, 2026
Merged

cws: add DNS response IPs and CNAMEs to activity dumps#515
gh-worker-dd-mergequeue-cf854d[bot] merged 3 commits into
DataDog:masterfrom
homoeconomics:daniel.zhou/CWS-6595-dns-response-ips-cnames

Conversation

@homoeconomics

@homoeconomics homoeconomics commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Adds a DNSResponseInfo message to the CWS activity-dump v1 schema, carrying the DNS response code, the resolved IP addresses, and the CNAME targets. It is nested on DNSInfo as field 6, so the answers stay attached to the question that produced them.

message DNSInfo {
    string name = 1;
    uint32 type = 2;
    uint32 class = 3;
    uint32 size = 4;
    uint32 count = 5;

    DNSResponseInfo response = 6;   // new
}

message DNSResponseInfo {           // new
    uint32 response_code = 1;
    repeated string ips = 2;
    repeated string cnames = 3;
}

Motivation

CWS-6595.

Activity dumps and workload profiles record which domains a workload queried, but not what those queries resolved to. That leaves the DNS section of a profile impossible to correlate with the bind / connect / flow nodes sitting in the same tree.

The agent already has this data and throws it away: the probe decodes the full DNS response, extracts the IPs and CNAMEs, and hands the event to the activity tree, which drops the answers because the wire format has nowhere to put them. This PR is the schema half of the fix; the agent-side change follows once this is tagged.

Additional Notes

Why nested per question rather than flat lists on DNSNode. DNS answers are per question type — A returns IPv4, AAAA returns IPv6 — and DNSNode.Requests is already a per-qtype list. Pooling ips / cnames onto DNSNode would have been a smaller wire format but would lose which question produced which answer, and give the response code nowhere sensible to live.

Why string for IPs. Matches the existing convention in this file (BindNode.ip, IPPortContext.ip) and the agent's existing utils.GetIPStringFromIPNet helper used by the event serializer.

Regeneration. Only the CWS Activity Dumps v1 target was regenerated (activity_dump.pb.go, activity_dump_vtproto.pb.go, and the Java bindings). Running the full codegen.protoc task would have churned every unrelated payload in the repo. The generated diff is larger than the schema change because inserting a message renumbers the downstream msgTypes indices; there is no generator-version drift.

Possible Drawbacks / Trade-offs

Compatibility — purely additive, per the checklist in REVIEWING.md:

  • Old sender → new receiver: fine, response is simply absent and decodes to nil.
  • New sender → old receiver: fine, field 6 lands in unknownFields and is ignored.
  • No field IDs are removed, so nothing needs reserved.

Payload size — dumps grow for workloads that do DNS. The agent side bounds this: answers accumulate as a set capped at 32 IPs and 8 CNAMEs per question, and profiles remain subject to the existing security_profile.v2.max_dump_size cap.

Describe how to test/QA your changes

go build ./cws/... plus a marshal → unmarshal round trip over SecDump. I verified locally that:

  • resolved IPs (v4 and v6) and CNAMEs survive MarshalVT / UnmarshalVT;
  • a DNSInfo with no answers decodes back to a nil response rather than an empty struct;
  • the vtproto and standard proto.Marshal paths agree (proto.Equal);
  • the ReturnToVTPool reset path is clean, since Profile.EncodeJSON relies on it.

Functional coverage lives in the consuming datadog-agent PR, which adds unit tests for the merge and encode/decode paths and an activity-dump functional test asserting resolved IPs reach a real dump.

Reviewer's Checklist

Reviewers: please see the review guidelines.

Activity dumps and workload profiles record which domains a workload
queried, but not what those queries resolved to, so the DNS section of a
profile cannot be correlated with the bind/connect/flow nodes in the same
tree.

Add a DNSResponseInfo message carrying the response code, the resolved IP
addresses and the CNAME targets, nested on DNSInfo as field 6. Nesting it
per question rather than pooling it on DNSNode keeps A and AAAA answers
attached to the question that produced them, matching the existing
per-qtype DNSInfo list.

IPs are strings, consistent with BindNode.ip and IPPortContext.ip.

The change is purely additive: a sender using the old definition and a
receiver using the new one interoperate in both directions, and no field
IDs are removed or reused.

Regenerated the Go and Java bindings for the CWS Activity Dumps v1 target.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@homoeconomics
homoeconomics requested a review from a team as a code owner August 25, 2026 22:02

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9e187debf5

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread proto/cws/dumpsv1/activity_dump.proto Outdated
// The IPs and CNAMEs are accumulated across every response seen for that
// question, so this is a union rather than a single response.
message DNSResponseInfo {
uint32 response_code = 1;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve every observed DNS response code

When one aggregated question returns different outcomes over time—for example, NXDOMAIN first and NOERROR with an IP later—this scalar can retain only one response code, even though the surrounding message explicitly unions answers across every observed response. Whichever code the producer retains or overwrites, the dump loses the other outcome and can misleadingly associate the unioned IPs/CNAMEs with a single status; represent observed codes as a set or per-code counts instead.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, fixed in db9b40aresponse_code is now repeated uint32 response_codes.

You identified a real inconsistency: I was unioning ips and cnames across every response for a question but keeping status as a scalar. The damaging ordering is NOERROR-then-NXDOMAIN, where last-write-wins leaves a record claiming the domain doesn't exist while still listing the addresses it resolved to. All three fields are now unions with the same semantics, so both outcomes survive in either order.

Regression test on the agent side asserts exactly that, in both orderings:

root.insert(dnsResponse("flap.example.com", A, NOERROR, []net.IPNet{ip("1.2.3.4")}))
root.insert(dnsResponse("flap.example.com", A, NXDOMAIN, nil))
// ResponseCodes == {NOERROR, NXDOMAIN}, IPs still {1.2.3.4}

I verified it isn't vacuous by reverting to the scalar behaviour — three subtests fail.

On the other half of the suggestion: I did not add per-code counts, deliberately. DNS samples are rate limited in the kernel before user space sees them, so a count here would report how many samples happened to survive the limiter, not how often the workload actually observed that outcome — misleading in a way the bare set is not. It would also cut against the surrounding design, which discards frequency at the kernel boundary on purpose (the activity tree records which targets a workload touched, not how many times). Codes are capped at 8 per question as a backstop; RFC 6895 leaves few enough that real traffic won't approach it.

One thing worth flagging for human reviewers: per-response fidelity is still not preserved, and that's intentional. You can see that a question returned both NOERROR and NXDOMAIN, but not which response produced which IP. Keeping that would mean storing a list of individual responses, which defeats the aggregation this message exists for and would grow without bound for CDN-backed domains.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded: a reviewer asked for response codes to be dropped entirely, so DNSResponseInfo is now just ips and cnames (3b39e70). With no status field left there is nothing to contradict the unioned answers, so this is resolved by removal rather than by the union I'd added.

Review feedback on DataDog#515: `response_code` was a scalar while `ips` and
`cnames` were explicitly unioned across every response observed for a
question. A question that resolved and later went NXDOMAIN would keep only
one of the two codes, so the dump could report NXDOMAIN while still listing
the addresses the domain had resolved to — a record that contradicts itself.

Make it `repeated uint32 response_codes` so all three fields are unions
with the same semantics.

Occurrence counts were also suggested and are deliberately not added. DNS
samples are rate limited in the kernel before user space sees them, so a
count would report how many samples happened to survive rather than how
often the workload observed that outcome.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Comment thread proto/cws/dumpsv1/activity_dump.proto Outdated
Review feedback on DataDog#515: response codes are not needed here, the resolved
IPs and CNAME targets are enough. They were beyond the scope of CWS-6595,
which asks for the IPs and CNAMEs.

Removing them also moots the earlier scalar-vs-union inconsistency: the two
remaining fields are both unions accumulated across every response observed
for a question, with no status field left to contradict them.

DNSResponseInfo is new in this PR and has never been released, so ips and
cnames are renumbered to 1 and 2 rather than leaving a reserved gap.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@homoeconomics

Copy link
Copy Markdown
Contributor Author

/merge

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Sep 3, 2026

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-09-03 13:53:13 UTC ℹ️ Start processing command /merge


2026-09-03 13:53:20 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 3m (p90).


2026-09-03 13:55:46 UTC ℹ️ MergeQueue: This merge request was merged

@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot merged commit 84910fb into DataDog:master Sep 3, 2026
6 checks passed
homoeconomics added a commit to DataDog/datadog-agent that referenced this pull request Sep 9, 2026
Picks up DNSResponseInfo from DataDog/agent-payload#515, which the DNS
response encoding depends on.

This pinned the pseudo-version v5.0.210-0.20260903135538-84910fb4573a
before, because agent-payload had merged #515 but not yet cut a tag. The
v5.0.210 tag now exists and contains that merge, so this is the plain tag.

tasks/agent_payload_version.bzl is regenerated by dda inv tidy.

Environment: Datadog workspace

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
homoeconomics added a commit to DataDog/datadog-agent that referenced this pull request Sep 10, 2026
Picks up DNSResponseInfo from DataDog/agent-payload#515, which the DNS
response encoding depends on.

This pinned the pseudo-version v5.0.210-0.20260903135538-84910fb4573a
before, because agent-payload had merged #515 but not yet cut a tag. The
v5.0.210 tag now exists and contains that merge, so this is the plain tag.

tasks/agent_payload_version.bzl is regenerated by dda inv tidy.

Environment: Datadog workspace

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants