cws: add DNS response IPs and CNAMEs to activity dumps - #515
Conversation
Activity dumps and workload profiles record which domains a workload queried, but not what those queries resolved to, so the DNS section of a profile cannot be correlated with the bind/connect/flow nodes in the same tree. Add a DNSResponseInfo message carrying the response code, the resolved IP addresses and the CNAME targets, nested on DNSInfo as field 6. Nesting it per question rather than pooling it on DNSNode keeps A and AAAA answers attached to the question that produced them, matching the existing per-qtype DNSInfo list. IPs are strings, consistent with BindNode.ip and IPPortContext.ip. The change is purely additive: a sender using the old definition and a receiver using the new one interoperate in both directions, and no field IDs are removed or reused. Regenerated the Go and Java bindings for the CWS Activity Dumps v1 target. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9e187debf5
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| // The IPs and CNAMEs are accumulated across every response seen for that | ||
| // question, so this is a union rather than a single response. | ||
| message DNSResponseInfo { | ||
| uint32 response_code = 1; |
There was a problem hiding this comment.
Preserve every observed DNS response code
When one aggregated question returns different outcomes over time—for example, NXDOMAIN first and NOERROR with an IP later—this scalar can retain only one response code, even though the surrounding message explicitly unions answers across every observed response. Whichever code the producer retains or overwrites, the dump loses the other outcome and can misleadingly associate the unioned IPs/CNAMEs with a single status; represent observed codes as a set or per-code counts instead.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Good catch, fixed in db9b40a — response_code is now repeated uint32 response_codes.
You identified a real inconsistency: I was unioning ips and cnames across every response for a question but keeping status as a scalar. The damaging ordering is NOERROR-then-NXDOMAIN, where last-write-wins leaves a record claiming the domain doesn't exist while still listing the addresses it resolved to. All three fields are now unions with the same semantics, so both outcomes survive in either order.
Regression test on the agent side asserts exactly that, in both orderings:
root.insert(dnsResponse("flap.example.com", A, NOERROR, []net.IPNet{ip("1.2.3.4")}))
root.insert(dnsResponse("flap.example.com", A, NXDOMAIN, nil))
// ResponseCodes == {NOERROR, NXDOMAIN}, IPs still {1.2.3.4}I verified it isn't vacuous by reverting to the scalar behaviour — three subtests fail.
On the other half of the suggestion: I did not add per-code counts, deliberately. DNS samples are rate limited in the kernel before user space sees them, so a count here would report how many samples happened to survive the limiter, not how often the workload actually observed that outcome — misleading in a way the bare set is not. It would also cut against the surrounding design, which discards frequency at the kernel boundary on purpose (the activity tree records which targets a workload touched, not how many times). Codes are capped at 8 per question as a backstop; RFC 6895 leaves few enough that real traffic won't approach it.
One thing worth flagging for human reviewers: per-response fidelity is still not preserved, and that's intentional. You can see that a question returned both NOERROR and NXDOMAIN, but not which response produced which IP. Keeping that would mean storing a list of individual responses, which defeats the aggregation this message exists for and would grow without bound for CDN-backed domains.
There was a problem hiding this comment.
Superseded: a reviewer asked for response codes to be dropped entirely, so DNSResponseInfo is now just ips and cnames (3b39e70). With no status field left there is nothing to contradict the unioned answers, so this is resolved by removal rather than by the union I'd added.
Review feedback on DataDog#515: `response_code` was a scalar while `ips` and `cnames` were explicitly unioned across every response observed for a question. A question that resolved and later went NXDOMAIN would keep only one of the two codes, so the dump could report NXDOMAIN while still listing the addresses the domain had resolved to — a record that contradicts itself. Make it `repeated uint32 response_codes` so all three fields are unions with the same semantics. Occurrence counts were also suggested and are deliberately not added. DNS samples are rate limited in the kernel before user space sees them, so a count would report how many samples happened to survive rather than how often the workload observed that outcome. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Review feedback on DataDog#515: response codes are not needed here, the resolved IPs and CNAME targets are enough. They were beyond the scope of CWS-6595, which asks for the IPs and CNAMEs. Removing them also moots the earlier scalar-vs-union inconsistency: the two remaining fields are both unions accumulated across every response observed for a question, with no status field left to contradict them. DNSResponseInfo is new in this PR and has never been released, so ips and cnames are renumbered to 1 and 2 rather than leaving a reserved gap. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
|
84910fb
into
DataDog:master
Picks up DNSResponseInfo from DataDog/agent-payload#515, which the DNS response encoding depends on. This pinned the pseudo-version v5.0.210-0.20260903135538-84910fb4573a before, because agent-payload had merged #515 but not yet cut a tag. The v5.0.210 tag now exists and contains that merge, so this is the plain tag. tasks/agent_payload_version.bzl is regenerated by dda inv tidy. Environment: Datadog workspace Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Picks up DNSResponseInfo from DataDog/agent-payload#515, which the DNS response encoding depends on. This pinned the pseudo-version v5.0.210-0.20260903135538-84910fb4573a before, because agent-payload had merged #515 but not yet cut a tag. The v5.0.210 tag now exists and contains that merge, so this is the plain tag. tasks/agent_payload_version.bzl is regenerated by dda inv tidy. Environment: Datadog workspace Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
What does this PR do?
Adds a
DNSResponseInfomessage to the CWS activity-dump v1 schema, carrying the DNS response code, the resolved IP addresses, and the CNAME targets. It is nested onDNSInfoas field 6, so the answers stay attached to the question that produced them.Motivation
CWS-6595.
Activity dumps and workload profiles record which domains a workload queried, but not what those queries resolved to. That leaves the DNS section of a profile impossible to correlate with the
bind/connect/ flow nodes sitting in the same tree.The agent already has this data and throws it away: the probe decodes the full DNS response, extracts the IPs and CNAMEs, and hands the event to the activity tree, which drops the answers because the wire format has nowhere to put them. This PR is the schema half of the fix; the agent-side change follows once this is tagged.
Additional Notes
Why nested per question rather than flat lists on
DNSNode. DNS answers are per question type — A returns IPv4, AAAA returns IPv6 — andDNSNode.Requestsis already a per-qtype list. Poolingips/cnamesontoDNSNodewould have been a smaller wire format but would lose which question produced which answer, and give the response code nowhere sensible to live.Why
stringfor IPs. Matches the existing convention in this file (BindNode.ip,IPPortContext.ip) and the agent's existingutils.GetIPStringFromIPNethelper used by the event serializer.Regeneration. Only the CWS Activity Dumps v1 target was regenerated (
activity_dump.pb.go,activity_dump_vtproto.pb.go, and the Java bindings). Running the fullcodegen.protoctask would have churned every unrelated payload in the repo. The generated diff is larger than the schema change because inserting a message renumbers the downstreammsgTypesindices; there is no generator-version drift.Possible Drawbacks / Trade-offs
Compatibility — purely additive, per the checklist in
REVIEWING.md:responseis simply absent and decodes tonil.unknownFieldsand is ignored.reserved.Payload size — dumps grow for workloads that do DNS. The agent side bounds this: answers accumulate as a set capped at 32 IPs and 8 CNAMEs per question, and profiles remain subject to the existing
security_profile.v2.max_dump_sizecap.Describe how to test/QA your changes
go build ./cws/...plus a marshal → unmarshal round trip overSecDump. I verified locally that:MarshalVT/UnmarshalVT;DNSInfowith no answers decodes back to anilresponse rather than an empty struct;proto.Marshalpaths agree (proto.Equal);ReturnToVTPoolreset path is clean, sinceProfile.EncodeJSONrelies on it.Functional coverage lives in the consuming datadog-agent PR, which adds unit tests for the merge and encode/decode paths and an activity-dump functional test asserting resolved IPs reach a real dump.
Reviewer's Checklist
Reviewers: please see the review guidelines.