[ctrace] Trace decoder utility development

[thorstendb-ARM]

Purpose

ctrace combines a CMSIS trace-run configuration with raw CoreSight trace data. It decodes the trace into backend-independent events and can produce CSV or CTF output for further analysis, including Trace Compass.

Architecture

Architecture description

Supported today

• Raw SWO stream decoding through OpenCSD
• ITM software trace packets on stimulus ports 1 through 31
• DWT data and address trace, including configured value types and optional PC/address information
• Exception and interrupt trace, including entry, exit, and return
• Periodic PC sampling, including processor-sleep indications and a Trace Compass sleep timeline
• Architectural DWT event-counter and PMU trace-on-overflow output; PMU counters currently use provisional names Event0 through Event7
• Armv8-M DWT comparator-match packets without additional data
• Local and Global Timestamp packets, hardware synchronization, overflow, decoder-error, and explicit data-loss events
• CSV, CTF 1.8, and Trace Compass XML output
• Target, event-type, and CoreSight Trace Bus ID filtering
• Stream-specific decoder state and CoreSight Trace Bus routing context
• Validation without generating output files
• Processing of multiple solution sets
• Reporting of info, warning, and error annotations attached to consumed trace-run references
• Integration tests for regular SWO input, reset-like recovery, PC sampling, event counters, PMU packets, and DWT match packets

ITM stimulus port 0 is decoded for stream integrity but intentionally excluded from CSV and CTF payload output. Formatted Trace Buffer input is discovered but currently reported and skipped.

OpenCSD error handling

Invalid ITM packet sequences and packet headers are treated as recoverable stream errors:

1. Events before the reported raw offset are retained.
2. Events at or after the failing offset are discarded.
3. The OpenCSD decoder is reset.
4. Decoding resumes only after OpenCSD finds a real hardware ITM synchronization sequence.
5. The lost byte range is emitted as an ordered error/data-loss event.
6. Pending DWT correlation is cleared and timestamp quality is marked unreliable.

Recoverable trace corruption is reported as [error] and causes a non-zero exit status, but decoding continues after the next valid synchronization point.

A non-recoverable OpenCSD failure aborts the current raw input and removes incomplete output artifacts. Other solution sets can still be processed.

Implemented pull requests

• Initial ctrace implementation – #2548
• Align trace-run metadata with the specification – #2581
• Preserve exception returns in CTF – #2587
• Add periodic PC sampling and sleep output – #2588
• Recognize PMU trace-on-overflow packets – #2589
• Report trace-run reference diagnostics – #2590
• Output DWT and PMU event counters – #2591
• Decode Armv8-M DWT match packets – #2592

Current development

• Decode compressed Armv8-M DWT packets – #2593
• Preserve DWT address-fragment widths – #2594
• Document the generated CMSIS CTF profile, event groups, field semantics, and Trace Compass representation

Remaining work

DWT and PMU

• Integrate and reconcile the compressed-packet and address-fragment changes from feat(ctrace): decode compressed Armv8-M DWT packets #2593 and feat(ctrace): preserve DWT address fragment widths #2594 after review
• Preserve logical trace references, processor bindings, grouped resources, and exact setup bindings when source arrays are expanded
• Complete Armv7-M linked-comparator, range, and value-match semantics
• Complete remaining Armv8-M and Armv8.1-M linked-comparator and range reconstruction
• Resolve programmable PMU counter assignments and names from trace-run configuration
• Validate PMU and the remaining Armv8-M packet paths with hardware captures

Multi-CPU and multiple trace sources

• Preserve processor identity and map CoreSight Trace Bus IDs to the corresponding pname
• Decode formatted streams from multiple sources through an OpenCSD DecodeTree
• Represent CPU identity consistently in backend-independent events, CSV, CTF, and Trace Compass
• Support separate trace clock domains and cross-stream time synchronization

Trace Buffer, ETB, and ETF

• Finalize trace-channel declaration and raw-file discovery instead of adding further filename heuristics; see ctrace: Support default file name scheme changes in spec (for TB channel) #2573
• Read Trace Buffer captures produced by ETB/ETF components
• Decode formatted Trace Buffer input by CoreSight Trace Bus ID
• Reuse the existing diagnostics, selection, and output pipeline for decoded Trace Buffer events

ETB and ETF are trace storage and transport components. They must not be conflated with ETM, which is an instruction-trace source that can be carried by such a formatted stream.

Instruction trace

• Add ETM instruction-trace decoding
• Supply target-image and memory-map access required by the ETM decoder
• Define CSV, CTF, and Trace Compass representations for decoded ETM instruction trace
• Add MTB instruction-trace decoding and output

CTF, dependencies, and release

• Publish and maintain the CMSIS CTF event-profile documentation with the implementation
• Review cmsis_ctf_profile_version whenever a compatibility-relevant event layout changes
• Replace OpenCSD private common/ and interfaces/ headers with supported public APIs
• Update OpenCSD after the documented empty-buffer issue is fixed upstream
• Complete release archive, license, provenance, signing, SBOM, and platform validation

References

• CMSIS-Toolbox Trace specification
• Trace-directory naming and discovery – #2573
• OpenCSD CoreSight trace decoder
• OpenCSD DecodeTree
• OpenCSD decoder configuration and memory images
• Eclipse Trace Compass documentation
• Common Trace Format 1.8.3 specification
• CTF file-content discussion – vscode-cmsis-debugger #996
• Trace integration – vscode-cmsis-debugger #315
• Trace configuration – vscode-cmsis-debugger #983

[jreineckearm]

Let's make sure to investigate use of a DecodeTree as soon as we go for a formatted trace stream from multiple sources. This seems to be the recommended OpenCSD way for scalability.

Same for how for extra info for instruction trace decoding: Configuring Decoders, Adding in Memory Images.