From 8918c4fec4a99b174d480c043000fd4405678282 Mon Sep 17 00:00:00 2001 From: hathach Date: Thu, 23 Jul 2026 10:18:11 +0700 Subject: [PATCH 01/11] docs(skills): rename debug skills, drop the PC-host/TinyUSB-device assumption Rename usb-target-debug -> target-debug, usb-debug -> usb-kernel-debug, usb-recover -> usb-kernel-recover (script filenames unchanged), and make all debug skills/agents decide tool applicability by which end of the link runs Linux: TinyUSB may run the device or host stack, and its peer may be a Linux PC, another TinyUSB board, or a Linux gadget (e.g. Raspberry Pi UDC). - usbmon: exists only when a Linux PC is the link's host - usb-kernel-debug: either Linux end; allowlist gains dwc3/libcomposite/udc_core for the gadget side of a Linux peer - usb-sniffer: the only full-visibility capture when TinyUSB is the host - target-debug: covers dcd_* and hcd_*/tuh_ debugging; channel choice by topology - update target-debugger/hil-operator agents, pre-pr, hil-validate.js, and the USB_RECOVER path constant in test/hil/usbtest.py - CLAUDE.md: fold the dcd/hcd datasheet cross-check rule into the read-doc line --- .claude/agents/hil-operator.md | 6 +-- .claude/agents/target-debugger.md | 31 +++++++----- .claude/skills/pre-pr/SKILL.md | 2 +- .../SKILL.md | 42 ++++++++++------- .claude/skills/usb-debug/SKILL.md | 36 -------------- .claude/skills/usb-kernel-debug/SKILL.md | 47 +++++++++++++++++++ .../scripts/usb_dyndbg.sh | 11 +++-- .../SKILL.md | 6 +-- .../scripts/usb_recover.sh | 0 .claude/skills/usb-sniffer/SKILL.md | 14 +++--- .claude/skills/usbmon/SKILL.md | 4 +- .claude/skills/usbtest/SKILL.md | 2 +- .claude/workflows/hil-validate.js | 2 +- CLAUDE.md | 2 +- test/hil/usbtest.py | 6 +-- 15 files changed, 120 insertions(+), 91 deletions(-) rename .claude/skills/{usb-target-debug => target-debug}/SKILL.md (81%) delete mode 100644 .claude/skills/usb-debug/SKILL.md create mode 100644 .claude/skills/usb-kernel-debug/SKILL.md rename .claude/skills/{usb-debug => usb-kernel-debug}/scripts/usb_dyndbg.sh (74%) rename .claude/skills/{usb-recover => usb-kernel-recover}/SKILL.md (91%) rename .claude/skills/{usb-recover => usb-kernel-recover}/scripts/usb_recover.sh (100%) diff --git a/.claude/agents/hil-operator.md b/.claude/agents/hil-operator.md index c48ceb8bd..d19eca047 100644 --- a/.claude/agents/hil-operator.md +++ b/.claude/agents/hil-operator.md @@ -8,8 +8,8 @@ model: sonnet You operate physical USB test hardware. These repo skills are your source of truth — read the relevant one BEFORE acting: - `.claude/skills/hil/SKILL.md` — run `hostname` first (host `ci` = local mode with `test/hil/tinyusb.json`; host `htpc` = local `local.json` or remote via `test/hil/hil_ci.sh`); the board lock protocol; exact `hil_test.py` invocations. -- `.claude/skills/usb-recover/SKILL.md` — only when a device/fixture is wedged or processes hang in D state. -- `.claude/skills/usb-debug/SKILL.md` — only when you need to explain WHY the host rejected a device (dmesg analysis). +- `.claude/skills/usb-kernel-recover/SKILL.md` — only when a device/fixture on the rig's Linux host is wedged or processes hang in D state. +- `.claude/skills/usb-kernel-debug/SKILL.md` — only when you need to explain WHY the Linux kernel rejected a device (dmesg analysis). ## Board lock protocol (CI runs concurrently — NEVER stop the actions-runner) @@ -30,7 +30,7 @@ The GitHub Actions runner keeps running during your work. Per-board flock locks - HIL runs take 2–5 min per board: use Bash timeouts >= 20 min (1200000 ms) and NEVER cancel early. - One hardware action at a time. You are never run concurrently with another hil-operator. -- On test failure: retry once with `-v -r 1` appended (one verbose attempt for diagnosis — the first run already did the flake-retries). If a board/fixture stops enumerating or tools hang in D state, consult usb-recover and capture `dmesg | tail -50` into `detail`; set `wedged` true. +- On test failure: retry once with `-v -r 1` appended (one verbose attempt for diagnosis — the first run already did the flake-retries). If a board/fixture stops enumerating or tools hang in D state, consult usb-kernel-recover and capture `dmesg | tail -50` into `detail`; set `wedged` true. ## Output contract diff --git a/.claude/agents/target-debugger.md b/.claude/agents/target-debugger.md index c1df47cb2..861600428 100644 --- a/.claude/agents/target-debugger.md +++ b/.claude/agents/target-debugger.md @@ -1,25 +1,32 @@ --- name: target-debugger -description: Root-cause one USB misbehavior on real HIL hardware by instrumenting the TinyUSB device side — TU_LOG/RTT, RAM ring-buffer trace, GDB autopsy, J-Link PC-sampling — correlated with host-side and wire-level capture. Long serial debug loop under one held board lock; strictly one instance. Produces a diagnosis with on-target evidence (plus a candidate fix when one emerges), never a merged patch. +description: Root-cause one USB misbehavior on real HIL hardware by instrumenting the TinyUSB target — device or host stack — with TU_LOG/RTT, RAM ring-buffer trace, GDB autopsy, J-Link PC-sampling, correlated with capture from the link's other end (Linux PC host, another TinyUSB board, or a Linux gadget peer) and the wire. Long serial debug loop under one held board lock; strictly one instance. Produces a diagnosis with on-target evidence (plus a candidate fix when one emerges), never a merged patch. model: opus --- You debug one failing USB behavior on one physical board until you can name the -mechanism — or report exactly what you ruled out. These repo skills are your -source of truth; read the relevant SKILL.md BEFORE acting: +mechanism — or report exactly what you ruled out. The target may run the device +stack, the host stack, or both; its link peer may be the Linux PC, another +TinyUSB board, or a Linux gadget (e.g. a Raspberry Pi) — pick capture channels +by which end runs Linux, not by habit. These repo skills are your source of +truth; read the relevant SKILL.md BEFORE acting: -- `.claude/skills/usb-target-debug/SKILL.md` — your primary playbook: technique - choice by intrusiveness, capture recipes, GDB autopsy, all rig warnings. +- `.claude/skills/target-debug/SKILL.md` — your primary playbook: technique + choice by intrusiveness, channel choice by link topology, capture recipes, + GDB autopsy, all rig warnings. - `.claude/skills/hil/SKILL.md` — host/config selection, board lock protocol, `hil_test.py` invocation. -- `.claude/skills/usbmon/SKILL.md` — host-side URB capture (the default posture - is dual-side: host + target simultaneously). +- `.claude/skills/usbmon/SKILL.md` — Linux-host URB capture; exists only when a + Linux PC is the link's host (the default posture is dual-side: both ends + simultaneously). - `.claude/skills/usb-sniffer/SKILL.md` — wire-level capture with the hardware - tap, when the host can't see the bus (device never enumerates, pre-URB - failures) or when usbmon and device logs disagree — the wire arbitrates. -- `.claude/skills/usb-debug/SKILL.md` — why the host acted (dmesg/dynamic debug). -- `.claude/skills/usb-recover/SKILL.md` — only when the DUT or fixture wedges - the host stack. + tap: when the host can't see the bus (device never enumerates, pre-URB + failures), when usbmon and target logs disagree — the wire arbitrates — or + when TinyUSB is the host and no end has usbmon. +- `.claude/skills/usb-kernel-debug/SKILL.md` — why the Linux kernel acted + (dmesg/dynamic debug); the PC host, or a Linux gadget peer's device side. +- `.claude/skills/usb-kernel-recover/SKILL.md` — only when the DUT or fixture + wedges the rig PC's Linux host stack. ## The loop (deliberately serial — no fan-out) diff --git a/.claude/skills/pre-pr/SKILL.md b/.claude/skills/pre-pr/SKILL.md index d4c35f7e5..3f062db78 100644 --- a/.claude/skills/pre-pr/SKILL.md +++ b/.claude/skills/pre-pr/SKILL.md @@ -38,5 +38,5 @@ Invoke the Workflow tool: - Per-stage table: unit / build: / size / pvs, then HIL per board — pass/fail with the first error for each failure. - If the hardware result has non-empty `locked` (a CI job held those boards): ask the user with AskUserQuestion — **Force now** (re-invoke `hil-validate` with `force: true` for those boards; user accepts the risk of colliding with a mid-test CI job), **Keep waiting** (re-invoke `hil-validate` for them after a few minutes; ask again if still locked), or **Accept** the partial verdict. Never force without the user's answer. -- Wedged boards: point at `.claude/skills/usb-recover/SKILL.md`. +- Wedged boards: point at `.claude/skills/usb-kernel-recover/SKILL.md`. - End with a clear ship / no-ship verdict and what to fix first. diff --git a/.claude/skills/usb-target-debug/SKILL.md b/.claude/skills/target-debug/SKILL.md similarity index 81% rename from .claude/skills/usb-target-debug/SKILL.md rename to .claude/skills/target-debug/SKILL.md index 71fac98f2..c82237ea3 100644 --- a/.claude/skills/usb-target-debug/SKILL.md +++ b/.claude/skills/target-debug/SKILL.md @@ -1,23 +1,28 @@ --- -name: usb-target-debug -description: Use when a TinyUSB device misbehaves on real hardware and host-side capture can't explain it — a HIL test fails but usbmon shows only Submits with no Completes, the device silently NAKs, wedges, STALLs, babbles, or drops data, EP0 starves, an ISR or DCD/HCD state bug is suspected — and you need device-side evidence: TU_LOG/RTT logs, GDB state dumps, a RAM ring-buffer event trace, or PC-sampling of where the core spins. +name: target-debug +description: Use when TinyUSB firmware — device or host stack — misbehaves on real hardware and capture from the other end can't explain it: a HIL test fails but usbmon shows only Submits with no Completes, the device silently NAKs, wedges, STALLs, babbles, or drops data, EP0 starves, tuh_ enumeration of an attached device fails, an ISR or DCD/HCD state bug is suspected — and you need target-side evidence: TU_LOG/RTT logs, GDB state dumps, a RAM ring-buffer event trace, or PC-sampling of where the core spins. --- -# usb-target-debug — device-side capture & debugging on the HIL rig +# target-debug — target-side capture & debugging on the HIL rig -Completes the debugging trio (the `usb-sniffer` skill adds a fourth, -wire-level view when hardware tapping is available): +The **target** is whichever MCU runs TinyUSB — device stack (`dcd_*`), host +stack (`hcd_*`/`tuh_*`), or both. Its link peer is not always a Linux PC: a +TinyUSB host may face another TinyUSB board or a Linux gadget (e.g. a +Raspberry Pi). Pick capture channels by which end runs Linux, not by habit: -| Skill | Answers | -|---|---| -| `usbmon` | what the host actually exchanged (URBs) | -| `usb-debug` | why the host acted (dmesg / dynamic debug) | -| **`usb-target-debug`** | **what the device did** (logs, driver state, PC) | -| `usb-sniffer` | what crossed the wire (PIDs, handshakes, resets — hardware tap) | +| Skill | Answers | Exists when | +|---|---|---| +| `usbmon` | what the Linux host exchanged (URBs) | a Linux PC is the link's host | +| `usb-kernel-debug` | why the Linux kernel acted (dmesg / dynamic debug) | Linux on either end: PC host or Linux gadget peer | +| **`target-debug`** | **what the target did** (logs, driver state, PC) | always — either role, needs a debug probe | +| `usb-sniffer` | what crossed the wire (PIDs, handshakes, resets) | hardware tap cabled in — role-agnostic | For enumeration/transfer bugs the default posture is **dual-side capture** — -usbmon on the host *and* a target-side channel, simultaneously — not -host-first-then-escalate. +both ends simultaneously, not one-side-first-then-escalate: usbmon plus a +target channel when a Linux PC is the host. When TinyUSB is the host there is +no usbmon on either end — pair the target channel with the wire +(`usb-sniffer`) and, if the peer is a Linux gadget, `usb-kernel-debug` on the +peer. ## Rig discipline — lock first, always @@ -91,7 +96,8 @@ wedged; do not rebuild while the wedge is still on the board. The debug-loop specifics: ```gdb -p/x _usbd_dev.ep_status # usbd core [epnum][dir] (1=IN): busy/stalled/claimed +p/x _usbd_dev.ep_status # device stack: usbd [epnum][dir] (1=IN): busy/stalled/claimed +p _usbh_devices[0] # host stack: usbh per-device state (addr, enum/config) p/x # per-port names — read the board's dcd_*.c first x/32wx # raw EP/FIFO regs; base = the macro the dcd uses watch xfer_status[2][1].total_len # HW watchpoint (Cortex-M: ~4); dwc2 names shown @@ -153,7 +159,9 @@ top entries are the spin site; a flat histogram = core is servicing normally. ## Dual-side capture — the default for enumeration/transfer bugs -Start both channels, then trigger the failing test: +Start both channels, then trigger the failing test (Linux-PC-host link shown; +TinyUSB-as-host: swap the usbmon line for a `usb-sniffer` capture, plus +`usb-kernel-debug` on the peer if it is a Linux gadget): ```bash .claude/skills/usbmon/scripts/usbcap.sh cafe: 30 /tmp/host.pcapng & # host URBs (usbmon skill) @@ -172,8 +180,8 @@ the wire itself: `usb-sniffer` skill (hardware tap, PID-level). - **Halting/resetting via the probe does NOT disconnect the device**: a DWC2 soft-connect pullup stays up through core halt *and* reset, so the host's - stuck URBs stay stuck and a wedged DUT stays wedged — recover the host side - with the `usb-recover` skill. + stuck URBs stay stuck and a wedged DUT stays wedged — recover the Linux + host side with the `usb-kernel-recover` skill. - **A bug that vanishes under LOG=2 is a timing bug**, not fixed: switch to the ring buffer; if it vanishes under GDB too, PC-sampling only. - **UART TU_LOG blocks in the write path** (worst perturbation, including diff --git a/.claude/skills/usb-debug/SKILL.md b/.claude/skills/usb-debug/SKILL.md deleted file mode 100644 index 20ab7d764..000000000 --- a/.claude/skills/usb-debug/SKILL.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -name: usb-debug -description: Use when USB enumeration fails or misbehaves and usbmon alone can't explain WHY the host acted — port reset storms, repeated re-enumeration, address errors, xHCI ring/command errors, "device descriptor read error", babble, or when you need the host driver's own reasoning from dmesg on the ci HIL rig. ---- - -# usb-debug — host-side kernel dynamic debug for USB - -usbmon shows the URBs; kernel **dynamic debug** shows the host driver's -*reasoning* usbmon can't: port resets and their causes, enumeration retries, -address (re)assignment, EP halts, xHCI ring/command errors. - -Run this skill's `scripts/usb_dyndbg.sh` with `sudo` (abbreviated to -`usb_dyndbg.sh` in the examples below). It flips the dynamic-debug print flag -for an allowlisted set of USB host modules only: - -```bash -sudo usb_dyndbg.sh on usbcore xhci_hcd # enable +p; pick modules from `lsusb -t` Driver= -sudo usb_dyndbg.sh status [module] # list enabled print sites -sudo usb_dyndbg.sh off usbcore xhci_hcd # ALWAYS turn off when done — very noisy -``` - -Allowlisted modules: `usbcore xhci_hcd xhci_pci xhci_pci_renesas ehci_hcd -ehci_pci ohci_hcd ohci_pci uhci_hcd dwc2 cdc_acm usb_storage uas`. - -## Workflow - -1. `sudo usb_dyndbg.sh on usbcore ` — `usbcore` for enumeration/hub - logic, plus the controller module (`lsusb -t` shows the driver per bus). -2. Reproduce (replug / re-enumerate / rerun the failing test) while following - `sudo dmesg -w` (or grab `sudo dmesg | tail` afterwards). -3. `sudo usb_dyndbg.sh off ...` — leaving it on floods the log and skews timing. - -Pair with the `usbmon` skill: usbmon for what crossed the bus, dynamic debug for -why the host reacted. For a wedged device/bus use the `usb-recover` skill. - -Requires `CONFIG_DYNAMIC_DEBUG` and mounted debugfs (standard on distro kernels). diff --git a/.claude/skills/usb-kernel-debug/SKILL.md b/.claude/skills/usb-kernel-debug/SKILL.md new file mode 100644 index 000000000..e4169b049 --- /dev/null +++ b/.claude/skills/usb-kernel-debug/SKILL.md @@ -0,0 +1,47 @@ +--- +name: usb-kernel-debug +description: Use when USB enumeration fails or misbehaves and packet/URB capture can't explain WHY the Linux kernel acted — port reset storms, repeated re-enumeration, address errors, xHCI ring/command errors, "device descriptor read error", babble — on whichever end of the link runs Linux: the PC host when testing a TinyUSB device, or a Linux gadget peer (e.g. Raspberry Pi) when testing the TinyUSB host stack. +--- + +# usb-kernel-debug — Linux kernel dynamic debug for USB + +Kernel **dynamic debug** shows the Linux side's *reasoning* that packet +capture can't: port resets and their causes, enumeration retries, address +(re)assignment, EP halts, xHCI ring/command errors. It applies wherever Linux +sits in the link — the rig PC when it is the host, or a Linux gadget peer +(dwc2/UDC + gadget modules) when TinyUSB is the host. It cannot see inside +the TinyUSB MCU — that is the `target-debug` skill. + +Run this skill's `scripts/usb_dyndbg.sh` with `sudo` (abbreviated to +`usb_dyndbg.sh` in the examples below). It flips the dynamic-debug print flag +for an allowlisted set of USB modules only: + +```bash +sudo usb_dyndbg.sh on usbcore xhci_hcd # enable +p; pick modules from `lsusb -t` Driver= +sudo usb_dyndbg.sh status [module] # list enabled print sites +sudo usb_dyndbg.sh off usbcore xhci_hcd # ALWAYS turn off when done — very noisy +``` + +Allowlisted modules: `usbcore xhci_hcd xhci_pci xhci_pci_renesas ehci_hcd +ehci_pci ohci_hcd ohci_pci uhci_hcd dwc2 dwc3 cdc_acm usb_storage uas +libcomposite udc_core` (`dwc2`/`dwc3` + the last two cover a Linux gadget +peer's device side). + +## Workflow + +1. `sudo usb_dyndbg.sh on usbcore ` — `usbcore` for enumeration/hub + logic, plus the controller module (`lsusb -t` shows the driver per bus). + On a gadget peer: `dwc2` (or `dwc3`) + `udc_core` + `libcomposite` instead — + run on the peer itself (its SSH/serial console); the script is self-contained, + copy it over or use the raw `dynamic_debug/control` writes from the `usbmon` + skill. +2. Reproduce (replug / re-enumerate / rerun the failing test) while following + `sudo dmesg -w` (or grab `sudo dmesg | tail` afterwards). +3. `sudo usb_dyndbg.sh off ...` — leaving it on floods the log and skews timing. + +On a Linux-PC-host link, pair with the `usbmon` skill: usbmon for what crossed +the bus, dynamic debug for why the kernel reacted. A gadget peer's UDC has no +usbmon — pair with `usb-sniffer` on the wire instead. For a wedged device/bus +on the rig PC use the `usb-kernel-recover` skill. + +Requires `CONFIG_DYNAMIC_DEBUG` and mounted debugfs (standard on distro kernels). diff --git a/.claude/skills/usb-debug/scripts/usb_dyndbg.sh b/.claude/skills/usb-kernel-debug/scripts/usb_dyndbg.sh similarity index 74% rename from .claude/skills/usb-debug/scripts/usb_dyndbg.sh rename to .claude/skills/usb-kernel-debug/scripts/usb_dyndbg.sh index 0dc880469..3923cdc6d 100755 --- a/.claude/skills/usb-debug/scripts/usb_dyndbg.sh +++ b/.claude/skills/usb-kernel-debug/scripts/usb_dyndbg.sh @@ -1,7 +1,7 @@ #!/usr/bin/env bash -# usb_dyndbg.sh — toggle kernel dynamic-debug on USB host drivers; run with sudo. -# Flips +p/-p only on an allowlisted set of USB modules, so it can't reach -# arbitrary kernel debug or unrelated subsystems. +# usb_dyndbg.sh — toggle kernel dynamic-debug on USB drivers (host or gadget +# side); run with sudo. Flips +p/-p only on an allowlisted set of USB modules, +# so it can't reach arbitrary kernel debug or unrelated subsystems. # # Usage: # sudo usb_dyndbg.sh on ... # enable +p (e.g. usbcore xhci_hcd) @@ -10,8 +10,9 @@ set -euo pipefail CTL=/sys/kernel/debug/dynamic_debug/control -# Allowlist: USB host-controller + core + common host class drivers. -ALLOW='usbcore xhci_hcd xhci_pci xhci_pci_renesas ehci_hcd ehci_pci ohci_hcd ohci_pci uhci_hcd dwc2 cdc_acm usb_storage uas' +# Allowlist: USB core + host-controller + common class drivers, plus the +# gadget/UDC side of a Linux peer (dwc2/dwc3, udc_core, libcomposite). +ALLOW='usbcore xhci_hcd xhci_pci xhci_pci_renesas ehci_hcd ehci_pci ohci_hcd ohci_pci uhci_hcd dwc2 dwc3 cdc_acm usb_storage uas libcomposite udc_core' die() { echo "usb_dyndbg: $*" >&2; exit 1; } usage() { diff --git a/.claude/skills/usb-recover/SKILL.md b/.claude/skills/usb-kernel-recover/SKILL.md similarity index 91% rename from .claude/skills/usb-recover/SKILL.md rename to .claude/skills/usb-kernel-recover/SKILL.md index beb6fd862..3f03722fe 100644 --- a/.claude/skills/usb-recover/SKILL.md +++ b/.claude/skills/usb-kernel-recover/SKILL.md @@ -1,9 +1,9 @@ --- -name: usb-recover -description: Use when a USB device or fixture on the ci HIL rig is stuck, hung, not enumerating, or wedged after a failed flash or test, or when processes touching USB (testusb, JLinkExe, uhubctl, libusb tools) start hanging in D state. +name: usb-kernel-recover +description: Use when a USB device or fixture attached to the ci HIL rig's Linux host is stuck, hung, not enumerating, or wedged after a failed flash or test, or when processes touching USB (testusb, JLinkExe, uhubctl, libusb tools) start hanging in D state. Linux-kernel-side only — a bus owned by a TinyUSB host is out of reach (reset the target / cycle its VBUS instead); the rig's probes and serial fixtures always remain in scope. --- -# USB Recovery on the HIL Rig +# USB Recovery on the HIL Rig (Linux kernel side) Run this skill's `scripts/usb_recover.sh` with `sudo` (abbreviated to `usb_recover.sh` in the examples below). It wraps the sysfs reset actions, a diff --git a/.claude/skills/usb-recover/scripts/usb_recover.sh b/.claude/skills/usb-kernel-recover/scripts/usb_recover.sh similarity index 100% rename from .claude/skills/usb-recover/scripts/usb_recover.sh rename to .claude/skills/usb-kernel-recover/scripts/usb_recover.sh diff --git a/.claude/skills/usb-sniffer/SKILL.md b/.claude/skills/usb-sniffer/SKILL.md index 7c2cd2644..a20aecf05 100644 --- a/.claude/skills/usb-sniffer/SKILL.md +++ b/.claude/skills/usb-sniffer/SKILL.md @@ -1,6 +1,6 @@ --- name: usb-sniffer -description: Use when you need wire-level USB evidence that host-side capture can't provide — a device that never enumerates (usbmon shows nothing or only Submits), suspected NAK storms/STALL/babble/bad handshakes, bus-reset or enumeration timing, split-transaction issues, or a usbmon-vs-device-log disagreement the wire must arbitrate. Captures LS/FS/HS packets (PIDs, tokens, handshakes, SE0/line states) with the ataradov usb-sniffer hardware into Wireshark pcapng. +description: Use when you need wire-level USB evidence that host-side capture can't provide — a device that never enumerates (usbmon shows nothing or only Submits), suspected NAK storms/STALL/babble/bad handshakes, bus-reset or enumeration timing, split-transaction issues, a usbmon-vs-device-log disagreement the wire must arbitrate, or any link where TinyUSB is the host (no Linux PC host to run usbmon on). Captures LS/FS/HS packets (PIDs, tokens, handshakes, SE0/line states) with the ataradov usb-sniffer hardware into Wireshark pcapng. --- # usb-sniffer — wire-level capture with the ataradov hardware analyzer @@ -9,14 +9,16 @@ Extends the debugging trio with the layer below URBs: | Skill | Answers | |---|---| -| `usbmon` | what the host software exchanged (URBs) | -| `usb-debug` | why the host acted (dmesg / dynamic debug) | -| `usb-target-debug` | what the device firmware did | +| `usbmon` | what a Linux PC host exchanged (URBs) | +| `usb-kernel-debug` | why the Linux kernel acted (dmesg / dynamic debug) | +| `target-debug` | what the TinyUSB target did (device or host role) | | **`usb-sniffer`** | **what actually crossed D+/D-** (PIDs, handshakes, resets, timing) | Reach for it when usbmon can't see (device never binds, pre-enumeration -failures) or can't be trusted (URB completed but did the wire really ACK?). -For everything visible in URBs, usbmon is cheaper — no hardware, no locks. +failures), can't be trusted (URB completed but did the wire really ACK?), or +doesn't exist — a link where TinyUSB is the host has no usbmon on either end +(an MCU host runs no kernel; a Linux gadget peer's UDC bypasses usbmon). +Where a Linux PC is the host, usbmon is cheaper — no hardware, no locks. ## Rig inventory — find the sniffer and what it taps diff --git a/.claude/skills/usbmon/SKILL.md b/.claude/skills/usbmon/SKILL.md index 85ec33248..a2e7c1ac1 100644 --- a/.claude/skills/usbmon/SKILL.md +++ b/.claude/skills/usbmon/SKILL.md @@ -1,11 +1,11 @@ --- name: usbmon -description: Use when capturing, analyzing, or debugging USB bus traffic for TinyUSB device development on Linux — enumeration failures, STALLed control transfers, missing/short bulk or interrupt transfers, isochronous/audio dropouts, or descriptor problems. Captures host-side URBs with usbmon + tshark into a Wireshark pcapng and decodes them. Use whenever you need to see what the host actually exchanged with a device on real hardware, even if the user just says "sniff USB", "capture the enumeration", or "why won't my device enumerate". +description: Use when capturing, analyzing, or debugging USB bus traffic on a link where a Linux PC is the host (TinyUSB in device role) — enumeration failures, STALLed control transfers, missing/short bulk or interrupt transfers, isochronous/audio dropouts, or descriptor problems. Captures host-side URBs with usbmon + tshark into a Wireshark pcapng and decodes them. Not applicable when TinyUSB is the host — no URBs traverse the PC (use usb-sniffer / target-debug). Use whenever you need to see what the Linux host actually exchanged with a device on real hardware, even if the user just says "sniff USB", "capture the enumeration", or "why won't my device enumerate". --- # usbmon — capture & debug USB traffic -`usbmon` records host-side **URBs** — control / bulk / interrupt / isochronous transfers, descriptors, class requests, STALLs, short packets — i.e. exactly what the host exchanged with a device. Use it to debug a TinyUSB device on real hardware. (It's host/URB-level, not wire-level; for SOF/ACK/electrical use a hardware analyzer.) +`usbmon` records host-side **URBs** — control / bulk / interrupt / isochronous transfers, descriptors, class requests, STALLs, short packets — i.e. exactly what the host exchanged with a device. Use it to debug a TinyUSB device on real hardware. (It's host/URB-level, not wire-level; for SOF/ACK/electrical use a hardware analyzer.) It exists only on the Linux host side of a link: when TinyUSB runs the *host* stack (peer = another TinyUSB board or a Linux gadget, e.g. a Raspberry Pi), neither end has usbmon — capture the wire (`usb-sniffer` skill) or instrument the target (`target-debug` skill). **Setup (assumed in place):** `usbmon` loaded and a udev rule `SUBSYSTEM=="usbmon", GROUP="wireshark", MODE="0640"` with your user in the `wireshark` group — so `tshark` captures with no `sudo`. Freshly added to the group? The running shell doesn't have it yet (group adds need a new login) — wrap captures in `sg wireshark -c 'tshark -i usbmon3 -s 128 -a duration:30 -w /tmp/cap.pcapng'`; reading a finished `.pcapng` (`tshark -r`) needs no group. `-s 128` (snaplen) keeps only URB headers/status, not payloads — use it for long/high-throughput captures. diff --git a/.claude/skills/usbtest/SKILL.md b/.claude/skills/usbtest/SKILL.md index 8146d94e4..850781d6e 100644 --- a/.claude/skills/usbtest/SKILL.md +++ b/.claude/skills/usbtest/SKILL.md @@ -1,6 +1,6 @@ --- name: usbtest -description: Use when running, debugging, or porting the Linux usbtest/testusb battery (examples/device/usbtest, cafe:4010) — device "did not bind", SET_CONFIGURATION fails, a case fails with errno 110/32/5/71, toggle-clear/halt/unlink/iso failures, iso packets dropped, or a new MCU/DCD needs the full 30/30 sign-off. +description: Use when running, debugging, or porting the Linux usbtest/testusb battery (examples/device/usbtest, cafe:4010) — device "did not bind", SET_CONFIGURATION fails, a case fails with errno 110/32/5/71, toggle-clear/halt/unlink/iso failures, iso packets dropped, or a new MCU/DCD needs the full 30/30 sign-off. Needs a Linux PC as the link's host driving TinyUSB in device role — it exercises the DCD, not the TinyUSB host stack. --- # usbtest — porting & debugging the Linux kernel USB battery diff --git a/.claude/workflows/hil-validate.js b/.claude/workflows/hil-validate.js index aa0556abc..50559135f 100644 --- a/.claude/workflows/hil-validate.js +++ b/.claude/workflows/hil-validate.js @@ -52,7 +52,7 @@ if (!args.force) { } const wedged = results.filter(r => r.wedged).map(r => r.board) -if (wedged.length) log(`WEDGED boards needing usb-recover: ${wedged.join(', ')}`) +if (wedged.length) log(`WEDGED boards needing usb-kernel-recover: ${wedged.join(', ')}`) // Workers cannot prompt the user — surface still-locked boards for the main // session to ask: force (re-invoke with force: true), wait, or accept. const locked = args.force ? [] : results.filter(r => !r.pass && r.detail.startsWith('board locked')).map(r => r.board) diff --git a/CLAUDE.md b/CLAUDE.md index 2acdc3a63..77dab4565 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -116,7 +116,7 @@ Cutting a release — version bump, regenerated files, the per-release changelog ## References -- MCU reference manuals, datasheets, schematics: before answering register/bitfield/pinout/errata/timing questions from memory or the web, use the `read-doc` skill (`.claude/skills/read-doc/SKILL.md`) to search and read them from `$HOME/Documents/calibre-library` (skill no-ops if the library is absent). +- MCU reference manuals, datasheets, schematics: before answering register/bitfield/pinout/errata/timing questions from memory or the web — or changing a specific dcd/hcd driver — use the `read-doc` skill (`.claude/skills/read-doc/SKILL.md`) to cross-check against docs in `$HOME/Documents/calibre-library`; tell the user if the needed document is missing (skill no-ops if the library is absent). - Supported MCUs/boards: `hw/bsp/` and `docs/reference/boards.rst`. - USB classes: `src/class/{cdc,hid,msc,audio,…}/` — each has `*_device.c` and `*_host.c`. - Key files: `src/tusb.h`, `src/tusb_config.h`, `tools/get_deps.py`, `tools/build.py`, `test/unit-test/project.yml`. diff --git a/test/hil/usbtest.py b/test/hil/usbtest.py index a2841f0b6..e17705a48 100755 --- a/test/hil/usbtest.py +++ b/test/hil/usbtest.py @@ -38,7 +38,7 @@ PID = '4010' GZ_REF = '0525 a4a0' # copy Gadget Zero's capability profile (ctrl_out+iso+intr) SYS_USB = Path('/sys/bus/usb/devices') DRIVER = Path('/sys/bus/usb/drivers/usbtest') -USB_RECOVER = Path(__file__).resolve().parents[2] / '.claude/skills/usb-recover/scripts/usb_recover.sh' +USB_RECOVER = Path(__file__).resolve().parents[2] / '.claude/skills/usb-kernel-recover/scripts/usb_recover.sh' PATTERN_PARAM = Path('/sys/module/usbtest/parameters/pattern') # Battery per tier, in run order: control sanity first, then simple bulk, @@ -391,7 +391,7 @@ def main(): if pci: print(f'aborting battery: kernel-side hang, device wedged mid-transfer.\n' f'auto-recovering: sudo {USB_RECOVER} pci-reset {pci} ' - f'(see .claude/skills/usb-recover)', file=sys.stderr) + f'(see .claude/skills/usb-kernel-recover)', file=sys.stderr) # FLR frees the D-state ioctl without the device lock; must run BEFORE # any unbind/remove_id, which would deadlock the bus otherwise if sudo([str(USB_RECOVER), 'pci-reset', pci]).returncode != 0: @@ -418,7 +418,7 @@ def main(): try: if unrecovered_hang: # testusb is still stuck in a usbfs ioctl holding the device lock; remove_id/unbind - # would join the convoy and deadlock the bus (see usb-recover skill) — leave it be + # would join the convoy and deadlock the bus (see usb-kernel-recover skill) — leave it be print('skipping cleanup after unrecovered hang: reboot required to release the bus', file=sys.stderr) elif not args.keep_binding: From 80a000e2f027cbeeda5a1a461b7c475ec22a3c91 Mon Sep 17 00:00:00 2001 From: hathach Date: Thu, 23 Jul 2026 11:38:26 +0700 Subject: [PATCH 02/11] fix(rp2040): make stdio_rtt_init static LOGGER=rtt builds of any rp2040 example fail with -Werror=missing-prototypes (stdio_rtt_init has no prototype and is only called from family.c). Found by building cdc_msc -DLOG=2 -DLOGGER=rtt for raspberry_pi_pico. --- hw/bsp/rp2040/family.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/bsp/rp2040/family.c b/hw/bsp/rp2040/family.c index 55feec159..15f179656 100644 --- a/hw/bsp/rp2040/family.c +++ b/hw/bsp/rp2040/family.c @@ -153,7 +153,7 @@ static stdio_driver_t stdio_rtt = { .in_chars = stdio_rtt_read }; -void stdio_rtt_init(void) { +static void stdio_rtt_init(void) { stdio_set_driver_enabled(&stdio_rtt, true); } #endif From b1becd8f5fcaf4a8a07aaa76fa68d3e9f7bb18a3 Mon Sep 17 00:00:00 2001 From: hathach Date: Fri, 24 Jul 2026 14:55:59 +0700 Subject: [PATCH 03/11] docs(target-debug): manuals, breakpoint/watchpoint arsenal, RTT via OpenOCD - Link J-Link UM08001, OpenOCD and GDB (Tenth Ed.) manuals - bp/wp depth with halt-per-hit cost model. Verified on stm32f407disco (J-Link) + raspberry_pi_pico (OpenOCD): FPB/DWT budget reads (M4 6 bp/4 wp, M0+ 4/2 exact), 'Hardware watchpoint' confirmation rule (software fallback single-steps = USB death), OpenOCD data-VALUE watchpoints, dprintf + breakpoint command lists exercised on hardware; JLinkGDBServer -singlerun lifecycle gotcha - RTT is not J-Link-only: OpenOCD rtt setup/start/server verified on pico (control block found at the nm address, LOG=2 boot banner captured over nc) --- .claude/skills/target-debug/SKILL.md | 78 +++++++++++++++++++- .idea/codeStyles/Project.xml | 10 +++ .idea/codeStyles/codeStyleConfig.xml | 5 ++ .idea/improve-debug-skill-agent.iml | 2 + .idea/inspectionProfiles/Project_Default.xml | 17 +++++ .idea/modules.xml | 8 ++ 6 files changed, 117 insertions(+), 3 deletions(-) create mode 100644 .idea/codeStyles/Project.xml create mode 100644 .idea/codeStyles/codeStyleConfig.xml create mode 100644 .idea/improve-debug-skill-agent.iml create mode 100644 .idea/inspectionProfiles/Project_Default.xml create mode 100644 .idea/modules.xml diff --git a/.claude/skills/target-debug/SKILL.md b/.claude/skills/target-debug/SKILL.md index c82237ea3..21dea38db 100644 --- a/.claude/skills/target-debug/SKILL.md +++ b/.claude/skills/target-debug/SKILL.md @@ -60,7 +60,9 @@ that IS a finding (timing-sensitive): move down in intrusiveness, not up. ## TU_LOG capture Build with `LOG=2` (`LOG=3` adds per-transfer noise and much more timing skew). -`LOGGER=rtt` routes it over the debug probe (J-Link only) — no UART wiring: +`LOGGER=rtt` routes it over the debug probe — no UART wiring. SEGGER's host +tools need a J-Link, but OpenOCD serves the same RTT buffer on ST-Link / +CMSIS-DAP / WCH-Link boards: ```bash # RTT: JLinkGDBServer from CLAUDE.md "GDB Debugging" + -RTTTelnetPort, then: @@ -69,6 +71,18 @@ timeout 20s JLinkRTTClient > /tmp/rtt.log # non-interactive capture stty -F /dev/ttyACM 115200 raw && timeout 20s cat /dev/ttyACM | tee /tmp/uart.log ``` +```bash +# OpenOCD RTT (any probe OpenOCD drives) — in telnet :4444 (or -c equivalents): +rtt setup 0x20000000 0x8000 "SEGGER RTT" # search range = RAM ORIGIN + LENGTH (from the .ld / map file) +rtt start # after firmware booted; rerun after each reflash +rtt server start 19021 0 +# then: timeout 20s nc localhost 19021 > /tmp/rtt.log +``` + +OpenOCD polls the buffer: bursty logs can drop lines a J-Link would keep — +prefer J-Link where both exist; the drain-model warning below applies +unchanged. + An RTT-built firmware that has since wedged still holds a log tail in RAM — but ONLY what fits the drain model: the default SEGGER mode (NO_BLOCK_SKIP) **drops** writes once the ring fills with no reader, so an undrained target @@ -85,8 +99,11 @@ reads don't halt the target. ## GDB — state autopsy and watchpoints Connect/load recipes per probe family (J-Link, OpenOCD for ST-Link / -CMSIS-DAP / WCH-Link) are in CLAUDE.md "GDB Debugging". Release builds keep -DWARF (`MinSizeRel`), so `p`/struct access works on HIL firmware. +CMSIS-DAP / WCH-Link) are in CLAUDE.md "GDB Debugging". For scripted/batch +sessions add `-singlerun` to JLinkGDBServer — the server exits with the +connection; back-to-back server relaunches race the probe handle and hang at +startup. Release builds keep DWARF (`MinSizeRel`), so `p`/struct access works +on HIL firmware. **Autopsy of a wedged board: attach and halt ONLY** — skip CLAUDE.md's `monitor reset halt` + `load` (those are for fresh starts; a reset destroys @@ -104,6 +121,50 @@ watch xfer_status[2][1].total_len # HW watchpoint (Cortex-M: ~4); dwc2 names break dcd_int_handler # works, but see warning below ``` +**Hardware budget — read it off the chip, not from memory** (verified: F407/M4 += 6 bp + 4 wp, rp2040/M0+ = 4 + 2; M7 typically 8/4): + +```gdb +p ((*(unsigned*)0xE0002000)>>4) & 0xF # FPB NUM_CODE = hw breakpoints (M7 adds bits[14:12]) +p (*(unsigned*)0xE0001000)>>28 # DWT_CTRL NUMCOMP = watchpoint comparators +``` + +- `hbreak`/`thbreak` force a hardware breakpoint (code in flash can't take a + software break unless the probe does flash breakpoints — J-Link does, + OpenOCD needs `bp 2 hw`); `tbreak` = one-shot. +- `watch -l ` watches the *address* the expression evaluates to once — + cheap and what you almost always want; `rwatch`/`awatch` trap reads/any + access (hardware-only — they error rather than fall back). OpenOCD (telnet + :4444) adds a data-VALUE match GDB cannot express: `wp 4 w + [mask]` — fires only when the written value matches (e.g. catch who writes + 0 into a busy flag, ignoring writes of 1). +- **Demand the word "Hardware" in the confirmation.** `watch` silently falls + back to a SOFTWARE watchpoint when no DWT comparator fits (expression too + wide/complex, budget exhausted): GDB then single-steps the whole program — + hundreds of times slower, certain USB death. `Watchpoint 2:` without + "Hardware" = delete it; narrowing the expression (`watch -l`, cast to a + 4-byte int) is the real fix. +- Conditional breaks/watches (`break dcd_edpt_xfer if ep_addr==0x81`) are + evaluated by GDB on the HOST with our stubs — neither JLinkGDBServer nor + OpenOCD supports target-side agent expressions on Cortex-M — so every hit + is a halt+resume (~ms) whether the condition matches or not: fine + post-wedge or on cold paths, wrong under live USB traffic. +- `commands ... end` auto-runs GDB commands at each hit (start with + `silent`, end with `continue` for hands-free evidence collection) — same + halt-per-hit cost. +- `dprintf ,"fmt",args` = printf without recompiling. Stay on the + default `dprintf-style gdb` (host prints): the `call` style runs the + target's own printf mid-halt and `agent` needs stub support — neither is + viable on these probes. Same cost model as conditional breaks; for + ISR-rate events use the RAM ring buffer instead. +- Stepping while the USB ISR fires between every step is chaos: OpenOCD + `cortex_m maskisr steponly` masks interrupts during single-steps only. + The bus keeps running either way — the host may still reset a device that + stops responding mid-step. +- While halted you can poke state to test a hypothesis (`set var + _usbd_dev.ep_status[2][1].busy = 0`) — but that invalidates the snapshot + as post-mortem evidence; dump first, poke after. + While halted the device answers **nothing**: host control transfers time out in ~5 s and the OS may reset/re-enumerate — after `continue`, the bus traffic shows recovery, not the original bug. Prefer one halt for a post-mortem dump @@ -176,6 +237,17 @@ number on the target gives a shared clock when you need finer alignment. When host and target evidence disagree, or the host sees nothing at all, add the wire itself: `usb-sniffer` skill (hardware tap, PID-level). +## Manuals + +- J-Link / J-Trace User Guide (UM08001): — flash breakpoints, RTT, SWO, monitor mode, Commander commands. +- OpenOCD User's Guide: — `rtt`, `bp`/`wp`, `cortex_m vector_catch` / `maskisr`, `itm`/`tpiu`. +- "Debugging with GDB" (the official manual; §5.1 covers break/watch/dprintf): + calibre library first (`read-doc` skill) — use the **Tenth Edition (GDB 18)** + copy, not the 2002 Ninth-Edition txt also present; fallback + `curl -sL -o /tmp/gdb.pdf https://sourceware.org/gdb/current/onlinedocs/gdb.pdf` + (the HTML mirror blocks fetchers; the PDF works). The installed + `arm-none-eabi-gdb`'s `help ` is authoritative for what this rig runs. + ## Warnings - **Halting/resetting via the probe does NOT disconnect the device**: a DWC2 diff --git a/.idea/codeStyles/Project.xml b/.idea/codeStyles/Project.xml new file mode 100644 index 000000000..35c56fc87 --- /dev/null +++ b/.idea/codeStyles/Project.xml @@ -0,0 +1,10 @@ + + + + + + + + \ No newline at end of file diff --git a/.idea/codeStyles/codeStyleConfig.xml b/.idea/codeStyles/codeStyleConfig.xml new file mode 100644 index 000000000..79ee123c2 --- /dev/null +++ b/.idea/codeStyles/codeStyleConfig.xml @@ -0,0 +1,5 @@ + + + + \ No newline at end of file diff --git a/.idea/improve-debug-skill-agent.iml b/.idea/improve-debug-skill-agent.iml new file mode 100644 index 000000000..4c9423543 --- /dev/null +++ b/.idea/improve-debug-skill-agent.iml @@ -0,0 +1,2 @@ + + \ No newline at end of file diff --git a/.idea/inspectionProfiles/Project_Default.xml b/.idea/inspectionProfiles/Project_Default.xml new file mode 100644 index 000000000..6b55c5c28 --- /dev/null +++ b/.idea/inspectionProfiles/Project_Default.xml @@ -0,0 +1,17 @@ + + + + \ No newline at end of file diff --git a/.idea/modules.xml b/.idea/modules.xml new file mode 100644 index 000000000..e97c64966 --- /dev/null +++ b/.idea/modules.xml @@ -0,0 +1,8 @@ + + + + + + + + \ No newline at end of file From 21bbcb5bbff18454979a1dc9aaa210e938a004b5 Mon Sep 17 00:00:00 2001 From: hathach Date: Fri, 24 Jul 2026 14:55:59 +0700 Subject: [PATCH 04/11] docs(target-debug): vector catch, SWO trace, verifybin, FreeRTOS threads; table integration MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Vector catch + Cortex-M fault autopsy, verified with a deliberate bad-load on stm32f407disco: CFSR=0x8200 (BFARVALID|PRECISERR), BFAR = exact bad address, stacked pc addr2lined to the faulting line; gotchas recorded (stale FPB comparators fire phantom SIGTRAPs — scrub first; arm DEMCR after reset; loads precise / stores imprecise; ARMv6-M has no CFSR/BFAR) - SWO exception trace + hw PC sampling gate PASSED on F407: 680 KB of packets in 3 s (0x17 PC samples in flash range, 0x0E SysTick enter/exit); JLinkSWOViewerCL decodes stimulus only — raw SWORead is the recipe; SWOStart needs an explicit speed headless - verifybin 'Verify successful.'; FreeRTOS -rtos plugin lists all 6 cdc_msc_freertos tasks after a run->stop cycle (plain attach = 0xDEAD placeholder); semihosting anti-note; monitor-mode pointer (untested) - Intrusiveness table gains the new rows; agent playbook bullet updated; retrieval gate 5/5 with a fresh reader; executed plan committed --- .claude/agents/target-debugger.md | 3 +- .claude/skills/target-debug/SKILL.md | 176 ++++-- ...26-07-23-target-debug-skill-enhancement.md | 569 ++++++++++++++++++ 3 files changed, 686 insertions(+), 62 deletions(-) create mode 100644 docs/superpowers/plans/2026-07-23-target-debug-skill-enhancement.md diff --git a/.claude/agents/target-debugger.md b/.claude/agents/target-debugger.md index 861600428..4d874e38c 100644 --- a/.claude/agents/target-debugger.md +++ b/.claude/agents/target-debugger.md @@ -13,7 +13,8 @@ truth; read the relevant SKILL.md BEFORE acting: - `.claude/skills/target-debug/SKILL.md` — your primary playbook: technique choice by intrusiveness, channel choice by link topology, capture recipes, - GDB autopsy, all rig warnings. + breakpoint/watchpoint budget and cost model, vector catch + fault autopsy, + SWO trace, GDB autopsy, all rig warnings. - `.claude/skills/hil/SKILL.md` — host/config selection, board lock protocol, `hil_test.py` invocation. - `.claude/skills/usbmon/SKILL.md` — Linux-host URB capture; exists only when a diff --git a/.claude/skills/target-debug/SKILL.md b/.claude/skills/target-debug/SKILL.md index 21dea38db..d0ceef2a9 100644 --- a/.claude/skills/target-debug/SKILL.md +++ b/.claude/skills/target-debug/SKILL.md @@ -18,11 +18,10 @@ Raspberry Pi). Pick capture channels by which end runs Linux, not by habit: | `usb-sniffer` | what crossed the wire (PIDs, handshakes, resets) | hardware tap cabled in — role-agnostic | For enumeration/transfer bugs the default posture is **dual-side capture** — -both ends simultaneously, not one-side-first-then-escalate: usbmon plus a -target channel when a Linux PC is the host. When TinyUSB is the host there is -no usbmon on either end — pair the target channel with the wire -(`usb-sniffer`) and, if the peer is a Linux gadget, `usb-kernel-debug` on the -peer. +both ends simultaneously: usbmon + a target +channel when a Linux PC is the host; TinyUSB-as-host has no usbmon on either +end — target channel + the wire (`usb-sniffer`), plus `usb-kernel-debug` on a +Linux gadget peer. ## Rig discipline — lock first, always @@ -52,9 +51,12 @@ that IS a finding (timing-sensitive): move down in intrusiveness, not up. | Technique | Intrusiveness | Reach for it when | |---|---|---| | PC-sampling | none — no halt, no code change | core wedged/spinning somewhere unknown (rusb2 FRDY) | +| SWO exception trace / hw PC-sample | none — needs SWO pin wired | ISR ordering/timing with zero code change | +| Vector catch | none until a fault fires | crash-shaped wedges — autopsy AT the faulting pc | | RAM ring-buffer | ~tens of cycles per event | ISR ordering/timing bugs (musb babble) | -| TU_LOG (RTT) | µs per line | logic bugs that survive logging | -| TU_LOG (UART) | ms per line — blocking write | same, when no J-Link on the board | +| TU_LOG (RTT) | µs per line | logic bugs that survive logging (J-Link or OpenOCD rtt) | +| TU_LOG (UART) | ms per line — blocking write | same, when no debug-probe RTT path | +| dprintf / conditional breakpoint | halt+resume per hit (~ms) | low-rate probes post-wedge; never ISR-rate events | | GDB halt / breakpoints | stops USB service entirely | post-mortem state autopsy once wedged | ## TU_LOG capture @@ -73,15 +75,14 @@ stty -F /dev/ttyACM 115200 raw && timeout 20s cat /dev/ttyACM | tee /tmp/u ```bash # OpenOCD RTT (any probe OpenOCD drives) — in telnet :4444 (or -c equivalents): -rtt setup 0x20000000 0x8000 "SEGGER RTT" # search range = RAM ORIGIN + LENGTH (from the .ld / map file) +rtt setup 0x20000000 0x8000 "SEGGER RTT" # RAM ORIGIN + LENGTH (from the .ld/map) rtt start # after firmware booted; rerun after each reflash rtt server start 19021 0 # then: timeout 20s nc localhost 19021 > /tmp/rtt.log ``` -OpenOCD polls the buffer: bursty logs can drop lines a J-Link would keep — -prefer J-Link where both exist; the drain-model warning below applies -unchanged. +OpenOCD polls — bursty logs can drop lines; prefer J-Link where both +exist. The drain-model warning below applies unchanged. An RTT-built firmware that has since wedged still holds a log tail in RAM — but ONLY what fits the drain model: the default SEGGER mode (NO_BLOCK_SKIP) @@ -99,10 +100,8 @@ reads don't halt the target. ## GDB — state autopsy and watchpoints Connect/load recipes per probe family (J-Link, OpenOCD for ST-Link / -CMSIS-DAP / WCH-Link) are in CLAUDE.md "GDB Debugging". For scripted/batch -sessions add `-singlerun` to JLinkGDBServer — the server exits with the -connection; back-to-back server relaunches race the probe handle and hang at -startup. Release builds keep DWARF (`MinSizeRel`), so `p`/struct access works +CMSIS-DAP / WCH-Link) are in CLAUDE.md "GDB Debugging". Scripted sessions: JLinkGDBServer `-singlerun` (exits with the +connection) — back-to-back relaunches race the probe handle and hang. Release builds keep DWARF (`MinSizeRel`), so `p`/struct access works on HIL firmware. **Autopsy of a wedged board: attach and halt ONLY** — skip CLAUDE.md's @@ -121,55 +120,87 @@ watch xfer_status[2][1].total_len # HW watchpoint (Cortex-M: ~4); dwc2 names break dcd_int_handler # works, but see warning below ``` -**Hardware budget — read it off the chip, not from memory** (verified: F407/M4 -= 6 bp + 4 wp, rp2040/M0+ = 4 + 2; M7 typically 8/4): +**Hardware budget — read it off the chip** (verified: F407/M4 = 6 bp + 4 wp, +rp2040/M0+ = 4 + 2; M7 typically 8/4): ```gdb p ((*(unsigned*)0xE0002000)>>4) & 0xF # FPB NUM_CODE = hw breakpoints (M7 adds bits[14:12]) p (*(unsigned*)0xE0001000)>>28 # DWT_CTRL NUMCOMP = watchpoint comparators ``` -- `hbreak`/`thbreak` force a hardware breakpoint (code in flash can't take a - software break unless the probe does flash breakpoints — J-Link does, - OpenOCD needs `bp 2 hw`); `tbreak` = one-shot. -- `watch -l ` watches the *address* the expression evaluates to once — - cheap and what you almost always want; `rwatch`/`awatch` trap reads/any - access (hardware-only — they error rather than fall back). OpenOCD (telnet - :4444) adds a data-VALUE match GDB cannot express: `wp 4 w - [mask]` — fires only when the written value matches (e.g. catch who writes - 0 into a busy flag, ignoring writes of 1). +- `hbreak`/`thbreak` force a hardware breakpoint (software breaks in flash + need flash-breakpoint support — J-Link has it; OpenOCD: `bp 2 hw`); + `tbreak` = one-shot. +- `watch -l ` watches the address expr evaluates to once — almost + always what you want; `rwatch`/`awatch` trap reads/any access (hardware- + only — they error, never fall back). OpenOCD adds a data-VALUE match GDB + can't express: `wp 4 w [mask]` — catch who writes 0 into a + busy flag, ignoring writes of 1. - **Demand the word "Hardware" in the confirmation.** `watch` silently falls - back to a SOFTWARE watchpoint when no DWT comparator fits (expression too - wide/complex, budget exhausted): GDB then single-steps the whole program — - hundreds of times slower, certain USB death. `Watchpoint 2:` without - "Hardware" = delete it; narrowing the expression (`watch -l`, cast to a - 4-byte int) is the real fix. -- Conditional breaks/watches (`break dcd_edpt_xfer if ep_addr==0x81`) are - evaluated by GDB on the HOST with our stubs — neither JLinkGDBServer nor - OpenOCD supports target-side agent expressions on Cortex-M — so every hit - is a halt+resume (~ms) whether the condition matches or not: fine - post-wedge or on cold paths, wrong under live USB traffic. -- `commands ... end` auto-runs GDB commands at each hit (start with - `silent`, end with `continue` for hands-free evidence collection) — same - halt-per-hit cost. -- `dprintf ,"fmt",args` = printf without recompiling. Stay on the - default `dprintf-style gdb` (host prints): the `call` style runs the - target's own printf mid-halt and `agent` needs stub support — neither is - viable on these probes. Same cost model as conditional breaks; for - ISR-rate events use the RAM ring buffer instead. -- Stepping while the USB ISR fires between every step is chaos: OpenOCD - `cortex_m maskisr steponly` masks interrupts during single-steps only. - The bus keeps running either way — the host may still reset a device that - stops responding mid-step. -- While halted you can poke state to test a hypothesis (`set var - _usbd_dev.ep_status[2][1].busy = 0`) — but that invalidates the snapshot - as post-mortem evidence; dump first, poke after. + back to a software watchpoint when no DWT comparator fits — GDB then + single-steps the whole program, hundreds of times slower: certain USB + death. Plain `Watchpoint 2:` = delete it and narrow the expression + (`watch -l`, cast to a 4-byte int). +- Conditional breaks (`break ... if ep_addr==0x81`) and `dprintf + ,"fmt",args` (printf without recompiling; keep `dprintf-style gdb`) + are host-evaluated — no Cortex-M agent expressions in our stubs: every hit + halts+resumes (~ms) even when the condition is false. Post-wedge/cold + paths only; ISR-rate events belong in the RAM ring. +- `commands ... end` (start `silent`, end `continue`) auto-collects + evidence per hit — same halt-per-hit cost. +- Stepping with the USB ISR firing between steps is chaos: OpenOCD + `cortex_m maskisr steponly`. The bus runs either way — the host may still + reset a halted-looking device. +- Poking state while halted (`set var _usbd_dev.ep_status[2][1].busy = 0`) + tests a hypothesis but invalidates the post-mortem — dump first, poke after. +- FreeRTOS examples: `-rtos GDBServer/RTOSPlugin_FreeRTOS` (OpenOCD: `-rtos + FreeRTOS`) → `info threads` lists every task with state/prio/frame + (verified: 6 tasks). It populates only after a + run→stop cycle — plain attach shows one 0xDEAD placeholder. Semihosting is + never the answer (traps + halts per call — RTT instead). **Monitor-mode + debugging** (J-Link, M3+) keeps chosen IRQs serviced at a breakpoint — + needs SEGGER's JLINK_MONITOR files + `SetMonModeDebug=1`; not set up here: + (untested). While halted the device answers **nothing**: host control transfers time out in ~5 s and the OS may reset/re-enumerate — after `continue`, the bus traffic shows recovery, not the original bug. Prefer one halt for a post-mortem dump over stepping through live USB traffic. +## Vector catch + fault autopsy — catch the crash, not the wedge + +A wedge that is really a fault (HardFault loop, lockup) autopsies best AT +the faulting instruction. Two hardware-proven gotchas: **FPB/DWT comparators +survive reflash and dead sessions** — a stale one fires as a phantom SIGTRAP +at an unrelated line of NEW firmware — and J-Link's reset strategy manages +vector-catch bits: scrub first, arm AFTER reset: + +```gdb +# scrub: FP_COMP0..5 = 0xE0002008..201C, DWT_FUNCTIONn = 0xE0001028 + n*0x10 +set *(unsigned*)0xE0002008 = 0 +# ... (repeat per comparator; count from the budget reads above) +# arm (after monitor reset; tool-agnostic — works via JLinkExe w4 too): +set *(unsigned*)0xE000EDFC |= (1<<10)|(1<<9)|(1<<8)|(1<<7)|(1<<6)|(1<<5)|(1<<4) +# = VC_HARDERR|INTERR|BUSERR|STATERR|CHKERR|NOCPERR|MMERR; bit0 VC_CORERESET halts at reset +``` + +OpenOCD native: `cortex_m vector_catch hard_err bus_err state_err chk_err mm_err`. +It halts at exception ENTRY (pc = handler, LR = EXC_RETURN 0xFFFFFFFx); decode: + +```gdb +p/x *(unsigned*)0xE000ED28 # CFSR — low byte MemManage, byte1 BusFault, top half UsageFault +p/x *(unsigned*)0xE000ED2C # HFSR — bit30 FORCED = an escalated lower-priority fault +p/x *(unsigned*)0xE000ED38 # BFAR — faulting address (valid if CFSR bit15 BFARVALID) +x/8wx $msp # stacked frame: r0 r1 r2 r3 r12 lr pc xpsr — pc = culprit +``` + +`addr2line -e ` names the line (verified: CFSR 0x8200, +BFAR = the bad address, stacked pc = the faulting ldr). Loads fault +precisely; stores usually IMPRECISERR (BFAR invalid, pc late). ARMv6-M has no +CFSR/BFAR, only VC_HARDERR|VC_CORERESET — stacked frame alone. Still a halt +(host URB timeouts apply); clear DEMCR (`&= ~0x7F0`) before handing back; +RISC-V: breakpoint the trap handler; mcause/mepc/mtval are the CFSR/BFAR analogs. + ## RAM ring-buffer trace The zero-print instrument (cracked the musb babble): a small event ring in the @@ -218,11 +249,31 @@ arm-none-eabi-addr2line -e -f -a 0x ... # PCs → funct OpenOCD variant: repeat `mdw 0xE000101C` over telnet :4444. The histogram's top entries are the spin site; a flat histogram = core is servicing normally. +### SWO — hardware-timed trace on one pin (J-Link; verified on F407) + +If SWO (TRACESWO) is wired, DWT emits packets with ZERO code change: +**exception trace** (DWT_CTRL bit16 — every IRQ enter/exit, timestamped) and +**hardware PC sampling** (bit12), better histograms than DWT_PCSR polling. +SWOViewer tools decode only ITM *stimulus* (TinyUSB emits none) — capture +raw: + +```bash +# JLinkExe -CommandFile: +w4 E0001000, 0x00011401 # EXCTRCENA|PCSAMPLENA|SYNCTAP|CYCCNTENA +SWOStart 4000000 # explicit speed — autodetect fails headless +Sleep 3000 +SWORead # hex: 0x17+4B LE = PC sample, 0x0E+2B = IRQ enter/exit +``` + +Verified: 680 KB in 3 s (flash-range PC samples + SysTick enter/exit). +SWORead stuck at 0 = SWO pin not wired (many boards route only SWDIO/SWCLK). +Restore DWT_CTRL when done. + ## Dual-side capture — the default for enumeration/transfer bugs -Start both channels, then trigger the failing test (Linux-PC-host link shown; -TinyUSB-as-host: swap the usbmon line for a `usb-sniffer` capture, plus -`usb-kernel-debug` on the peer if it is a Linux gadget): +Start both channels, then trigger the failing test (Linux-PC-host shown; +TinyUSB-as-host: swap usbmon for `usb-sniffer`, + `usb-kernel-debug` on a +Linux gadget peer): ```bash .claude/skills/usbmon/scripts/usbcap.sh cafe: 30 /tmp/host.pcapng & # host URBs (usbmon skill) @@ -239,14 +290,13 @@ the wire itself: `usb-sniffer` skill (hardware tap, PID-level). ## Manuals -- J-Link / J-Trace User Guide (UM08001): — flash breakpoints, RTT, SWO, monitor mode, Commander commands. -- OpenOCD User's Guide: — `rtt`, `bp`/`wp`, `cortex_m vector_catch` / `maskisr`, `itm`/`tpiu`. -- "Debugging with GDB" (the official manual; §5.1 covers break/watch/dprintf): - calibre library first (`read-doc` skill) — use the **Tenth Edition (GDB 18)** - copy, not the 2002 Ninth-Edition txt also present; fallback +- J-Link (UM08001): — flash breakpoints, RTT, SWO, monitor mode, Commander. +- OpenOCD: — `rtt`, `bp`/`wp`, `cortex_m vector_catch`/`maskisr`, `itm`/`tpiu`. +- "Debugging with GDB" (§5.1 = break/watch/dprintf): Tenth Edition (GDB 18) + via calibre/`read-doc` — NOT the 2002 Ninth-Edition txt also there — or `curl -sL -o /tmp/gdb.pdf https://sourceware.org/gdb/current/onlinedocs/gdb.pdf` - (the HTML mirror blocks fetchers; the PDF works). The installed - `arm-none-eabi-gdb`'s `help ` is authoritative for what this rig runs. + (the HTML mirror blocks fetchers). Installed `arm-none-eabi-gdb` + `help ` is authoritative here. ## Warnings @@ -268,6 +318,10 @@ the wire itself: `usb-sniffer` skill (hardware tap, PID-level). freshly-reset chip (e.g. NVIC ISER = 0) reads like a smoking gun. Read DHCSR (0xE000EDF0: bit 17 S_HALT, bit 25 S_RESET_ST) with every snapshot, and cross-check against something the device demonstrably still does. +- **"Flash OK" can lie** (silent no-op — old firmware keeps running). When + behavior contradicts the flashed code: `objcopy -O binary fw.elf + /tmp/fw.bin`, then `verifybin /tmp/fw.bin,` (J-Link, verified) + or `verify_image` (OpenOCD); on mismatch reflash before debugging further. - **A marginal link can fake a deterministic firmware bug** — down to failing the same test at the same iteration twice. "USB disconnect" in dmesg on a freshly re-cabled port (high devnum = churn) means the plug, not the code: diff --git a/docs/superpowers/plans/2026-07-23-target-debug-skill-enhancement.md b/docs/superpowers/plans/2026-07-23-target-debug-skill-enhancement.md new file mode 100644 index 000000000..edace21c7 --- /dev/null +++ b/docs/superpowers/plans/2026-07-23-target-debug-skill-enhancement.md @@ -0,0 +1,569 @@ +# target-debug Skill & target-debugger Agent Enhancement Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Extend `.claude/skills/target-debug/SKILL.md` (and its agent) with the full debugger facility arsenal from the J-Link, OpenOCD, and GDB manuals — breakpoint/watchpoint depth, OpenOCD RTT, vector catch + fault autopsy, SWO/ITM trace, flash verification — each recipe hardware-verified on the ci rig before it lands unmarked. + +**Architecture:** The skill's organizing spine is its intrusiveness table ("pick the least intrusive technique that can answer the question"); every new facility slots into that model with an honest cost row. Recipes keep the existing dense, copy-paste style. The skill's value is that its recipes are *proven on this rig* — so each task pairs drafting with a bounded hardware verification, and anything unverifiable lands tagged `(untested)` or is dropped. + +**Tech Stack:** arm-none-eabi-gdb 15.2, OpenOCD 0.12.0+dev, SEGGER J-Link V7.94b (`JLinkExe`, `JLinkGDBServer`, `JLinkSWOViewerCLExe`), ci rig boards from `test/hil/tinyusb.json` (10 jlink / 6 openocd / 1 stlink probes). + +**Reference manual:** "Debugging with GDB", **Tenth Edition** (for GDB 18.0.50) — prefer the calibre-library copy via the `read-doc` skill, but **verify the edition on the title page first**: the library also holds an outdated Ninth Edition (2002, GDB 5.1.1, txt) that predates `dprintf`/`watch -l` — do not use it. Fallback fetch: `curl -sL -o /tmp/gdb.pdf https://sourceware.org/gdb/current/onlinedocs/gdb.pdf` (HTML pages block fetchers; the PDF does not). Sections used by this plan: §5.1.2 Setting Watchpoints, §5.1.6 Break Conditions, §5.1.7 Breakpoint Command Lists, §5.1.8 Dynamic Printf (PDF page = book page + 18). NOTE: the manual documents GDB 18; the rig runs 15.2 — the installed `arm-none-eabi-gdb`'s `help ` is authoritative for feature availability. + +## Global Constraints + +- Worktree: `/home/hathach/code/tinyusb/.claude/worktrees/improve-debug-skill-agent`, branch `claude/improve-debug-skill-agent`. All paths below are relative to it. +- J-Link User Guide link must be exactly `https://kb.segger.com/UM08001_J-Link_/_J-Trace_User_Guide` (user-specified, verified live 2026-07-23). +- **Hardware-verify before landing**: a recipe is committed unmarked only with captured evidence from a rig board; otherwise tag it `(untested)` inline or drop it. Record evidence (command + output snippet) in the task's commit message body. +- Rig discipline (from `hil` + `target-debug` skills): `python3 test/hil/board_lock.py hold --reason "skill-enhance verify: "` before touching hardware, `release` after; reflash pristine firmware before release; NEVER stop the actions-runner; one J-Link client per probe at a time; we are ON host `ci` (config `test/hil/tinyusb.json`). +- Hardware tasks are strictly serial (one board session at a time). Bash timeouts ≥ 10 min for flash+debug cycles. +- Style: match the skill's existing voice — dense, recipe-first, caveats inline. Skill word budget after all tasks: ≤ 2 700 words (`wc -w`, currently 1 763). +- Run `pre-commit run --files ` before every commit. No Co-Authored-By trailers. +- Board selection is runtime data (boards come/go, locks): resolve with the exact python snippet in Task 2 Step 2 and reuse `$JB` (jlink board) / `$OB` (openocd board) thereafter. + +--- + +### Task 1: Manuals reference block + +**Files:** +- Modify: `.claude/skills/target-debug/SKILL.md` (insert new `## Manuals` section immediately before `## Warnings`) + +**Interfaces:** +- Produces: `## Manuals` section that later tasks' text may reference as "see Manuals". + +- [x] **Step 1: Insert the Manuals section** + +In `.claude/skills/target-debug/SKILL.md`, find the line `## Warnings` and insert immediately before it: + +```markdown +## Manuals + +- J-Link / J-Trace User Guide (UM08001): — flash breakpoints, RTT, SWO, monitor mode, Commander commands. +- OpenOCD User's Guide: — `rtt`, `bp`/`wp`, `cortex_m vector_catch` / `maskisr`, `itm`/`tpiu`. +- "Debugging with GDB" (the official manual; §5.1 covers break/watch/dprintf): + calibre library first (`read-doc` skill) — use the **Tenth Edition (GDB 18)** + copy, not the 2002 Ninth-Edition txt also present; fallback + `curl -sL -o /tmp/gdb.pdf https://sourceware.org/gdb/current/onlinedocs/gdb.pdf` + (the HTML mirror blocks fetchers; the PDF works). The installed + `arm-none-eabi-gdb`'s `help ` is authoritative for what this rig runs. + +``` + +- [x] **Step 2: Verify formatting and word count** + +Run: `cd /home/hathach/code/tinyusb/.claude/worktrees/improve-debug-skill-agent && grep -A5 '^## Manuals' .claude/skills/target-debug/SKILL.md && wc -w .claude/skills/target-debug/SKILL.md` +Expected: section present before `## Warnings`; word count ≤ 1 830. + +- [x] **Step 3: Commit** + +```bash +cd /home/hathach/code/tinyusb/.claude/worktrees/improve-debug-skill-agent +pre-commit run --files .claude/skills/target-debug/SKILL.md +git add .claude/skills/target-debug/SKILL.md +git commit -m "docs(target-debug): link J-Link UM08001, OpenOCD and GDB manuals" +``` + +--- + +### Task 2: Breakpoint & watchpoint arsenal (GDB + OpenOCD) + +**Files:** +- Modify: `.claude/skills/target-debug/SKILL.md` — extend the `## GDB — state autopsy and watchpoints` section +- Read-only reference: `test/hil/tinyusb.json` (board resolution) + +**Interfaces:** +- Consumes: nothing from other tasks. +- Produces: board env vars `$JB`, `$OB` resolution snippet (reused by Tasks 3-6); the "halt-per-hit cost model" wording that Task 7's table row cites. + +- [x] **Step 1: Draft the section extension** + +In `.claude/skills/target-debug/SKILL.md`, the GDB section currently ends with the paragraph beginning `While halted the device answers **nothing**`. Insert immediately BEFORE that paragraph: + +```markdown +**Hardware budget — read it off the chip, not from memory** (counts differ +per core: M0+ typically 4 bp/2 wp, M3/M4 6/4, M7 8/4): + +```gdb +p ((*(unsigned*)0xE0002000)>>4) & 0xF # FPB NUM_CODE = hw breakpoints (M7 adds bits[14:12]) +p (*(unsigned*)0xE0001000)>>28 # DWT_CTRL NUMCOMP = watchpoint comparators +``` + +- `hbreak`/`thbreak` force a hardware breakpoint (code in flash can't take a + software break unless the probe does flash breakpoints — J-Link does, + OpenOCD needs `bp 2 hw`); `tbreak` = one-shot. +- `watch -l ` watches the *address* the expression evaluates to once — + cheap and what you almost always want; `rwatch`/`awatch` trap reads/any + access (hardware-only — they error rather than fall back). OpenOCD (telnet + :4444) adds a data-VALUE match GDB cannot express: `wp 4 w + [mask]` — fires only when the written value matches (e.g. catch who writes + 0 into a busy flag, ignoring writes of 1). +- **Demand the word "Hardware" in the confirmation.** `watch` silently falls + back to a SOFTWARE watchpoint when no DWT comparator fits (expression too + wide/complex, budget exhausted): GDB then single-steps the whole program — + hundreds of times slower, certain USB death. `Watchpoint 2:` without + "Hardware" = delete it; `set can-use-hw-watchpoints 1` is the default but + narrowing the expression (`watch -l`, cast to a 4-byte int) is the real fix. +- Conditional breaks/watches (`break dcd_edpt_xfer if ep_addr==0x81`) are + evaluated by GDB on the HOST with our stubs — neither JLinkGDBServer nor + OpenOCD supports target-side agent expressions on Cortex-M — so every hit + is a halt+resume (~ms) whether the condition matches or not: fine + post-wedge or on cold paths, wrong under live USB traffic. +- `commands ... end` auto-runs GDB commands at each hit (start with + `silent`, end with `continue` for hands-free evidence collection) — same + halt-per-hit cost. +- `dprintf ,"fmt",args` = printf without recompiling. Stay on the + default `dprintf-style gdb` (host prints): the `call` style runs the + target's own printf mid-halt and `agent` needs stub support — neither is + viable on these probes. Same cost model as conditional breaks; for + ISR-rate events use the RAM ring buffer instead. +- Stepping while the USB ISR fires between every step is chaos: OpenOCD + `cortex_m maskisr steponly` masks interrupts during single-steps only. + The bus keeps running either way — the host may still reset a device that + stops responding mid-step. +- While halted you can poke state to test a hypothesis (`set var + _usbd_dev.ep_status[2][1].busy = 0`) — but that invalidates the snapshot + as post-mortem evidence; dump first, poke after. +``` + +- [x] **Step 2: Resolve verification boards (runtime data)** + +```bash +cd /home/hathach/code/tinyusb +python3 - <<'EOF' +import json +cfg = json.load(open('test/hil/tinyusb.json')) +jl = [b['name'] for b in cfg['boards'] if b['flasher']['name']=='jlink'] +oo = [b['name'] for b in cfg['boards'] if b['flasher']['name']=='openocd'] +print('JLINK candidates:', jl) +print('OPENOCD candidates:', oo) +EOF +``` +Pick the first candidate of each that `python3 test/hil/board_lock.py status` shows unlocked; export as `JB=` `OB=`. Look up `flasher.uid` for each in `test/hil/tinyusb.json` (`JB_UID`, `OB_UID`) and `JLINK_DEVICE`/`OPENOCD_OPTION` from `hw/bsp/*/boards/$JB/board.cmake` (family via `ls -d hw/bsp/*/boards/$JB`). + +- [x] **Step 3: Hardware-verify the budget reads on both probe families** + +```bash +python3 test/hil/board_lock.py hold $JB --reason "skill-enhance verify: bp/wp budget" +printf 'mem32 E0002000, 1\nmem32 E0001000, 1\nqc\n' | \ + JLinkExe -device $JLINK_DEVICE -SelectEmuBySN $JB_UID -if swd -speed 4000 -autoconnect 1 -nogui 1 +python3 test/hil/board_lock.py release $JB +``` +Expected: two register values; decode NUM_CODE and NUMCOMP by hand and check they are plausible (2-8 range). Repeat for `$OB` via `openocd $OPENOCD_OPTION -c init -c 'mdw 0xE0002000' -c 'mdw 0xE0001000' -c shutdown` under its own lock. +If a register reads 0 on one board, note which core and adjust the skill text's example counts if contradicted. + +- [x] **Step 4: Hardware-verify dprintf + commands round-trip on $JB** + +With the board lock held and an already-flashed example (any; do not reflash), a JLinkGDBServer on :2331 (per CLAUDE.md GDB Debugging), run bounded — `commands` blocks cannot be passed via `-ex`, so use a command file: + +```bash +cat > /tmp/bpcmd.gdb <<'EOF' +target remote :2331 +set var $count=0 +watch -l *(unsigned*)&_usbd_dev +delete +dprintf tud_task_ext,"tick\n" +break tud_task_ext +commands 3 +silent +set var $count=$count+1 +continue +end +continue& +shell sleep 3 +interrupt +print $count +EOF +timeout 120 arm-none-eabi-gdb -batch -x /tmp/bpcmd.gdb \ + $(find examples/cmake-build-$JB -name 'cdc_msc.elf' | head -1) +``` +Expected: the `watch` line answers `Hardware watchpoint 1:` (the word +"Hardware" present — this is the skill's software-fallback check, then +deleted), "tick" lines printed, and `$count > 0`. (`tud_task_ext` is the real +symbol — `tud_task` is an inline wrapper; the breakpoint is number 3 after +the watchpoint and dprintf.) Kill the GDB server, reflash pristine +(`ninja`-flash target or `hil_test.py` flash path), release the lock. + +- [x] **Step 5: Apply the Step-1 text, run pre-commit, commit** + +```bash +cd /home/hathach/code/tinyusb/.claude/worktrees/improve-debug-skill-agent +pre-commit run --files .claude/skills/target-debug/SKILL.md +git add .claude/skills/target-debug/SKILL.md +git commit -m "docs(target-debug): breakpoint/watchpoint arsenal with halt-per-hit cost model + +Verified on (J-Link) + (OpenOCD): FPB/DWT budget reads, dprintf, +breakpoint command lists. " +``` + +--- + +### Task 3: OpenOCD RTT — RTT is not J-Link-only + +**Files:** +- Modify: `.claude/skills/target-debug/SKILL.md` — `## TU_LOG capture` section + +**Interfaces:** +- Consumes: `$OB`, `$OB_UID`, `$OPENOCD_OPTION` from Task 2 Step 2. +- Produces: the corrected claim "RTT works on any OpenOCD-driven probe" that Task 7's agent text repeats. + +- [x] **Step 1: Replace the J-Link-only claim** + +In the `## TU_LOG capture` section, replace: + +```markdown +Build with `LOG=2` (`LOG=3` adds per-transfer noise and much more timing skew). +`LOGGER=rtt` routes it over the debug probe (J-Link only) — no UART wiring: +``` + +with: + +```markdown +Build with `LOG=2` (`LOG=3` adds per-transfer noise and much more timing skew). +`LOGGER=rtt` routes it over the debug probe — no UART wiring. SEGGER's host +tools need a J-Link, but OpenOCD serves the same RTT buffer on ST-Link / +CMSIS-DAP / WCH-Link boards: +``` + +- [x] **Step 2: Add the OpenOCD RTT recipe** + +Immediately after the existing J-Link/UART capture code block (ends with `... | tee /tmp/uart.log`), add: + +```markdown +```bash +# OpenOCD RTT (any probe OpenOCD drives) — in telnet :4444 (or -c equivalents): +rtt setup 0x20000000 0x8000 "SEGGER RTT" # search range = RAM ORIGIN + LENGTH (from the .ld / map file) +rtt start # after firmware booted; rerun after each reflash +rtt server start 19021 0 +# then: timeout 20s nc localhost 19021 > /tmp/rtt.log +``` + +OpenOCD polls the buffer (default 10 ms): bursty logs can drop lines a J-Link +would keep — prefer J-Link where both exist; the drain-model warning below +applies unchanged. +``` + +- [x] **Step 3: Hardware-verify on $OB** + +```bash +python3 test/hil/board_lock.py hold $OB --reason "skill-enhance verify: openocd rtt" +cd examples/device/cdc_msc && cmake -B build-rtt -DBOARD=$OB -G Ninja -DCMAKE_BUILD_TYPE=MinSizeRel -DLOG=2 -DLOGGER=rtt && cmake --build build-rtt +# flash it (ninja -C build-rtt cdc_msc-openocd), then: +openocd $OPENOCD_OPTION & # gdb :3333, telnet :4444 +{ echo 'rtt setup 0x20000000 0x8000 "SEGGER RTT"'; echo 'rtt start'; echo 'rtt server start 19021 0'; sleep 1; } | nc -q1 localhost 4444 +timeout 10s nc localhost 19021 > /tmp/ob_rtt.log; head /tmp/ob_rtt.log +``` +Expected: TinyUSB boot banner / log lines in `/tmp/ob_rtt.log`. Adjust the search range from the board's linker script if the control block isn't found ("rtt: No control block found") and mirror any correction into the Step-2 text. Kill openocd, reflash pristine cdc_msc (no LOG), release lock, delete `build-rtt`. + +- [x] **Step 4: Commit** + +```bash +pre-commit run --files .claude/skills/target-debug/SKILL.md +git add .claude/skills/target-debug/SKILL.md +git commit -m "docs(target-debug): RTT via OpenOCD on non-J-Link probes + +Verified on : rtt setup/start/server + nc capture of boot log. +" +``` + +--- + +### Task 4: Vector catch + fault autopsy + +**Files:** +- Modify: `.claude/skills/target-debug/SKILL.md` — new section after `## GDB — state autopsy and watchpoints` + +**Interfaces:** +- Consumes: `$JB` from Task 2. (Corrected during execution: $OB/rp2040 is ARMv6-M — no CFSR/BFAR and only VC_HARDERR, so the full autopsy verify needs the ARMv7-M $JB; the payload is a bad LOAD because stores fault imprecisely with BFAR invalid.) +- Produces: section title `## Vector catch + fault autopsy` cited by Task 7's table row. + +- [x] **Step 1: Insert the new section** + +After the GDB section (i.e. before `## RAM ring-buffer trace`), insert: + +```markdown +## Vector catch + fault autopsy — catch the crash, not the wedge + +A "wedge" that is really a fault (HardFault loop, lockup) autopsies best AT +the faulting instruction, not minutes later. Arm before reproducing: + +```gdb +# tool-agnostic (any probe, incl. J-Link): DEMCR trap bits — halt on fault +set *(unsigned*)0xE000EDFC |= (1<<10)|(1<<9)|(1<<8)|(1<<7)|(1<<6)|(1<<5)|(1<<4) +# = VC_HARDERR|INTERR|BUSERR|STATERR|CHKERR|NOCPERR|MMERR; bit0 VC_CORERESET halts at reset +``` + +OpenOCD native form: `cortex_m vector_catch hard_err bus_err state_err chk_err mm_err`. +When it fires the core halts at the fault; decode: + +```gdb +p/x *(unsigned*)0xE000ED28 # CFSR — low byte MemManage, byte1 BusFault, top half UsageFault +p/x *(unsigned*)0xE000ED2C # HFSR — bit30 FORCED = an escalated lower-priority fault +p/x *(unsigned*)0xE000ED38 # BFAR — faulting address (valid if CFSR bit15 BFARVALID) +x/8wx $msp # stacked frame: r0 r1 r2 r3 r12 lr pc xpsr — pc = culprit +``` + +`arm-none-eabi-addr2line -e ` names the line. Caveats: a +vector-catch halt is still a halt (host-side URB timeouts apply); the bits +persist until power-cycle — clear them (`... &= ~0x7F1`) before handing the +board back; RISC-V ports have no DEMCR — use a breakpoint on the trap handler. +``` + +- [x] **Step 2: Hardware-verify with a deliberate fault on $JB (ARMv7-M)** + +Create the fault build (NOT committed): + +```bash +python3 test/hil/board_lock.py hold $OB --reason "skill-enhance verify: vector catch" +cd examples/device/cdc_msc +# temporary patch — revert after: fault 5 s after boot +python3 - <<'EOF' +import pathlib +p = pathlib.Path('src/main.c'); s = p.read_text() +import re +s = re.sub(r'\\nint main\\(void\\)', + '\\nstatic void _fault_after_5s(void){ static uint32_t t0=0; if(!t0) t0=tusb_time_millis_api();' + ' if(tusb_time_millis_api()-t0>5000) (void)*(volatile uint32_t*)0xCF000000u; }\\n\\nint main(void)', s, count=1) # board_millis is gone; helper must sit after the includes +s = s.replace('led_blinking_task();', 'led_blinking_task(); _fault_after_5s();', 1) +p.write_text(s) +EOF +grep -n '_fault_after_5s' src/main.c # expect 3 hits: definition + call + (none in decl block) +cmake -B build-fault -DBOARD=$OB -G Ninja -DCMAKE_BUILD_TYPE=MinSizeRel && cmake --build build-fault +``` +(If `app_led_task`/`board_millis` anchors differ in the current `main.c`, place the same 3-line helper on whatever per-loop task function exists — the fault line `*(volatile uint32_t*)0xCF000000u = 0;` is the payload.) +Flash `build-fault`, then: + +```bash +openocd $OPENOCD_OPTION -c init -c 'cortex_m vector_catch hard_err bus_err' & +timeout 60 arm-none-eabi-gdb -batch -ex 'target remote :3333' -ex 'monitor reset run' \ + -ex 'shell sleep 8' -ex 'interrupt' \ + -ex 'p/x *(unsigned*)0xE000ED28' -ex 'p/x *(unsigned*)0xE000ED38' -ex 'x/8wx $msp' \ + build-fault/cdc_msc.elf +``` +Expected: halted in the fault path, CFSR BusFault bits set, **BFAR = 0xCF000000**, stacked pc addr2lines to `_fault_after_5s`. If the write is silently ignored on this core (some buses RAZ/WI), switch payload to a NULL-function call `((void(*)(void))0x1)();` and note UsageFault/INVSTATE instead. + +- [x] **Step 3: Clean up hardware state** + +`git checkout -- src/main.c`, delete `build-fault/`, clear DEMCR bits (`set *(unsigned*)0xE000EDFC &= ~0x7F1` via a final gdb attach or power-cycle note), reflash pristine cdc_msc, `board_lock.py release $OB`. + +- [x] **Step 4: Commit** + +```bash +pre-commit run --files .claude/skills/target-debug/SKILL.md +git add .claude/skills/target-debug/SKILL.md +git commit -m "docs(target-debug): vector catch + Cortex-M fault autopsy recipe + +Verified on : deliberate bad-address write halted via vector_catch, +CFSR= BFAR=0xCF000000, stacked pc resolved by addr2line." +``` + +--- + +### Task 5: SWO/ITM experiment — exception trace & hardware PC sampling + +This is an EXPERIMENT task with an explicit gate: the section lands **unmarked only if packets are actually captured** on a rig board; otherwise it lands tagged `(untested — SWO wiring unconfirmed on this rig)`. Budget: 30 min of hardware time, then decide. + +**Files:** +- Modify: `.claude/skills/target-debug/SKILL.md` — new subsection inside the PC-sampling section (after the OpenOCD variant paragraph) + +**Interfaces:** +- Consumes: `$JB`, `$JB_UID`, `$JLINK_DEVICE` from Task 2. +- Produces: verified-or-tagged status consumed by Task 7's table row for SWO. + +- [x] **Step 1: Probe for SWO output (gate experiment)** + +```bash +python3 test/hil/board_lock.py hold $JB --reason "skill-enhance verify: SWO" +# arm DWT sources while the fw runs (background mem write, no halt): +printf 'w4 E0001000, 0x00011401\nqc\n' | JLinkExe -device $JLINK_DEVICE -SelectEmuBySN $JB_UID -if swd -speed 4000 -autoconnect 1 -nogui 1 +# EXCTRCENA(16)|PCSAMPLENA(12)|SYNCTAP(10)|CYCCNTENA(0); tune POSTPRESET[4:1] if PC samples flood — then hand the probe to the viewer: +timeout 20s JLinkSWOViewerCLExe -device $JLINK_DEVICE -usb $JB_UID -swofreq 4000000 -itmmask 0xFFFFFFFF | head -40 +``` +Gate: ANY decoded output (stimulus, PC samples, exception packets) = SWO wired on `$JB` → land unmarked with the observed invocation. No output → try one more J-Link board, then land tagged. Either way `release $JB` after reflashing nothing (this experiment flashes nothing). + +- [x] **Step 2: Insert the section (wording per gate outcome)** + +Append to the `## PC-sampling` section: + +```markdown +### SWO/ITM — hardware-timed trace on one pin (J-Link) + +If the board routes SWO (TRACESWO), DWT emits packets with ZERO code change: +**exception trace** (`DWT_CTRL` bit16 EXCTRCENA) — every IRQ enter/exit, +timestamped, the ISR-ordering evidence the ring buffer needs code for — and +**hardware PC sampling** (bit12 PCSAMPLENA), better histograms than DWT_PCSR +polling. Arm the bits, then give the probe to the viewer (one client rule): + +```bash +printf 'w4 E0001000, 0x00011401\nqc\n' | JLinkExe -device $JLINK_DEVICE -SelectEmuBySN ... +timeout 20s JLinkSWOViewerCLExe -device $JLINK_DEVICE -usb -swofreq 4000000 -itmmask 0xFFFFFFFF +``` + +SWO needs the pin physically wired to the probe — many rig boards route only +SWDIO/SWCLK. If the viewer shows nothing, that is the wiring, not the recipe. +``` + +If the gate FAILED on both boards, append ` (untested — SWO wiring unconfirmed on this rig)` to the subsection heading and keep the text. + +- [x] **Step 3: Commit** + +```bash +pre-commit run --files .claude/skills/target-debug/SKILL.md +git add .claude/skills/target-debug/SKILL.md +git commit -m "docs(target-debug): SWO exception-trace / hw PC-sampling recipe + +Gate result on : ." +``` + +--- + +### Task 6: Flash verification, FreeRTOS thread awareness, semihosting & monitor-mode notes + +**Files:** +- Modify: `.claude/skills/target-debug/SKILL.md` — `## Warnings` section + GDB section tail + +**Interfaces:** +- Consumes: `$JB`, `$JB_UID`, `$JLINK_DEVICE` from Task 2. +- Produces: warning-list entries cited in Task 7's retrieval test scenarios. + +- [x] **Step 1: Add flash-content verification to Warnings** + +In `## Warnings`, after the "A marginal link can fake a deterministic firmware bug" bullet, add: + +```markdown +- **"Flash OK" can lie** (silent no-op: old firmware keeps running after a + green flash). When behavior contradicts the code you think is flashed, + verify flash against the build: + `arm-none-eabi-objcopy -O binary fw.elf /tmp/fw.bin`, then J-Link + `verifybin /tmp/fw.bin,` (Commander) or OpenOCD + `verify_image /tmp/fw.bin ` — a mismatch means reflash with + verification before debugging another minute. +``` + +- [x] **Step 2: Add FreeRTOS + semihosting + monitor-mode notes to the GDB section** + +Append to the end of the `## GDB — state autopsy and watchpoints` section (after the Task-2 additions): + +```markdown +FreeRTOS examples (`*_freertos`): add `-rtos GDBServer/RTOSPlugin_FreeRTOS` +to JLinkGDBServer (OpenOCD: `-rtos FreeRTOS` on the target) and `info +threads` / `thread ` shows every task's stack — a USB task blocked on a +queue vs. spinning is one `bt` away. Semihosting is never the answer here: +each call traps and halts the core — RTT does the same job without stopping. +**Monitor-mode debugging** (J-Link, M3+) can keep the USB ISR serviced while +you sit at a breakpoint — needs SEGGER's `JLINK_MONITOR.c`/ISR files compiled +in + `SetMonModeDebug=1`; not set up in this repo, reach for it when a bug +truly needs live breakpoints without killing the bus: + (untested). +``` + +- [x] **Step 3: Hardware-verify verifybin + FreeRTOS awareness on $JB** + +```bash +python3 test/hil/board_lock.py hold $JB --reason "skill-enhance verify: verifybin+rtos" +# (a) verifybin positive path against whatever is flashed — first reflash a known build: +# flash examples/cmake-build-$JB/device/cdc_msc, then: +arm-none-eabi-objcopy -O binary examples/cmake-build-$JB/device/cdc_msc/cdc_msc.elf /tmp/fw.bin +printf 'verifybin /tmp/fw.bin,\nqc\n' | \ + JLinkExe -device $JLINK_DEVICE -SelectEmuBySN $JB_UID -if swd -speed 4000 -autoconnect 1 -nogui 1 +# (b) rtos plugin: flash cdc_msc_freertos for $JB (build if missing), start +JLinkGDBServer -device $JLINK_DEVICE -select usb=$JB_UID -if swd -speed 4000 -port 2331 -nogui -rtos GDBServer/RTOSPlugin_FreeRTOS & +timeout 60 arm-none-eabi-gdb -batch -ex 'target remote :2331' -ex 'monitor halt' -ex 'info threads' \ + +python3 test/hil/board_lock.py release $JB # after pristine reflash +``` +Expected: (a) `Verify successful.` (b) `info threads` lists FreeRTOS tasks (`usbd`, `IDLE`, ...). If the plugin errors ("Could not load RTOS plugin"), drop the JLinkGDBServer variant from the Step-2 text and keep only the OpenOCD `-rtos FreeRTOS` form tagged `(untested)`. + +- [x] **Step 4: Commit** + +```bash +pre-commit run --files .claude/skills/target-debug/SKILL.md +git add .claude/skills/target-debug/SKILL.md +git commit -m "docs(target-debug): flash verifybin, FreeRTOS thread awareness, monitor-mode pointer + +Verified on : verifybin 'Verify successful.'; info threads listed tasks." +``` + +--- + +### Task 7: Intrusiveness table integration, agent update, retrieval test + +**Files:** +- Modify: `.claude/skills/target-debug/SKILL.md` — the technique/intrusiveness table +- Modify: `.claude/agents/target-debugger.md` — primary-playbook bullet + +**Interfaces:** +- Consumes: verified/untested status of every technique from Tasks 2-6. + +- [x] **Step 1: Extend the intrusiveness table** + +The table under `## Pick the least intrusive technique that can answer the question` currently has 5 rows (PC-sampling → GDB halt). Replace it with (keep the header row and any wording the earlier tasks did not contradict): + +```markdown +| Technique | Intrusiveness | Reach for it when | +|---|---|---| +| PC-sampling | none — no halt, no code change | core wedged/spinning somewhere unknown (rusb2 FRDY) | +| SWO exception trace / hw PC-sample | none — needs SWO pin wired | ISR ordering/timing with zero code change | +| Vector catch | none until a fault fires | crash-shaped wedges — autopsy AT the faulting pc | +| RAM ring-buffer | ~tens of cycles per event | ISR ordering/timing bugs (musb babble) | +| TU_LOG (RTT) | µs per line | logic bugs that survive logging (J-Link or OpenOCD rtt) | +| TU_LOG (UART) | ms per line — blocking write | same, when no debug-probe RTT path | +| dprintf / conditional breakpoint | halt+resume per hit (~ms) | low-rate probes post-wedge; never ISR-rate events | +| GDB halt / breakpoints | stops USB service entirely | post-mortem state autopsy once wedged | +``` + +If Task 5's gate failed, keep the SWO row but append ` (untested)` in its "Reach for it" cell. + +- [x] **Step 2: Update the agent's playbook bullet** + +In `.claude/agents/target-debugger.md`, replace: + +```markdown +- `.claude/skills/target-debug/SKILL.md` — your primary playbook: technique + choice by intrusiveness, channel choice by link topology, capture recipes, + GDB autopsy, all rig warnings. +``` + +with: + +```markdown +- `.claude/skills/target-debug/SKILL.md` — your primary playbook: technique + choice by intrusiveness, channel choice by link topology, capture recipes, + breakpoint/watchpoint budget and cost model, vector catch + fault autopsy, + GDB autopsy, all rig warnings. +``` + +- [x] **Step 3: Word-count and stale-reference check** + +Run: `wc -w .claude/skills/target-debug/SKILL.md` — expected ≤ 2 700. If over, trim prose (not recipes) until under. +Run: `grep -n 'J-Link only' .claude/skills/target-debug/SKILL.md` — expected: no output (Task 3 removed the claim). + +- [x] **Step 4: Retrieval test (skill-TDD GREEN gate)** + +Dispatch a fresh read-only subagent (Explore) that reads ONLY the updated `.claude/skills/target-debug/SKILL.md` and answers: + +1. "A CH32 board's firmware wedges; you suspect a HardFault loop. Least-intrusive next step?" — expected: vector catch (with the RISC-V caveat noted: CH32 is RISC-V → breakpoint on trap handler). +2. "You need RTT logs on an ST-Link-only board." — expected: OpenOCD `rtt setup/start/server`, NOT "impossible/J-Link only". +3. "Who is writing 0 into a busy flag, under live traffic?" — expected: OpenOCD value-match watchpoint `wp 4 w 0`, NOT a GDB conditional watch (halt-per-hit cost). +4. "Flash reported OK but behavior matches last week's build." — expected: verifybin/verify_image. +5. "You set `watch xfer_status[2][1]` and GDB answered `Watchpoint 2:` (no 'Hardware'). Proceed?" — expected: NO — software-watchpoint fallback single-steps the program; delete and narrow the expression. + +All five must route correctly; a miss = fix the text (usually the table row or a heading), re-test. + +- [x] **Step 5: Final commit** + +```bash +pre-commit run --files .claude/skills/target-debug/SKILL.md .claude/agents/target-debugger.md +git add .claude/skills/target-debug/SKILL.md .claude/agents/target-debugger.md +git commit -m "docs(target-debug): integrate new techniques into intrusiveness table; agent playbook bullet + +Retrieval test: 4/4 scenarios routed correctly." +``` + +--- + +## Deferred / out of scope (deliberate) + +- **ETM / J-Trace instruction trace** — no J-Trace hardware on the rig; UM08001 "Trace" chapter is linked for the day one arrives. +- **Monitor-mode debugging as a working recipe** — needs SEGGER monitor files compiled into firmware (a firmware feature, not a doc change); landed as a pointer + `(untested)` in Task 6. +- **ITM stimulus-port logging backend for TU_LOG** — would be a `lib/` + `LOGGER=itm` firmware feature; out of scope for a skill-doc plan. +- **GDB tracepoints (`trace`/`tfind`)** — need a tracing-capable stub; neither JLinkGDBServer nor OpenOCD implements them for Cortex-M. From 9c6c0390a0b163c8ea64145f759118795c45b47f Mon Sep 17 00:00:00 2001 From: hathach Date: Thu, 23 Jul 2026 13:38:13 +0700 Subject: [PATCH 05/11] =?UTF-8?q?docs(skills):=20formatting=20feedback=20?= =?UTF-8?q?=E2=80=94=20agent=20skill=20table,=20probe=20bullets,=20aligned?= =?UTF-8?q?=20columns?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - target-debugger: skill list is now a table referencing skills by name only (path pattern stated once). - target-debug: probe-mapping run-on paragraph split into bullets; drop the GDB Ninth-Edition caveat (calibre now holds the Tenth-Edition PDF, id 2264). - Align markdown table columns across target-debug, usb-sniffer, usbmon, hil, usbtest and the agent (7 tables); tables with paragraph-length cells left unpadded (usbmon symptom map, usbtest case map). --- .claude/agents/target-debugger.md | 30 +++++++----------- .claude/skills/hil/SKILL.md | 10 +++--- .claude/skills/target-debug/SKILL.md | 47 +++++++++++++++------------- .claude/skills/usb-sniffer/SKILL.md | 12 +++---- .claude/skills/usbmon/SKILL.md | 24 +++++++------- .claude/skills/usbtest/SKILL.md | 12 +++---- 6 files changed, 65 insertions(+), 70 deletions(-) diff --git a/.claude/agents/target-debugger.md b/.claude/agents/target-debugger.md index 4d874e38c..78110b179 100644 --- a/.claude/agents/target-debugger.md +++ b/.claude/agents/target-debugger.md @@ -8,26 +8,18 @@ You debug one failing USB behavior on one physical board until you can name the mechanism — or report exactly what you ruled out. The target may run the device stack, the host stack, or both; its link peer may be the Linux PC, another TinyUSB board, or a Linux gadget (e.g. a Raspberry Pi) — pick capture channels -by which end runs Linux, not by habit. These repo skills are your source of -truth; read the relevant SKILL.md BEFORE acting: +by which end runs Linux, not by habit. These repo skills (each at +`.claude/skills//SKILL.md`) are your source of truth; read the relevant +one BEFORE acting: -- `.claude/skills/target-debug/SKILL.md` — your primary playbook: technique - choice by intrusiveness, channel choice by link topology, capture recipes, - breakpoint/watchpoint budget and cost model, vector catch + fault autopsy, - SWO trace, GDB autopsy, all rig warnings. -- `.claude/skills/hil/SKILL.md` — host/config selection, board lock protocol, - `hil_test.py` invocation. -- `.claude/skills/usbmon/SKILL.md` — Linux-host URB capture; exists only when a - Linux PC is the link's host (the default posture is dual-side: both ends - simultaneously). -- `.claude/skills/usb-sniffer/SKILL.md` — wire-level capture with the hardware - tap: when the host can't see the bus (device never enumerates, pre-URB - failures), when usbmon and target logs disagree — the wire arbitrates — or - when TinyUSB is the host and no end has usbmon. -- `.claude/skills/usb-kernel-debug/SKILL.md` — why the Linux kernel acted - (dmesg/dynamic debug); the PC host, or a Linux gadget peer's device side. -- `.claude/skills/usb-kernel-recover/SKILL.md` — only when the DUT or fixture - wedges the rig PC's Linux host stack. +| Skill | Use for | +|--------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| target-debug | primary playbook — technique choice by intrusiveness, channel choice by link topology, capture recipes, bp/wp budget + cost model, vector catch + fault autopsy, SWO trace, GDB autopsy, rig warnings | +| hil | host/config selection, board lock protocol, `hil_test.py` invocation | +| usbmon | Linux-host URB capture; only when a Linux PC is the link's host (default posture: dual-side, both ends simultaneously) | +| usb-sniffer | wire-level capture (hardware tap): host can't see the bus, usbmon vs target logs disagree, or TinyUSB is the host (no usbmon anywhere) | +| usb-kernel-debug | why the Linux kernel acted (dmesg/dynamic debug); PC host or a Linux gadget peer's device side | +| usb-kernel-recover | only when the DUT or fixture wedges the rig PC's Linux host stack | ## The loop (deliberately serial — no fan-out) diff --git a/.claude/skills/hil/SKILL.md b/.claude/skills/hil/SKILL.md index 7b76bf1b8..2aeaa10af 100644 --- a/.claude/skills/hil/SKILL.md +++ b/.claude/skills/hil/SKILL.md @@ -7,11 +7,11 @@ description: Use when running TinyUSB Hardware-in-the-Loop (HIL) tests on physic Run TinyUSB HIL tests on real boards. **Run `hostname` first** — it tells you which host you are on, which determines the default config and whether remote mode is possible. -| Host | Local config | Remote (SSH → ci.lan)? | -|------|--------------|------------------------| -| `htpc` (dev PC) | `test/hil/local.json` | yes (large pool, `test/hil/tinyusb.json`) | -| `ci` (the rig) | `test/hil/tinyusb.json` (large pool) | no — can't SSH to htpc, and boards are already local | -| `hifiphile` (external rig) | `test/hil/hfp.json` | no outbound SSH to htpc/ci; SSH-reachable FROM both | +| Host | Local config | Remote (SSH → ci.lan)? | +|----------------------------|--------------------------------------|------------------------------------------------------| +| `htpc` (dev PC) | `test/hil/local.json` | yes (large pool, `test/hil/tinyusb.json`) | +| `ci` (the rig) | `test/hil/tinyusb.json` (large pool) | no — can't SSH to htpc, and boards are already local | +| `hifiphile` (external rig) | `test/hil/hfp.json` | no outbound SSH to htpc/ci; SSH-reachable FROM both | Default to **local**. Use **remote** only when on `htpc` and the user says `remote`/`ci.lan`. Never attempt remote on `ci`. diff --git a/.claude/skills/target-debug/SKILL.md b/.claude/skills/target-debug/SKILL.md index d0ceef2a9..a89162ed6 100644 --- a/.claude/skills/target-debug/SKILL.md +++ b/.claude/skills/target-debug/SKILL.md @@ -10,12 +10,12 @@ stack (`hcd_*`/`tuh_*`), or both. Its link peer is not always a Linux PC: a TinyUSB host may face another TinyUSB board or a Linux gadget (e.g. a Raspberry Pi). Pick capture channels by which end runs Linux, not by habit: -| Skill | Answers | Exists when | -|---|---|---| -| `usbmon` | what the Linux host exchanged (URBs) | a Linux PC is the link's host | +| Skill | Answers | Exists when | +|--------------------|----------------------------------------------------|---------------------------------------------------| +| `usbmon` | what the Linux host exchanged (URBs) | a Linux PC is the link's host | | `usb-kernel-debug` | why the Linux kernel acted (dmesg / dynamic debug) | Linux on either end: PC host or Linux gadget peer | -| **`target-debug`** | **what the target did** (logs, driver state, PC) | always — either role, needs a debug probe | -| `usb-sniffer` | what crossed the wire (PIDs, handshakes, resets) | hardware tap cabled in — role-agnostic | +| **`target-debug`** | **what the target did** (logs, driver state, PC) | always — either role, needs a debug probe | +| `usb-sniffer` | what crossed the wire (PIDs, handshakes, resets) | hardware tap cabled in — role-agnostic | For enumeration/transfer bugs the default posture is **dual-side capture** — both ends simultaneously: usbmon + a target @@ -35,12 +35,15 @@ python3 test/hil/board_lock.py release ``` Board → probe mapping: `test/hil/tinyusb.json` — `flasher.name` is the probe -family, `flasher.uid` the **probe serial** (many identical probes on the rig: -J-Link needs `-SelectEmuBySN ` / GDB server `-select usb=`; OpenOCD -`-c 'adapter serial '`). `JLINK_DEVICE` / `OPENOCD_OPTION` come from -`hw/bsp//boards//board.cmake` (or `board.mk`); find the family -with `ls -d hw/bsp/*/boards/`. Run on the host that owns the probe — -config is `test/hil/tinyusb.json` on ci, `local.json` on htpc (`hil` skill). +family, `flasher.uid` the **probe serial** (many identical probes on the rig): + +- Select the probe by serial: J-Link `-SelectEmuBySN `, its GDB server + `-select usb=`, OpenOCD `-c 'adapter serial '`. +- `JLINK_DEVICE` / `OPENOCD_OPTION`: from + `hw/bsp//boards//board.cmake` (or `board.mk`); family via + `ls -d hw/bsp/*/boards/`. +- Run on the host that owns the probe — config `test/hil/tinyusb.json` on ci, + `local.json` on htpc (`hil` skill). ## Pick the least intrusive technique that can answer the question @@ -48,16 +51,16 @@ Observation can mask the bug — the ch32v307 Heisenbug changed behavior under logging *and* under the debugger. If the bug disappears when instrumented, that IS a finding (timing-sensitive): move down in intrusiveness, not up. -| Technique | Intrusiveness | Reach for it when | -|---|---|---| -| PC-sampling | none — no halt, no code change | core wedged/spinning somewhere unknown (rusb2 FRDY) | -| SWO exception trace / hw PC-sample | none — needs SWO pin wired | ISR ordering/timing with zero code change | -| Vector catch | none until a fault fires | crash-shaped wedges — autopsy AT the faulting pc | -| RAM ring-buffer | ~tens of cycles per event | ISR ordering/timing bugs (musb babble) | -| TU_LOG (RTT) | µs per line | logic bugs that survive logging (J-Link or OpenOCD rtt) | -| TU_LOG (UART) | ms per line — blocking write | same, when no debug-probe RTT path | -| dprintf / conditional breakpoint | halt+resume per hit (~ms) | low-rate probes post-wedge; never ISR-rate events | -| GDB halt / breakpoints | stops USB service entirely | post-mortem state autopsy once wedged | +| Technique | Intrusiveness | Reach for it when | +|------------------------------------|--------------------------------|---------------------------------------------------------| +| PC-sampling | none — no halt, no code change | core wedged/spinning somewhere unknown (rusb2 FRDY) | +| SWO exception trace / hw PC-sample | none — needs SWO pin wired | ISR ordering/timing with zero code change | +| Vector catch | none until a fault fires | crash-shaped wedges — autopsy AT the faulting pc | +| RAM ring-buffer | ~tens of cycles per event | ISR ordering/timing bugs (musb babble) | +| TU_LOG (RTT) | µs per line | logic bugs that survive logging (J-Link or OpenOCD rtt) | +| TU_LOG (UART) | ms per line — blocking write | same, when no debug-probe RTT path | +| dprintf / conditional breakpoint | halt+resume per hit (~ms) | low-rate probes post-wedge; never ISR-rate events | +| GDB halt / breakpoints | stops USB service entirely | post-mortem state autopsy once wedged | ## TU_LOG capture @@ -293,7 +296,7 @@ the wire itself: `usb-sniffer` skill (hardware tap, PID-level). - J-Link (UM08001): — flash breakpoints, RTT, SWO, monitor mode, Commander. - OpenOCD: — `rtt`, `bp`/`wp`, `cortex_m vector_catch`/`maskisr`, `itm`/`tpiu`. - "Debugging with GDB" (§5.1 = break/watch/dprintf): Tenth Edition (GDB 18) - via calibre/`read-doc` — NOT the 2002 Ninth-Edition txt also there — or + via calibre/`read-doc`, or `curl -sL -o /tmp/gdb.pdf https://sourceware.org/gdb/current/onlinedocs/gdb.pdf` (the HTML mirror blocks fetchers). Installed `arm-none-eabi-gdb` `help ` is authoritative here. diff --git a/.claude/skills/usb-sniffer/SKILL.md b/.claude/skills/usb-sniffer/SKILL.md index a20aecf05..8d070709f 100644 --- a/.claude/skills/usb-sniffer/SKILL.md +++ b/.claude/skills/usb-sniffer/SKILL.md @@ -7,12 +7,12 @@ description: Use when you need wire-level USB evidence that host-side capture ca Extends the debugging trio with the layer below URBs: -| Skill | Answers | -|---|---| -| `usbmon` | what a Linux PC host exchanged (URBs) | -| `usb-kernel-debug` | why the Linux kernel acted (dmesg / dynamic debug) | -| `target-debug` | what the TinyUSB target did (device or host role) | -| **`usb-sniffer`** | **what actually crossed D+/D-** (PIDs, handshakes, resets, timing) | +| Skill | Answers | +|--------------------|--------------------------------------------------------------------| +| `usbmon` | what a Linux PC host exchanged (URBs) | +| `usb-kernel-debug` | why the Linux kernel acted (dmesg / dynamic debug) | +| `target-debug` | what the TinyUSB target did (device or host role) | +| **`usb-sniffer`** | **what actually crossed D+/D-** (PIDs, handshakes, resets, timing) | Reach for it when usbmon can't see (device never binds, pre-enumeration failures), can't be trusted (URB completed but did the wire really ACK?), or diff --git a/.claude/skills/usbmon/SKILL.md b/.claude/skills/usbmon/SKILL.md index a2e7c1ac1..490c79e7f 100644 --- a/.claude/skills/usbmon/SKILL.md +++ b/.claude/skills/usbmon/SKILL.md @@ -29,18 +29,18 @@ tshark -r cap.pcapng -Y 'usb.device_address==26' # filter to one device ## Filter (`-Y ''`) -| Goal | Expression | -|---|---| -| One device / endpoint | `usb.device_address==26` / `usb.endpoint_address==0x81` | -| IN (to host) / OUT (from host) | `usb.endpoint_address.direction==1` / `==0` | -| Submit / Complete event | `usb.urb_type=='S'` / `=='C'` (char literal: single quotes) | -| Control / bulk / interrupt / iso | `usb.transfer_type==2` / `3` / `1` / `0` | -| Only transfers carrying data | `usb.data_len>0` | -| GET_DESCRIPTOR / SET_ADDRESS / SET_CONFIGURATION | `usb.setup.bRequest==6` / `5` / `9` | -| SET_INTERFACE / CLEAR_FEATURE (clear-halt) | `usb.setup.bRequest==11` / `1` | -| Descriptor type DEVICE/CONFIG/STRING/HID-report | `usb.bDescriptorType==1` / `2` / `3` / `0x22` | -| Class / vendor requests | `usb.bmRequestType.type!=0` | -| STALLs / errors | `usb.urb_status!=0 && usb.urb_status!=-115` | +| Goal | Expression | +|--------------------------------------------------|-------------------------------------------------------------| +| One device / endpoint | `usb.device_address==26` / `usb.endpoint_address==0x81` | +| IN (to host) / OUT (from host) | `usb.endpoint_address.direction==1` / `==0` | +| Submit / Complete event | `usb.urb_type=='S'` / `=='C'` (char literal: single quotes) | +| Control / bulk / interrupt / iso | `usb.transfer_type==2` / `3` / `1` / `0` | +| Only transfers carrying data | `usb.data_len>0` | +| GET_DESCRIPTOR / SET_ADDRESS / SET_CONFIGURATION | `usb.setup.bRequest==6` / `5` / `9` | +| SET_INTERFACE / CLEAR_FEATURE (clear-halt) | `usb.setup.bRequest==11` / `1` | +| Descriptor type DEVICE/CONFIG/STRING/HID-report | `usb.bDescriptorType==1` / `2` / `3` / `0x22` | +| Class / vendor requests | `usb.bmRequestType.type!=0` | +| STALLs / errors | `usb.urb_status!=0 && usb.urb_status!=-115` | Combine with `&&` — e.g. one endpoint's data: `usb.endpoint_address==0x02 && usb.data_len>0`. diff --git a/.claude/skills/usbtest/SKILL.md b/.claude/skills/usbtest/SKILL.md index 850781d6e..76a01839c 100644 --- a/.claude/skills/usbtest/SKILL.md +++ b/.claude/skills/usbtest/SKILL.md @@ -80,12 +80,12 @@ python3 test/hil/usbtest.py --serial --keep-binding --tests 29 # one case ## Debug ladder (escalate in order) -| errno | Meaning | -|---|---| -| 110 | timeout — endpoint NAKing forever / device wedged | -| 32 | EPIPE — unexpected STALL | -| 5 | EIO — iso packet errors (check `dmesg`: "N errors out of M") | -| 71 | EPROTO — device answered wrong / too slow (after HC retries) | +| errno | Meaning | +|-------|--------------------------------------------------------------| +| 110 | timeout — endpoint NAKing forever / device wedged | +| 32 | EPIPE — unexpected STALL | +| 5 | EIO — iso packet errors (check `dmesg`: "N errors out of M") | +| 71 | EPROTO — device answered wrong / too slow (after HC retries) | 1. `usbtest.py` per-case output + its captured `dmesg` (`TEST n` markers bracket each case). 2. **usbmon** (`usbmon` skill): URB-level ground truth. **It cannot show data toggles or NAKs** — From b97f5dae5ba63015125595624e90e924b0b6d31b Mon Sep 17 00:00:00 2001 From: hathach Date: Thu, 23 Jul 2026 14:18:40 +0700 Subject: [PATCH 06/11] docs(target-debug): DWT data trace (verified both probe families); reorder to table; SWO enable chain MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - DWT data trace: FUNCTION 0b0011 (ARMv7-M ARM Table C1-21) streams value + accessor-PC packets per access, no halt, no code. Verified on stm32f407disco (J-Link SWORead: 451 KB, value = uptime ms, PC = tusb_time_millis_api) and stm32h743nucleo (OpenOCD/ST-Link tpiu capture: 607 KB, same decode). Caveat recorded: R/W-only trace floods on polled variables. - SWO enable chain documented: tools own TRCENA/ITM/TPIU; vendor part bites — H7 needs DBGMCU trace clocks, PB3 manually muxed to AF0, native stlink-dap (hla tpiu silently no-ops), the cfg's stm32h7x.swo object (the .tpiu object is the parallel port), traceclk = c_ck 400 MHz (wrong guesses: ratio-garbage or silence). - Sections reordered to match the intrusiveness table (least->most intrusive); cross-references fixed; table gains the data-trace row. --- .claude/skills/target-debug/SKILL.md | 251 ++++++++++++++++----------- 1 file changed, 148 insertions(+), 103 deletions(-) diff --git a/.claude/skills/target-debug/SKILL.md b/.claude/skills/target-debug/SKILL.md index a89162ed6..2514db86f 100644 --- a/.claude/skills/target-debug/SKILL.md +++ b/.claude/skills/target-debug/SKILL.md @@ -55,6 +55,7 @@ that IS a finding (timing-sensitive): move down in intrusiveness, not up. |------------------------------------|--------------------------------|---------------------------------------------------------| | PC-sampling | none — no halt, no code change | core wedged/spinning somewhere unknown (rusb2 FRDY) | | SWO exception trace / hw PC-sample | none — needs SWO pin wired | ISR ordering/timing with zero code change | +| DWT data trace | none — needs SWO pin wired | stream one address's accesses: value + accessor PC | | Vector catch | none until a fault fires | crash-shaped wedges — autopsy AT the faulting pc | | RAM ring-buffer | ~tens of cycles per event | ISR ordering/timing bugs (musb babble) | | TU_LOG (RTT) | µs per line | logic bugs that survive logging (J-Link or OpenOCD rtt) | @@ -62,6 +63,152 @@ that IS a finding (timing-sensitive): move down in intrusiveness, not up. | dprintf / conditional breakpoint | halt+resume per hit (~ms) | low-rate probes post-wedge; never ISR-rate events | | GDB halt / breakpoints | stops USB service entirely | post-mortem state autopsy once wedged | +## PC-sampling, SWO trace & DWT data trace — watch without halting + +`DWT_PCSR` (0xE000101C) returns the current PC on every read, target running +(Cortex-M3+; optional on M0+, reads 0 if absent; 0xFFFFFFFF = core halted or +WFI-asleep — `mem32 E000EDF0, 1`, DHCSR bit 17 S_HALT, tells which). One +probe serves one client: quit JLinkExe before starting JLinkGDBServer on the +same probe. Nailed the rusb2 FRDY wedge: + +```bash +for i in $(seq 300); do echo 'mem32 E000101C, 1'; done \ + | JLinkExe -device $JLINK_DEVICE -SelectEmuBySN -if swd -speed 4000 -autoconnect 1 -nogui 1 \ + | awk '/E000101C = /{print $3}' | sort | uniq -c | sort -rn | head +arm-none-eabi-addr2line -e -f -a 0x ... # PCs → functions +``` + +OpenOCD variant: repeat `mdw 0xE000101C` over telnet :4444. The histogram's +top entries are the spin site; a flat histogram = core is servicing normally. + +### SWO — hardware-timed trace on one pin (J-Link; verified on F407) + +If SWO (TRACESWO) is wired, DWT emits packets with ZERO code change: +**exception trace** (DWT_CTRL bit16 — every IRQ enter/exit, timestamped) and +**hardware PC sampling** (bit12), better histograms than DWT_PCSR polling. +SWOViewer tools decode only ITM *stimulus* (TinyUSB emits none) — capture +raw: + +```bash +# JLinkExe -CommandFile: +w4 E0001000, 0x00011401 # EXCTRCENA|PCSAMPLENA|SYNCTAP|CYCCNTENA +SWOStart 4000000 # explicit speed — autodetect fails headless +Sleep 3000 +SWORead # hex: 0x17+4B LE = PC sample, 0x0E+2B = IRQ enter/exit +``` + +Verified: 680 KB in 3 s (flash-range PC samples + SysTick enter/exit). +SWORead stuck at 0 = SWO pin not wired (many boards route only SWDIO/SWCLK). +Restore DWT_CTRL when done. + +### DWT data trace — stream one variable's accesses (value + PC), zero code + +The watchpoint comparators' non-halting sibling (ARMv7-M ARM Table C1-21; +absent on ARMv6-M): emit a packet on every access to a watched address +instead of halting. Verified on F407 (J-Link) and H743 (OpenOCD/ST-Link) — +both streamed `system_ticks`' live value plus the accessor PC +(`tusb_time_millis_api`): + +```bash +w4 E0001020, <&variable> # DWT_COMP0 (JLinkExe shown; OpenOCD: same via mww) +w4 E0001024, 0 # DWT_MASK0 = exact address +w4 E0001028, 0x3 # FUNCTION 0b0011: value + accessor-PC packets (0b0010: value only) +# stream: 0x47+4B = accessor PC, 0x87+4B = value, 0x70 = timestamp +``` + +Caveats: traces reads AND writes (no write-only encoding) — a variable the +main loop polls floods the pipe with read packets and squeezes out value +packets (seen on F407); disarm (`FUNCTION=0`) when done; costs one of the +DWT comparators. + +### Enabling SWO — the chain, and the vendor part that bites + +DEMCR.TRCENA → ITM (TCR/TER) → SWO/TPIU (protocol + prescaler) → pin mux. +Tools set the first three (`SWOStart` on SEGGER; `swo`/`tpiu` object +`enable` + `itm ports on` on OpenOCD) — pin mux and trace clocks are +per-family: + +- STM32F4: debug pins default to trace — nothing to configure. +- STM32H7 (verified, ST-Link): DBGMCU trace clocks + **PB3 muxed to AF0 by + hand** + native `stlink-dap.cfg` (the hla transport's tpiu path silently + does nothing) + the cfg-provided `stm32h7x.swo` object (`stm32h7x.tpiu` + is the parallel port — rejects uart). traceclk = c_ck 400 MHz, not HCLK: + too-slow guesses give ratio-garbled bytes, too-fast gives silence. + +```bash +openocd -f interface/stlink-dap.cfg -c 'adapter serial ' -f target/stm32h7x.cfg -c init \ + -c "mww 0x5C001004 0x00700000" \ + -c 'set m [read_memory 0x58020400 32 1]; mww 0x58020400 [expr {([lindex $m 0] & ~0xC0) | 0x80}]' \ + -c 'set a [read_memory 0x58020420 32 1]; mww 0x58020420 [expr {[lindex $a 0] & ~0xF000}]' \ + -c "stm32h7x.swo configure -protocol uart -traceclk 400000000 -pin-freq 2000000 -output /tmp/swo.bin" \ + -c "stm32h7x.swo enable" -c "itm ports on" \ + -c "sleep 3000" -c "stm32h7x.swo disable" -c shutdown # then decode /tmp/swo.bin +``` + +## Vector catch + fault autopsy — catch the crash, not the wedge + +A wedge that is really a fault (HardFault loop, lockup) autopsies best AT +the faulting instruction. Two hardware-proven gotchas: **FPB/DWT comparators +survive reflash and dead sessions** — a stale one fires as a phantom SIGTRAP +at an unrelated line of NEW firmware — and J-Link's reset strategy manages +vector-catch bits: scrub first, arm AFTER reset: + +```gdb +# scrub: FP_COMP0..5 = 0xE0002008..201C, DWT_FUNCTIONn = 0xE0001028 + n*0x10 +set *(unsigned*)0xE0002008 = 0 +# ... (repeat per comparator; count = the GDB section's budget reads) +# arm (after monitor reset; tool-agnostic — works via JLinkExe w4 too): +set *(unsigned*)0xE000EDFC |= (1<<10)|(1<<9)|(1<<8)|(1<<7)|(1<<6)|(1<<5)|(1<<4) +# = VC_HARDERR|INTERR|BUSERR|STATERR|CHKERR|NOCPERR|MMERR; bit0 VC_CORERESET halts at reset +``` + +OpenOCD native: `cortex_m vector_catch hard_err bus_err state_err chk_err mm_err`. +It halts at exception ENTRY (pc = handler, LR = EXC_RETURN 0xFFFFFFFx); decode: + +```gdb +p/x *(unsigned*)0xE000ED28 # CFSR — low byte MemManage, byte1 BusFault, top half UsageFault +p/x *(unsigned*)0xE000ED2C # HFSR — bit30 FORCED = an escalated lower-priority fault +p/x *(unsigned*)0xE000ED38 # BFAR — faulting address (valid if CFSR bit15 BFARVALID) +x/8wx $msp # stacked frame: r0 r1 r2 r3 r12 lr pc xpsr — pc = culprit +``` + +`addr2line -e ` names the line (verified: CFSR 0x8200, +BFAR = the bad address, stacked pc = the faulting ldr). Loads fault +precisely; stores usually IMPRECISERR (BFAR invalid, pc late). ARMv6-M has no +CFSR/BFAR, only VC_HARDERR|VC_CORERESET — stacked frame alone. Still a halt +(host URB timeouts apply); clear DEMCR (`&= ~0x7F0`) before handing back; +RISC-V: breakpoint the trap handler; mcause/mepc/mtval are the CFSR/BFAR analogs. + +## RAM ring-buffer trace + +The zero-print instrument (cracked the musb babble): a small event ring in the +dcd/hcd, dumped over GDB after the failure. Single-writer (ISR) — no locking: + +```c +typedef struct { uint16_t ev; uint16_t a; uint32_t b; } dbg_ev_t; +#define DBG_N 512 // power of two +static volatile dbg_ev_t dbg_ring[DBG_N]; // volatile REQUIRED: -Os dead-store- +static volatile uint32_t dbg_wr; // eliminates a write-only static array +static inline void DBG_EV(uint16_t ev, uint16_t a, uint32_t b) { + uint32_t i = dbg_wr++; + dbg_ring[i & (DBG_N - 1)] = (dbg_ev_t){ ev, a, b }; +} +// call sites: DBG_EV(__LINE__, ep_addr, count); — __LINE__ as event id +``` + +After building, `nm` the ELF for `dbg_ring`/`dbg_wr` — if they're missing the +compiler deleted your instrument and the run will "reproduce" with an empty ring. + +Order is the index; if durations matter add a `uint32_t t = DWT->CYCCNT` field +(enable once: `CoreDebug->DEMCR |= CoreDebug_DEMCR_TRCENA_Msk; DWT->CTRL |= 1;` +RISC-V: read `mcycle`). Let the failure happen, halt, then: + +```gdb +p dbg_wr # total events; oldest slot = dbg_wr & (DBG_N-1) once wrapped +p dbg_ring +dump binary memory /tmp/ring.bin &dbg_ring[0] &dbg_ring[512] +``` + ## TU_LOG capture Build with `LOG=2` (`LOG=3` adds per-transfer noise and much more timing skew). @@ -92,7 +239,7 @@ but ONLY what fits the drain model: the default SEGGER mode (NO_BLOCK_SKIP) **drops** writes once the ring fills with no reader, so an undrained target holds the first KB after boot, not the wedge tail. There is no overwrite mode in stock SEGGER RTT (only SKIP/TRIM/BLOCK): post-mortem RTT is evidence only -if a live drain was running — otherwise instrument with the RAM ring below. +if a live drain was running — otherwise instrument with the RAM ring above. Use `JLinkGDBServer -RTTTelnetPort 19021` + `JLinkRTTClient` for the drain (proven; note the server briefly halts the core on connect). `JLinkRTTLogger` fails to find the control block on some parts (LPC4088) even when it exists @@ -170,108 +317,6 @@ in ~5 s and the OS may reset/re-enumerate — after `continue`, the bus traffic shows recovery, not the original bug. Prefer one halt for a post-mortem dump over stepping through live USB traffic. -## Vector catch + fault autopsy — catch the crash, not the wedge - -A wedge that is really a fault (HardFault loop, lockup) autopsies best AT -the faulting instruction. Two hardware-proven gotchas: **FPB/DWT comparators -survive reflash and dead sessions** — a stale one fires as a phantom SIGTRAP -at an unrelated line of NEW firmware — and J-Link's reset strategy manages -vector-catch bits: scrub first, arm AFTER reset: - -```gdb -# scrub: FP_COMP0..5 = 0xE0002008..201C, DWT_FUNCTIONn = 0xE0001028 + n*0x10 -set *(unsigned*)0xE0002008 = 0 -# ... (repeat per comparator; count from the budget reads above) -# arm (after monitor reset; tool-agnostic — works via JLinkExe w4 too): -set *(unsigned*)0xE000EDFC |= (1<<10)|(1<<9)|(1<<8)|(1<<7)|(1<<6)|(1<<5)|(1<<4) -# = VC_HARDERR|INTERR|BUSERR|STATERR|CHKERR|NOCPERR|MMERR; bit0 VC_CORERESET halts at reset -``` - -OpenOCD native: `cortex_m vector_catch hard_err bus_err state_err chk_err mm_err`. -It halts at exception ENTRY (pc = handler, LR = EXC_RETURN 0xFFFFFFFx); decode: - -```gdb -p/x *(unsigned*)0xE000ED28 # CFSR — low byte MemManage, byte1 BusFault, top half UsageFault -p/x *(unsigned*)0xE000ED2C # HFSR — bit30 FORCED = an escalated lower-priority fault -p/x *(unsigned*)0xE000ED38 # BFAR — faulting address (valid if CFSR bit15 BFARVALID) -x/8wx $msp # stacked frame: r0 r1 r2 r3 r12 lr pc xpsr — pc = culprit -``` - -`addr2line -e ` names the line (verified: CFSR 0x8200, -BFAR = the bad address, stacked pc = the faulting ldr). Loads fault -precisely; stores usually IMPRECISERR (BFAR invalid, pc late). ARMv6-M has no -CFSR/BFAR, only VC_HARDERR|VC_CORERESET — stacked frame alone. Still a halt -(host URB timeouts apply); clear DEMCR (`&= ~0x7F0`) before handing back; -RISC-V: breakpoint the trap handler; mcause/mepc/mtval are the CFSR/BFAR analogs. - -## RAM ring-buffer trace - -The zero-print instrument (cracked the musb babble): a small event ring in the -dcd/hcd, dumped over GDB after the failure. Single-writer (ISR) — no locking: - -```c -typedef struct { uint16_t ev; uint16_t a; uint32_t b; } dbg_ev_t; -#define DBG_N 512 // power of two -static volatile dbg_ev_t dbg_ring[DBG_N]; // volatile REQUIRED: -Os dead-store- -static volatile uint32_t dbg_wr; // eliminates a write-only static array -static inline void DBG_EV(uint16_t ev, uint16_t a, uint32_t b) { - uint32_t i = dbg_wr++; - dbg_ring[i & (DBG_N - 1)] = (dbg_ev_t){ ev, a, b }; -} -// call sites: DBG_EV(__LINE__, ep_addr, count); — __LINE__ as event id -``` - -After building, `nm` the ELF for `dbg_ring`/`dbg_wr` — if they're missing the -compiler deleted your instrument and the run will "reproduce" with an empty ring. - -Order is the index; if durations matter add a `uint32_t t = DWT->CYCCNT` field -(enable once: `CoreDebug->DEMCR |= CoreDebug_DEMCR_TRCENA_Msk; DWT->CTRL |= 1;` -RISC-V: read `mcycle`). Let the failure happen, halt, then: - -```gdb -p dbg_wr # total events; oldest slot = dbg_wr & (DBG_N-1) once wrapped -p dbg_ring -dump binary memory /tmp/ring.bin &dbg_ring[0] &dbg_ring[512] -``` - -## PC-sampling (J-Link) — find where the core spins, without halting - -`DWT_PCSR` (0xE000101C) returns the current PC on every read, target running -(Cortex-M3+; optional on M0+, reads 0 if absent; 0xFFFFFFFF = core halted or -WFI-asleep — `mem32 E000EDF0, 1`, DHCSR bit 17 S_HALT, tells which). One -probe serves one client: quit JLinkExe before starting JLinkGDBServer on the -same probe. Nailed the rusb2 FRDY wedge: - -```bash -for i in $(seq 300); do echo 'mem32 E000101C, 1'; done \ - | JLinkExe -device $JLINK_DEVICE -SelectEmuBySN -if swd -speed 4000 -autoconnect 1 -nogui 1 \ - | awk '/E000101C = /{print $3}' | sort | uniq -c | sort -rn | head -arm-none-eabi-addr2line -e -f -a 0x ... # PCs → functions -``` - -OpenOCD variant: repeat `mdw 0xE000101C` over telnet :4444. The histogram's -top entries are the spin site; a flat histogram = core is servicing normally. - -### SWO — hardware-timed trace on one pin (J-Link; verified on F407) - -If SWO (TRACESWO) is wired, DWT emits packets with ZERO code change: -**exception trace** (DWT_CTRL bit16 — every IRQ enter/exit, timestamped) and -**hardware PC sampling** (bit12), better histograms than DWT_PCSR polling. -SWOViewer tools decode only ITM *stimulus* (TinyUSB emits none) — capture -raw: - -```bash -# JLinkExe -CommandFile: -w4 E0001000, 0x00011401 # EXCTRCENA|PCSAMPLENA|SYNCTAP|CYCCNTENA -SWOStart 4000000 # explicit speed — autodetect fails headless -Sleep 3000 -SWORead # hex: 0x17+4B LE = PC sample, 0x0E+2B = IRQ enter/exit -``` - -Verified: 680 KB in 3 s (flash-range PC samples + SysTick enter/exit). -SWORead stuck at 0 = SWO pin not wired (many boards route only SWDIO/SWCLK). -Restore DWT_CTRL when done. - ## Dual-side capture — the default for enumeration/transfer bugs Start both channels, then trigger the failing test (Linux-PC-host shown; From df3cea3d0e12c155e2a2e7f81dec031da38ab328 Mon Sep 17 00:00:00 2001 From: hathach Date: Fri, 24 Jul 2026 14:55:59 +0700 Subject: [PATCH 07/11] docs(superpowers): esp-target-debug design spec + executed implementation plan Spec (brainstormed): own-skill backend decision, PHY-conflict map, six verification gates, external-JTAG TODO. Plan executed same-day: all gates run on the rig; apptrace resolved per its own gate rule as (untested). --- .../2026-07-23-esp-target-debug-skill.md | 74 +++++++++++++ .../2026-07-23-esp-target-debug-design.md | 102 ++++++++++++++++++ 2 files changed, 176 insertions(+) create mode 100644 docs/superpowers/plans/2026-07-23-esp-target-debug-skill.md create mode 100644 docs/superpowers/specs/2026-07-23-esp-target-debug-design.md diff --git a/docs/superpowers/plans/2026-07-23-esp-target-debug-skill.md b/docs/superpowers/plans/2026-07-23-esp-target-debug-skill.md new file mode 100644 index 000000000..8b470eb74 --- /dev/null +++ b/docs/superpowers/plans/2026-07-23-esp-target-debug-skill.md @@ -0,0 +1,74 @@ +# esp-target-debug Skill Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Create `.claude/skills/esp-target-debug/SKILL.md` (Espressif built-in USB-Serial-JTAG debug backend) with every recipe verified on the rig's P4, the S3 PHY boundary verified both ways, plus pointer edits in `target-debug` and the `target-debugger` agent. + +**Architecture:** Per spec `docs/superpowers/specs/2026-07-23-esp-target-debug-design.md`. Verification-first: hardware gates 1–6 run before the skill text lands, so only proven content ships unmarked. One lock session per board. + +**Tech Stack:** ESP-IDF at `$HOME/code/esp-idf` (`export.sh` → `openocd-esp32`, `riscv32-esp-elf-gdb`, `xtensa-esp32s3-elf-gdb`, `esptool.py`), rig boards `espressif_p4_function_ev` (uid 6055F9F98715), `espressif_s3_devkitm` (uid 84F703C084E4). + +## Global Constraints + +- Worktree `/home/hathach/code/tinyusb/.claude/worktrees/improve-debug-skill-agent`, branch `claude/improve-debug-skill-agent`. +- Board-lock discipline per `hil` skill; reflash pristine firmware before release; evidence (command + output snippet) in commit message bodies. +- Formatting: aligned table columns, skill-name-only cross-references. +- Unverified content ships tagged `(untested)` or not at all. +- Espressif anything requires `. $HOME/code/esp-idf/export.sh` in that shell first. + +--- + +### Task 1: P4 recon + coexistence gate (spec gates 1) + +- [x] **Step 1: Environment + firmware recon** + +```bash +ls $HOME/code/esp-idf/export.sh && source $HOME/code/esp-idf/export.sh && which openocd riscv32-esp-elf-gdb +ls /home/hathach/code/tinyusb/examples/cmake-build-espressif_p4_function_ev 2>/dev/null || echo "no prebuilt" +lsusb -d 303a:1001 # USB-SJ devices present +``` +If no prebuilt firmware: build `device/cdc_msc_freertos` for the P4 (`idf.py -DBOARD=espressif_p4_function_ev build` in that example, per CLAUDE.md), else use the prebuilt binary. Identify the ELF path for gdb symbolization. + +- [x] **Step 2: Lock P4, ensure known firmware, confirm DUT traffic** + +```bash +python3 test/hil/board_lock.py hold espressif_p4_function_ev --reason "esp-target-debug verify: coexistence" +# flash known build (esptool/idf.py flash -p ), settle, then confirm enumeration: +lsusb | grep -i cafe # TinyUSB VID on the DUT port +# generate traffic: echo > /dev/ttyACM of the cdc, or timeout 5s cat +``` + +- [x] **Step 3: Attach openocd over USB-SJ while the device runs** + +```bash +openocd -f board/esp32p4-builtin.cfg -c 'adapter serial 6055F9F98715' & # gdb :3333 +riscv32-esp-elf-gdb -batch -ex 'target extended-remote :3333' -ex 'monitor halt' \ + -ex bt -ex 'monitor resume' +``` +Expected: backtrace with symbols; after resume the CDC device still answers (re-run the traffic check). Record: does the DUT drop off the bus during halt (host URB timeouts — expected per target-debug) and does it recover on resume without re-enumeration? + +- [x] **Step 4: Release-or-continue checkpoint** — keep the lock for Task 2 (same session). No commit yet; evidence to `/tmp/esp_evidence.txt`. + +### Task 2: P4 budget, watchpoint, threads, console (spec gates 2–4) + +- [x] **Step 1: Breakpoint/watchpoint budget** — RISC-V trigger count: in gdb `monitor riscv info` or set watchpoints until rejection; verify a hardware watchpoint on a TinyUSB variable (e.g. `watch -l` on a usbd counter) reports and hits. +- [x] **Step 2: FreeRTOS threads** — `info threads` after halt; expect ESP-IDF tasks incl. the USB task; note whether it works at attach or needs run→stop (mirror the ARM finding). +- [x] **Step 3: Console during traffic** — capture the USB-SJ console tty (the 303a:1001 CDC function) for a few seconds while DUT traffic runs; expect ESP-IDF log lines. Record the /dev node mapping by serial. +- [x] **Step 4: Reflash pristine, release P4 lock.** Evidence appended to `/tmp/esp_evidence.txt`. + +### Task 3: P4 apptrace spike — GATED (spec gate 5) + +Budget 30 min. `openocd -c 'esp apptrace start ...'` against a firmware built with apptrace enabled? Stock HIL firmware has no apptrace init — if a code change would be required, that's the gate answer: land apptrace as `(untested — needs CONFIG_APPTRACE + firmware init)` with the recipe sketch. Only a working capture lands unmarked. + +### Task 4: S3 boundary (spec gate 6) + +- [x] **Step 1: Lock S3, flash `board_test`** (no TinyUSB → PHY free). Attach `openocd -f board/esp32s3-builtin.cfg -c 'adapter serial 84F703C084E4'` + `xtensa-esp32s3-elf-gdb`: halt + bt works. +- [x] **Step 2: Flash a USB device example** — record the exact failure: does 303a:1001 vanish from lsusb (PHY switched), does openocd fail to attach or die mid-session? Capture verbatim error. +- [x] **Step 3: Reflash pristine (a USB example — that is the CI-expected state), release.** + +### Task 5: Write the skill + integration edits + commit + +- [x] **Step 1: Write `.claude/skills/esp-target-debug/SKILL.md`** per spec section order (role/defer, PHY map with verified boundary symptoms, toolchain+attach with the real commands from Tasks 1–4, technique mapping table with verified annotations, rig deltas, external-JTAG TODO). Aligned tables. +- [x] **Step 2: `target-debug` pointer** (2 lines, after probe-mapping bullets) + `target-debugger` agent table row. +- [x] **Step 3: pre-commit, single commit** with evidence summary from `/tmp/esp_evidence.txt`. +- [x] **Step 4: Retrieval sanity** — one fresh-subagent scenario: "debug a TinyUSB hang on the rig's P4" routes to esp-target-debug (not JLink recipes); "same on S3 while cdc_msc runs" routes to the PHY boundary + external-JTAG TODO. diff --git a/docs/superpowers/specs/2026-07-23-esp-target-debug-design.md b/docs/superpowers/specs/2026-07-23-esp-target-debug-design.md new file mode 100644 index 000000000..491466992 --- /dev/null +++ b/docs/superpowers/specs/2026-07-23-esp-target-debug-design.md @@ -0,0 +1,102 @@ +# esp-target-debug Skill Design + +Backend skill for debugging TinyUSB firmware on Espressif targets (rig: +`espressif_p4_function_ev`, `espressif_s3_devkitm`) via the chips' **built-in +USB-Serial-JTAG**, with external JTAG documented as a TODO until the rig has +an adapter. Companion to `target-debug`, which keeps the architecture-neutral +methodology (intrusiveness ladder, board locks, dual-side capture, diagnosis +standards) — this skill is the Espressif toolchain/probe backend, the same +boundary that makes `usb-kernel-debug` its own skill. + +## Goals + +- An agent can attach, halt, backtrace, set breakpoints/watchpoints, list + FreeRTOS threads, and capture logs on the rig's P4 **while TinyUSB device + traffic is live** — every recipe hardware-verified before landing unmarked + (the `target-debug` ethos). +- The S3's USB-SJ/OTG PHY conflict is mapped precisely, not hand-waved: + verified working via `board_test` (TinyUSB off — PHY free), verified failure + mode with a USB device example, external-JTAG escape hatch documented as + TODO. + +## Non-goals (deferred) + +- External JTAG bring-up (no adapter on the rig) — TODO section with S3 JTAG + pin notes (GPIO39-42) and openocd-esp32 adapter support pointers. +- Xtensa/S3 full parity under live USB traffic (needs external JTAG). +- ETM-class instruction trace; SystemView tooling beyond an apptrace spike. + +## Architecture + +New skill `.claude/skills/esp-target-debug/SKILL.md`; two integration edits: + +- `target-debug` gains a 2-line pointer under the probe-mapping bullets: + Espressif boards use a different toolchain, probe model, and trace story — + read `esp-target-debug`. +- `target-debugger` agent table gains an `esp-target-debug` row (name-only, + aligned columns, per the established conventions). + +Skill content (order): + +1. **Role + defer line** — methodology lives in `target-debug`; this file is + the Espressif backend. Built-in USB-SJ now; external JTAG TODO. +2. **PHY-conflict map** — + - S3: USB-SJ and OTG share one PHY (GPIO19/20). TinyUSB claiming the PHY + drops JTAG-over-USB mid-session: JTAG works for non-USB examples + (`board_test`), dies for USB device examples (verified boundary, exact + symptom recorded). External JTAG = the future escape hatch (TODO). + - P4: OTG-HS has a dedicated HS PHY; USB-SJ is separate — JTAG and the + TinyUSB DUT port coexist (verified). USB-SJ doubles as a live log + console during device traffic — the TU_LOG-equivalent channel. +3. **Toolchain & attach** — `. $HOME/code/esp-idf/export.sh` provides + `openocd-esp32` + `riscv32-esp-elf-gdb` (P4) / `xtensa-esp32s3-elf-gdb` + (S3). Rig path is raw openocd (HIL firmware isn't an idf project on disk): + `openocd -f board/esp32p4-builtin.cfg` with `adapter serial ` (USB-SJ + is VID 303A:1001; uid = the `flasher.uid` already in `tinyusb.json`), gdb + on :3333. `idf.py openocd` / `idf.py gdb` noted for idf-project work. +4. **Technique mapping table** (aligned) — ARM technique → Espressif + equivalent: + + | target-debug technique | Espressif backend | + |---|---| + | GDB autopsy, bp/wp | same flow; RISC-V trigger module (P4) / Xtensa 2 bp + 2 wp (S3); budget read verified on P4 | + | Vector catch | none — breakpoint the panic handler; decode `mcause`/`mepc`/`mtval` (P4) | + | SWO / DWT data trace | none — apptrace over JTAG is the analog (gated spike; lands `(untested)` if it fails) | + | RTT / TU_LOG | USB-SJ console — on P4 it coexists with DUT traffic | + | FreeRTOS threads | native in openocd-esp32 — `info threads` out of the box | + | verifybin | `esptool.py verify_flash` | + +5. **Rig discipline deltas** — same `board_lock.py` protocol; flasher is + esptool (serial-port-by-uid); reflash pristine before release; one client + per USB-SJ device. +6. **External JTAG — TODO** — S3 JTAG pins, adapter classes openocd-esp32 + supports, and the efuse caveat (JTAG pin selection), unverified. + +## Verification gates (execution order) + +All under board locks, serial, evidence in commit messages: + +1. **P4 coexistence (headline)**: flash a device example, confirm enumeration + + traffic on the DUT port, then attach openocd+gdb over USB-SJ → + halt, `bt`, resume — device stays functional after resume. +2. **P4 budget**: read trigger/watchpoint counts via openocd/gdb; set a + hardware watchpoint on a TinyUSB variable, confirm hit. +3. **P4 threads**: `info threads` lists ESP-IDF tasks (usbd task visible). +4. **P4 console**: capture USB-SJ console log output during device traffic. +5. **P4 apptrace spike (gated)**: bounded attempt; verified recipe or + `(untested)` tag. +6. **S3 boundary**: `board_test` flashed → attach works (halt+bt); then a USB + device example → record the exact JTAG failure symptom when the PHY + switches. No further S3 work (external JTAG TODO). + +## Constraints + +- Worktree `claude/improve-debug-skill-agent`; commit per gate; pre-commit + before each; no Co-Authored-By trailers. +- Formatting conventions already established: aligned table columns, + skill-name-only cross references, bullets over run-on paragraphs. +- Espressif builds need `export.sh` first (CLAUDE.md); P4/S3 examples build + via idf.py — reuse existing HIL-built firmware where possible instead of + rebuilding. +- Hardware-verify-before-landing: unverified content ships tagged + `(untested)` or not at all. From 5066ac7b31445f64aa6007fbe28f03e049aec0b6 Mon Sep 17 00:00:00 2001 From: hathach Date: Fri, 24 Jul 2026 14:55:59 +0700 Subject: [PATCH 08/11] =?UTF-8?q?docs(skills):=20esp-target-debug=20?= =?UTF-8?q?=E2=80=94=20Espressif=20built-in=20USB-JTAG=20backend,=20rig-ve?= =?UTF-8?q?rified?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit P4 (hand-wired USB-SJ breakout, GPIO24/25 from header J1): - COEXISTENCE verified: 303a:1001 + cafe:4008 enumerated simultaneously; gdb attach/halt/bt during live CDC traffic; symbolized app backtrace (tud_task_ext <- usb_device_task <- vPortTaskWrapper) - set ESP_RTOS FreeRTOS before board cfg -> full dual-core task list; without it, bare 'Remote target' - attach-may-reset nuance flagged (post-mortem autopsy caution) - console = UART0 (CP2102 flasher tty) on stock builds; D+/D- swap symptom documented (low-speed + error -71 vs full-speed) S3 (same-port PHY swap): - boundary captured live in dmesg: same hub port flips 303a:1001 -> cafe:4008 as the app boots; openocd 'esp_usb_jtag: could not find or open device!' verbatim - attach/halt/symbol resolution verified via board_test (usb_new_phy absent from ELF when CFG_TUD/TUH=0); app-context keep-alive quirk (~4 s unattended drop, -71 half-dead, UART esptool reset recovers); cpu1 OCD_ID=0 -> ESP_ONLYCPU=1; telnet-halt + gdb-read scripted pattern; RTC_CNTL PHY-mux reference (0x60008120) + esptool read_mem/write_mem - target-debug pointer + target-debugger agent table row --- .claude/agents/target-debugger.md | 1 + .claude/skills/esp-target-debug/SKILL.md | 99 ++++++++++++++++++++++++ .claude/skills/target-debug/SKILL.md | 2 + 3 files changed, 102 insertions(+) create mode 100644 .claude/skills/esp-target-debug/SKILL.md diff --git a/.claude/agents/target-debugger.md b/.claude/agents/target-debugger.md index 78110b179..68e431683 100644 --- a/.claude/agents/target-debugger.md +++ b/.claude/agents/target-debugger.md @@ -16,6 +16,7 @@ one BEFORE acting: |--------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | target-debug | primary playbook — technique choice by intrusiveness, channel choice by link topology, capture recipes, bp/wp budget + cost model, vector catch + fault autopsy, SWO trace, GDB autopsy, rig warnings | | hil | host/config selection, board lock protocol, `hil_test.py` invocation | +| esp-target-debug | Espressif S3/P4 backend: built-in USB-Serial-JTAG attach, PHY-conflict map, FreeRTOS threads via ESP_RTOS | | usbmon | Linux-host URB capture; only when a Linux PC is the link's host (default posture: dual-side, both ends simultaneously) | | usb-sniffer | wire-level capture (hardware tap): host can't see the bus, usbmon vs target logs disagree, or TinyUSB is the host (no usbmon anywhere) | | usb-kernel-debug | why the Linux kernel acted (dmesg/dynamic debug); PC host or a Linux gadget peer's device side | diff --git a/.claude/skills/esp-target-debug/SKILL.md b/.claude/skills/esp-target-debug/SKILL.md new file mode 100644 index 000000000..ca6c7c401 --- /dev/null +++ b/.claude/skills/esp-target-debug/SKILL.md @@ -0,0 +1,99 @@ +--- +name: esp-target-debug +description: Use when debugging TinyUSB firmware on Espressif boards (ESP32-S3/P4 on the rig — dcd_dwc2 examples, idf.py builds) with the chips' built-in USB-Serial-JTAG — attach/halt/backtrace, breakpoints, FreeRTOS task lists, console capture — or when JTAG "could not find or open device", the 303a:1001 port vanishes, or the S3's debug port turns into the TinyUSB device. +--- + +# esp-target-debug — Espressif built-in USB-JTAG backend + +Methodology — intrusiveness ladder, board locks, dual-side capture, diagnosis +standards — lives in `target-debug`; this skill is the Espressif backend: a +different gdb, a different openocd (fork), no probe serial (the debugger IS a +USB device), and a PHY story that decides whether JTAG exists at all. +Built-in USB-Serial-JTAG only; external JTAG is a TODO (no rig adapter). + +## The PHY map — decides everything (verified on the rig) + +| Board | USB-SJ vs TinyUSB OTG | JTAG while USB device runs? | +|--------------------------|---------------------------------------------------|-----------------------------| +| espressif_p4_function_ev | separate pins: USB-SJ GPIO24/25 (FS), OTG own HS PHY | **yes — coexist** (verified: 303a:1001 + cafe:4008 enumerated simultaneously, gdb attach during live CDC traffic) | +| espressif_s3_devkitm | ONE shared PHY/port | **no** — the same hub port flips 303a:1001 → cafe:4008 as the app boots (verified in dmesg); openocd then fails `esp_usb_jtag: could not find or open device!` | + +- S3 debugging windows: non-USB firmware (`board_test` — attach, halt, and + symbol resolution verified; `usb_new_phy` confirmed absent from the ELF + when `CFG_TUD/TUH_ENABLED` are 0), bootloader/ROM (always stable — chip + parked in download mode enumerates cleanly for minutes), or external JTAG + (TODO). +- **S3 app-context keep-alive quirk (verified)**: with app firmware running + and nothing attached, USB-SJ drops ~4 s after boot (device-side disconnect, + then half-dead `-71` setup failures until reset). Attach a client inside + the window — or once it survives the window it stays up. Recovery is + UART-side: `esptool.py --after hard_reset read_mac` on the CP2102 tty. +- **S3 batch-automation caveats**: this unit's cpu1 debug logic can fail + examination (`OCD_ID = 00000000`) — `-c 'set ESP_ONLYCPU 1'` degrades to + cpu0-only; xtensa-gdb batch `continue`/`interrupt` is async-flaky — for + scripted state reads, halt via telnet :4444 first, then attach gdb to the + stopped target (verified). Interactive sessions are unaffected. +- PHY mux reference: `RTC_CNTL_RTC_USB_CONF_REG` (0x60008120) bits + `SW_HW_USB_PHY_SEL`/`SW_USB_PHY_SEL` (TRM 10.56) — 0 = eFuse/hardware + control (the default we measured). `esptool.py read_mem/write_mem` peeks + and pokes registers over plain UART with the chip in download mode. +- P4 Function-EV has **no USB-SJ connector** — GPIO24 (D−, white) / GPIO25 + (D+, green) / GND are broken out from header J1 to a rig hub port (wired + 2026-07-23). Miswired D+/D− shows as `new low-speed USB device` + error + -71; correct shows `new full-speed`. +- Flashing always works regardless of PHY state: the rig flashes via the + CP2102N UART bridges (that's why `tinyusb.json` esptool uids are CP210x + serials, not MACs). + +## Attach + +```bash +. $HOME/code/esp-idf/export.sh # openocd-esp32, riscv32-/xtensa-esp32s3-elf-gdb, esptool +openocd -c 'set ESP_RTOS FreeRTOS' -f board/esp32p4-builtin.cfg \ + -c 'adapter serial 60:55:F9:F9:87:15' & # P4; S3: board/esp32s3-builtin.cfg + its MAC +riscv32-esp-elf-gdb -batch -ex 'target extended-remote :3333' \ + -ex 'tbreak tud_task_ext' -ex continue -ex bt -ex 'info threads' -ex detach +``` + +- `adapter serial` = the chip MAC **with colons** (`lsusb -v -d 303a:1001`, + or `/dev/serial/by-id/usb-Espressif_USB_JTAG_serial_debug_unit_-if00`). +- `set ESP_RTOS FreeRTOS` must precede the board cfg: with it, `info + threads` lists every task with name/state/CPU (verified: usbd Running + @CPU0, IDLE1 @CPU1, blinky, io, ipc0/1); without it, one bare + "Remote target". +- The ELF: `idf.py -B -DBOARD= build` under the example + (CLAUDE.md Espressif notes) — symbolized app backtraces verified + (`tud_task_ext` ← `usb_device_task` ← `vPortTaskWrapper`). +- **Attach may reset the target** — after a tbreak-continue the FreeRTOS + tick read 2 (boot-fresh) on a minutes-old session. Until pinned down, do + NOT trust built-in-JTAG attach for post-mortem autopsy of a wedged board + (`target-debug`'s attach-and-halt-only rule); capture state via console + or treat the reset as part of the reproduce cycle. +- Halting still stops USB service: expect the host to drop the DUT during + long halts; after detach the device may need a reset to re-enumerate + (UART-side `esptool.py read_mac` is a handy remote reset). + +## Technique mapping (vs the `target-debug` arsenal) + +| target-debug technique | Espressif backend | +|------------------------|-------------------| +| GDB autopsy, bp/wp | same flow via openocd-esp32 :3333; RISC-V triggers (P4) / Xtensa 2 bp + 2 wp (S3) | +| Vector catch | none — breakpoint the panic handler; `mcause`/`mepc`/`mtval` on P4 | +| SWO / DWT data trace | none — apptrace over JTAG is the analog (untested: needs CONFIG_APPTRACE + app init) | +| RTT / TU_LOG | console on **UART0 = the CP2102 flasher tty** by default (verified); USB-SJ console needs sdkconfig `ESP_CONSOLE_USB_SERIAL_JTAG` (untested) | +| FreeRTOS threads | native — `set ESP_RTOS FreeRTOS` (see Attach) | +| verifybin | `esptool.py verify_flash` (untested) | + +## Rig deltas + +- Locks/flash per `hil` skill; espressif flasher = esptool over the CP210x + UART tty (works with any PHY state, any firmware). +- One client per USB-SJ: openocd and any terminal on the USB-SJ CDC side + conflict the same way J-Link clients do. +- Reflash-pristine before release applies unchanged. + +## TODO — external JTAG (needs hardware) + +S3 JTAG pins GPIO39–42 (MTCK/MTDO/MTDI/MTMS) + any adapter openocd-esp32 +supports (ESP-Prog/FT2232-class); would give S3 debugging under live USB +traffic. Mind `EFUSE_DIS_PAD_JTAG` / JTAG-source strapping. Unverified. diff --git a/.claude/skills/target-debug/SKILL.md b/.claude/skills/target-debug/SKILL.md index 2514db86f..ff3f45d4e 100644 --- a/.claude/skills/target-debug/SKILL.md +++ b/.claude/skills/target-debug/SKILL.md @@ -44,6 +44,8 @@ family, `flasher.uid` the **probe serial** (many identical probes on the rig): `ls -d hw/bsp/*/boards/`. - Run on the host that owns the probe — config `test/hil/tinyusb.json` on ci, `local.json` on htpc (`hil` skill). +- Espressif boards (S3/P4): different toolchain, probe model, and PHY + constraints entirely — read `esp-target-debug` first. ## Pick the least intrusive technique that can answer the question From 22a155f091283596ee34550c42c075967841159b Mon Sep 17 00:00:00 2001 From: hathach Date: Fri, 24 Jul 2026 14:48:44 +0700 Subject: [PATCH 09/11] docs(skills): tighten esp-target-debug/target-debug; agent routes backends by board family MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - esp-target-debug: de-narrated (~1000 -> 847 words) — session-diary phrasing ('this unit', dates, 'we measured') replaced with durable reference facts; restructured per-board (P4 vs S3 notes); new Scripted-session gotchas section consolidates telnet-halt pattern, ESP_ONLYCPU, and ROM-frame guidance; UART-reset recipe stated once - target-debug: fix run-on seam from the -singlerun insertion - target-debugger agent: charter now resolves the board family FIRST and routes Espressif boards to esp-target-debug as primary playbook; skills table re-aligned - retrieval regression: 4/4 (agent routing, S3 keep-alive quirk, OpenOCD RTT on ST-Link, ROM-frame guidance) --- .claude/agents/target-debugger.md | 7 +- .claude/skills/esp-target-debug/SKILL.md | 103 ++++++++++++----------- .claude/skills/target-debug/SKILL.md | 7 +- 3 files changed, 64 insertions(+), 53 deletions(-) diff --git a/.claude/agents/target-debugger.md b/.claude/agents/target-debugger.md index 68e431683..e25ffa7f1 100644 --- a/.claude/agents/target-debugger.md +++ b/.claude/agents/target-debugger.md @@ -8,7 +8,10 @@ You debug one failing USB behavior on one physical board until you can name the mechanism — or report exactly what you ruled out. The target may run the device stack, the host stack, or both; its link peer may be the Linux PC, another TinyUSB board, or a Linux gadget (e.g. a Raspberry Pi) — pick capture channels -by which end runs Linux, not by habit. These repo skills (each at +by which end runs Linux, not by habit. Resolve the board's family first +(`ls -d hw/bsp/*/boards/`): Espressif boards are a different backend +entirely — esp-target-debug is your primary playbook there; every other +family uses target-debug's probe recipes directly. These repo skills (each at `.claude/skills//SKILL.md`) are your source of truth; read the relevant one BEFORE acting: @@ -16,7 +19,7 @@ one BEFORE acting: |--------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | target-debug | primary playbook — technique choice by intrusiveness, channel choice by link topology, capture recipes, bp/wp budget + cost model, vector catch + fault autopsy, SWO trace, GDB autopsy, rig warnings | | hil | host/config selection, board lock protocol, `hil_test.py` invocation | -| esp-target-debug | Espressif S3/P4 backend: built-in USB-Serial-JTAG attach, PHY-conflict map, FreeRTOS threads via ESP_RTOS | +| esp-target-debug | PRIMARY playbook for Espressif boards — built-in USB-Serial-JTAG attach, the PHY map that decides whether JTAG exists, FreeRTOS threads via ESP_RTOS; target-debug still supplies the methodology | | usbmon | Linux-host URB capture; only when a Linux PC is the link's host (default posture: dual-side, both ends simultaneously) | | usb-sniffer | wire-level capture (hardware tap): host can't see the bus, usbmon vs target logs disagree, or TinyUSB is the host (no usbmon anywhere) | | usb-kernel-debug | why the Linux kernel acted (dmesg/dynamic debug); PC host or a Linux gadget peer's device side | diff --git a/.claude/skills/esp-target-debug/SKILL.md b/.claude/skills/esp-target-debug/SKILL.md index ca6c7c401..c6d76c801 100644 --- a/.claude/skills/esp-target-debug/SKILL.md +++ b/.claude/skills/esp-target-debug/SKILL.md @@ -13,65 +13,74 @@ Built-in USB-Serial-JTAG only; external JTAG is a TODO (no rig adapter). ## The PHY map — decides everything (verified on the rig) -| Board | USB-SJ vs TinyUSB OTG | JTAG while USB device runs? | -|--------------------------|---------------------------------------------------|-----------------------------| -| espressif_p4_function_ev | separate pins: USB-SJ GPIO24/25 (FS), OTG own HS PHY | **yes — coexist** (verified: 303a:1001 + cafe:4008 enumerated simultaneously, gdb attach during live CDC traffic) | -| espressif_s3_devkitm | ONE shared PHY/port | **no** — the same hub port flips 303a:1001 → cafe:4008 as the app boots (verified in dmesg); openocd then fails `esp_usb_jtag: could not find or open device!` | +| Board | USB-SJ vs TinyUSB OTG | JTAG while USB device runs? | +|--------------------------|------------------------------------------------------|-----------------------------| +| espressif_p4_function_ev | separate pins: USB-SJ GPIO24/25 (FS), OTG own HS PHY | **yes — coexist** (verified: 303a:1001 + cafe:4008 simultaneously, gdb attach during live CDC traffic) | +| espressif_s3_devkitm | ONE shared PHY/port | **no** — the same hub port flips 303a:1001 → cafe:4008 as the app boots; openocd fails `esp_usb_jtag: could not find or open device!` | -- S3 debugging windows: non-USB firmware (`board_test` — attach, halt, and - symbol resolution verified; `usb_new_phy` confirmed absent from the ELF - when `CFG_TUD/TUH_ENABLED` are 0), bootloader/ROM (always stable — chip - parked in download mode enumerates cleanly for minutes), or external JTAG - (TODO). -- **S3 app-context keep-alive quirk (verified)**: with app firmware running - and nothing attached, USB-SJ drops ~4 s after boot (device-side disconnect, - then half-dead `-71` setup failures until reset). Attach a client inside - the window — or once it survives the window it stays up. Recovery is - UART-side: `esptool.py --after hard_reset read_mac` on the CP2102 tty. -- **S3 batch-automation caveats**: this unit's cpu1 debug logic can fail - examination (`OCD_ID = 00000000`) — `-c 'set ESP_ONLYCPU 1'` degrades to - cpu0-only; xtensa-gdb batch `continue`/`interrupt` is async-flaky — for - scripted state reads, halt via telnet :4444 first, then attach gdb to the - stopped target (verified). Interactive sessions are unaffected. +Flashing works in ANY PHY state: the rig flashes via the boards' CP2102N UART +bridges (hence `tinyusb.json` esptool uids are CP210x serials, not MACs). The +UART side is also the remote reset: `esptool.py --after hard_reset read_mac`. + +### P4 (Function-EV) notes + +- The board has **no USB-SJ connector** — GPIO24 (D−, white) / GPIO25 (D+, + green) / GND are broken out from header J1 to a rig hub port. Swapped + D+/D− enumerates as `new low-speed USB device` + error -71; correct shows + `new full-speed`. + +### S3 (DevKitM) notes + +- Debugging windows: non-USB firmware (`board_test` — attach/halt/symbol + resolution verified; `usb_new_phy` is absent from the ELF when + `CFG_TUD/TUH_ENABLED` are 0), bootloader/ROM (always stable), or external + JTAG (TODO). +- **Keep-alive quirk (verified)**: with app firmware running and nothing + attached, USB-SJ drops ~4 s after boot (device-side disconnect, then + half-dead `-71` setup failures until reset). Attach a client inside the + window — once it survives the window it stays up. Recover via the UART + reset above. - PHY mux reference: `RTC_CNTL_RTC_USB_CONF_REG` (0x60008120) bits - `SW_HW_USB_PHY_SEL`/`SW_USB_PHY_SEL` (TRM 10.56) — 0 = eFuse/hardware - control (the default we measured). `esptool.py read_mem/write_mem` peeks - and pokes registers over plain UART with the chip in download mode. -- P4 Function-EV has **no USB-SJ connector** — GPIO24 (D−, white) / GPIO25 - (D+, green) / GND are broken out from header J1 to a rig hub port (wired - 2026-07-23). Miswired D+/D− shows as `new low-speed USB device` + error - -71; correct shows `new full-speed`. -- Flashing always works regardless of PHY state: the rig flashes via the - CP2102N UART bridges (that's why `tinyusb.json` esptool uids are CP210x - serials, not MACs). + `SW_HW_USB_PHY_SEL`/`SW_USB_PHY_SEL` (TRM 10.56); 0 = eFuse/hardware + control (default). `esptool.py read_mem/write_mem` peeks and pokes + registers over plain UART with the chip in download mode. ## Attach ```bash . $HOME/code/esp-idf/export.sh # openocd-esp32, riscv32-/xtensa-esp32s3-elf-gdb, esptool openocd -c 'set ESP_RTOS FreeRTOS' -f board/esp32p4-builtin.cfg \ - -c 'adapter serial 60:55:F9:F9:87:15' & # P4; S3: board/esp32s3-builtin.cfg + its MAC + -c 'adapter serial ' & # S3: board/esp32s3-builtin.cfg riscv32-esp-elf-gdb -batch -ex 'target extended-remote :3333' \ -ex 'tbreak tud_task_ext' -ex continue -ex bt -ex 'info threads' -ex detach ``` -- `adapter serial` = the chip MAC **with colons** (`lsusb -v -d 303a:1001`, - or `/dev/serial/by-id/usb-Espressif_USB_JTAG_serial_debug_unit_-if00`). -- `set ESP_RTOS FreeRTOS` must precede the board cfg: with it, `info - threads` lists every task with name/state/CPU (verified: usbd Running - @CPU0, IDLE1 @CPU1, blinky, io, ipc0/1); without it, one bare - "Remote target". +- `adapter serial` = the chip MAC **with colons** (`lsusb -v -d 303a:1001`, or + `/dev/serial/by-id/usb-Espressif_USB_JTAG_serial_debug_unit_-if00`). +- `set ESP_RTOS FreeRTOS` must precede the board cfg: with it, `info threads` + lists every task with name/state/CPU (verified: usbd Running @CPU0, IDLE1 + @CPU1, ...); without it, one bare "Remote target". - The ELF: `idf.py -B -DBOARD= build` under the example (CLAUDE.md Espressif notes) — symbolized app backtraces verified (`tud_task_ext` ← `usb_device_task` ← `vPortTaskWrapper`). -- **Attach may reset the target** — after a tbreak-continue the FreeRTOS - tick read 2 (boot-fresh) on a minutes-old session. Until pinned down, do - NOT trust built-in-JTAG attach for post-mortem autopsy of a wedged board - (`target-debug`'s attach-and-halt-only rule); capture state via console - or treat the reset as part of the reproduce cycle. -- Halting still stops USB service: expect the host to drop the DUT during - long halts; after detach the device may need a reset to re-enumerate - (UART-side `esptool.py read_mac` is a handy remote reset). +- **Attach may reset the target** (a boot-fresh FreeRTOS tick observed on a + minutes-old session). Until pinned down, do NOT trust built-in-JTAG attach + for post-mortem autopsy of a wedged board (`target-debug`'s + attach-and-halt-only rule); capture state via console or treat the reset + as part of the reproduce cycle. +- Halting still stops USB service: the host may drop the DUT during long + halts; after detach the device may need the UART reset to re-enumerate. + +## Scripted-session gotchas (verified) + +- xtensa-gdb batch `continue`/`interrupt` is async-flaky — for scripted + state reads, halt via openocd telnet :4444 first, then attach gdb to the + stopped target. Interactive sessions are unaffected. +- cpu1 debug-logic examination can fail (`OCD_ID = 00000000`) — + `-c 'set ESP_ONLYCPU 1'` degrades to cpu0-only debugging. +- ROM-frame backtraces (`0x4004xxxx` on S3, `0x4fc0xxxx` on P4, all `??`) + mean the core idles in ROM — break in app code (`tbreak tud_task_ext`) + for symbolized frames. ## Technique mapping (vs the `target-debug` arsenal) @@ -86,11 +95,9 @@ riscv32-esp-elf-gdb -batch -ex 'target extended-remote :3333' \ ## Rig deltas -- Locks/flash per `hil` skill; espressif flasher = esptool over the CP210x - UART tty (works with any PHY state, any firmware). -- One client per USB-SJ: openocd and any terminal on the USB-SJ CDC side +- Locks per `hil` skill; reflash-pristine before release applies unchanged. +- One client per USB-SJ: openocd and a terminal on the USB-SJ CDC side conflict the same way J-Link clients do. -- Reflash-pristine before release applies unchanged. ## TODO — external JTAG (needs hardware) diff --git a/.claude/skills/target-debug/SKILL.md b/.claude/skills/target-debug/SKILL.md index ff3f45d4e..ab754dc09 100644 --- a/.claude/skills/target-debug/SKILL.md +++ b/.claude/skills/target-debug/SKILL.md @@ -252,9 +252,10 @@ reads don't halt the target. ## GDB — state autopsy and watchpoints Connect/load recipes per probe family (J-Link, OpenOCD for ST-Link / -CMSIS-DAP / WCH-Link) are in CLAUDE.md "GDB Debugging". Scripted sessions: JLinkGDBServer `-singlerun` (exits with the -connection) — back-to-back relaunches race the probe handle and hang. Release builds keep DWARF (`MinSizeRel`), so `p`/struct access works -on HIL firmware. +CMSIS-DAP / WCH-Link) are in CLAUDE.md "GDB Debugging"; script sessions with +JLinkGDBServer `-singlerun` — the server exits with the connection, and +back-to-back relaunches race the probe handle and hang. Release builds keep +DWARF (`MinSizeRel`), so `p`/struct access works on HIL firmware. **Autopsy of a wedged board: attach and halt ONLY** — skip CLAUDE.md's `monitor reset halt` + `load` (those are for fresh starts; a reset destroys From 154999dbac95484f3f9a02cbf7a7277d89e1e0e9 Mon Sep 17 00:00:00 2001 From: hathach Date: Fri, 24 Jul 2026 15:14:02 +0700 Subject: [PATCH 10/11] docs: address Codex/Copilot review on #3786 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - target-debug: fault frame lives on PSP when EXC_RETURN bit2 set (FreeRTOS tasks) — decode LR before choosing $msp/$psp (Codex; valid, our verify happened to fault on MSP) - esp-target-debug: show the Xtensa gdb for S3 in the attach recipe; clarify adapter serial = USB-SJ iSerial (colon MAC, hardware-verified) vs the CP2102N flasher uids in tinyusb.json (Codex; second half of the finding had the identifiers inverted) - esp plan: align serial form with the verified command; record the real console-gate outcome (UART0, USB-SJ console untested) (Copilot) - target-debug plan: Task 4 now consistently $JB/ARMv7-M matching the executed JLinkExe path (Copilot) - drop IDE-local .idea files swept in by the rename commit (Copilot) --- .claude/skills/esp-target-debug/SKILL.md | 8 ++++++-- .claude/skills/target-debug/SKILL.md | 2 ++ .idea/codeStyles/Project.xml | 10 ---------- .idea/codeStyles/codeStyleConfig.xml | 5 ----- .idea/improve-debug-skill-agent.iml | 2 -- .idea/inspectionProfiles/Project_Default.xml | 17 ----------------- .idea/misc.xml | 5 ++++- .idea/modules.xml | 8 -------- .idea/vcs.xml | 1 - .../plans/2026-07-23-esp-target-debug-skill.md | 4 ++-- ...2026-07-23-target-debug-skill-enhancement.md | 9 +++++---- 11 files changed, 19 insertions(+), 52 deletions(-) delete mode 100644 .idea/codeStyles/Project.xml delete mode 100644 .idea/codeStyles/codeStyleConfig.xml delete mode 100644 .idea/improve-debug-skill-agent.iml delete mode 100644 .idea/inspectionProfiles/Project_Default.xml delete mode 100644 .idea/modules.xml diff --git a/.claude/skills/esp-target-debug/SKILL.md b/.claude/skills/esp-target-debug/SKILL.md index c6d76c801..1af9fcf7f 100644 --- a/.claude/skills/esp-target-debug/SKILL.md +++ b/.claude/skills/esp-target-debug/SKILL.md @@ -53,10 +53,14 @@ openocd -c 'set ESP_RTOS FreeRTOS' -f board/esp32p4-builtin.cfg \ -c 'adapter serial ' & # S3: board/esp32s3-builtin.cfg riscv32-esp-elf-gdb -batch -ex 'target extended-remote :3333' \ -ex 'tbreak tud_task_ext' -ex continue -ex bt -ex 'info threads' -ex detach +# S3 is Xtensa: use xtensa-esp32s3-elf-gdb with the same arguments ``` -- `adapter serial` = the chip MAC **with colons** (`lsusb -v -d 303a:1001`, or - `/dev/serial/by-id/usb-Espressif_USB_JTAG_serial_debug_unit_-if00`). +- `adapter serial` = the chip MAC **with colons** — the USB-SJ device's + iSerial exactly as `lsusb -v -d 303a:1001` or + `/dev/serial/by-id/usb-Espressif_USB_JTAG_serial_debug_unit_-if00` + prints it. (The `tinyusb.json` esptool uids are the CP2102N *flasher* + serials — a different port; never pass those to openocd.) - `set ESP_RTOS FreeRTOS` must precede the board cfg: with it, `info threads` lists every task with name/state/CPU (verified: usbd Running @CPU0, IDLE1 @CPU1, ...); without it, one bare "Remote target". diff --git a/.claude/skills/target-debug/SKILL.md b/.claude/skills/target-debug/SKILL.md index ab754dc09..1dc55440f 100644 --- a/.claude/skills/target-debug/SKILL.md +++ b/.claude/skills/target-debug/SKILL.md @@ -172,6 +172,8 @@ p/x *(unsigned*)0xE000ED28 # CFSR — low byte MemManage, byte1 BusFault, top p/x *(unsigned*)0xE000ED2C # HFSR — bit30 FORCED = an escalated lower-priority fault p/x *(unsigned*)0xE000ED38 # BFAR — faulting address (valid if CFSR bit15 BFARVALID) x/8wx $msp # stacked frame: r0 r1 r2 r3 r12 lr pc xpsr — pc = culprit +# frame is on PSP when EXC_RETURN bit2 is set (LR = 0xFFFFFFFD — FreeRTOS +# tasks run on PSP): then x/8wx $psp instead. LR 0xFFFFFFF1/E9 = MSP. ``` `addr2line -e ` names the line (verified: CFSR 0x8200, diff --git a/.idea/codeStyles/Project.xml b/.idea/codeStyles/Project.xml deleted file mode 100644 index 35c56fc87..000000000 --- a/.idea/codeStyles/Project.xml +++ /dev/null @@ -1,10 +0,0 @@ - - - - - - - - \ No newline at end of file diff --git a/.idea/codeStyles/codeStyleConfig.xml b/.idea/codeStyles/codeStyleConfig.xml deleted file mode 100644 index 79ee123c2..000000000 --- a/.idea/codeStyles/codeStyleConfig.xml +++ /dev/null @@ -1,5 +0,0 @@ - - - - \ No newline at end of file diff --git a/.idea/improve-debug-skill-agent.iml b/.idea/improve-debug-skill-agent.iml deleted file mode 100644 index 4c9423543..000000000 --- a/.idea/improve-debug-skill-agent.iml +++ /dev/null @@ -1,2 +0,0 @@ - - \ No newline at end of file diff --git a/.idea/inspectionProfiles/Project_Default.xml b/.idea/inspectionProfiles/Project_Default.xml deleted file mode 100644 index 6b55c5c28..000000000 --- a/.idea/inspectionProfiles/Project_Default.xml +++ /dev/null @@ -1,17 +0,0 @@ - - - - \ No newline at end of file diff --git a/.idea/misc.xml b/.idea/misc.xml index 7ed4f1ab9..f0fcd6912 100644 --- a/.idea/misc.xml +++ b/.idea/misc.xml @@ -1,5 +1,8 @@ + + @@ -8,4 +11,4 @@ - + \ No newline at end of file diff --git a/.idea/modules.xml b/.idea/modules.xml deleted file mode 100644 index e97c64966..000000000 --- a/.idea/modules.xml +++ /dev/null @@ -1,8 +0,0 @@ - - - - - - - - \ No newline at end of file diff --git a/.idea/vcs.xml b/.idea/vcs.xml index d44b5516f..94a25f7f4 100644 --- a/.idea/vcs.xml +++ b/.idea/vcs.xml @@ -2,6 +2,5 @@ - \ No newline at end of file diff --git a/docs/superpowers/plans/2026-07-23-esp-target-debug-skill.md b/docs/superpowers/plans/2026-07-23-esp-target-debug-skill.md index 8b470eb74..d08902111 100644 --- a/docs/superpowers/plans/2026-07-23-esp-target-debug-skill.md +++ b/docs/superpowers/plans/2026-07-23-esp-target-debug-skill.md @@ -41,7 +41,7 @@ lsusb | grep -i cafe # TinyUSB VID on the DUT port - [x] **Step 3: Attach openocd over USB-SJ while the device runs** ```bash -openocd -f board/esp32p4-builtin.cfg -c 'adapter serial 6055F9F98715' & # gdb :3333 +openocd -f board/esp32p4-builtin.cfg -c 'adapter serial 60:55:F9:F9:87:15' & # gdb :3333 — USB-SJ iSerial = MAC with colons riscv32-esp-elf-gdb -batch -ex 'target extended-remote :3333' -ex 'monitor halt' \ -ex bt -ex 'monitor resume' ``` @@ -53,7 +53,7 @@ Expected: backtrace with symbols; after resume the CDC device still answers (re- - [x] **Step 1: Breakpoint/watchpoint budget** — RISC-V trigger count: in gdb `monitor riscv info` or set watchpoints until rejection; verify a hardware watchpoint on a TinyUSB variable (e.g. `watch -l` on a usbd counter) reports and hits. - [x] **Step 2: FreeRTOS threads** — `info threads` after halt; expect ESP-IDF tasks incl. the USB task; note whether it works at attach or needs run→stop (mirror the ARM finding). -- [x] **Step 3: Console during traffic** — capture the USB-SJ console tty (the 303a:1001 CDC function) for a few seconds while DUT traffic runs; expect ESP-IDF log lines. Record the /dev node mapping by serial. +- [x] **Step 3: Console during traffic** — OUTCOME: stock builds route the console to UART0 (the CP2102 flasher tty — boot log captured there); the USB-SJ CDC carries no log without sdkconfig `ESP_CONSOLE_USB_SERIAL_JTAG`, which stays (untested) in the skill. - [x] **Step 4: Reflash pristine, release P4 lock.** Evidence appended to `/tmp/esp_evidence.txt`. ### Task 3: P4 apptrace spike — GATED (spec gate 5) diff --git a/docs/superpowers/plans/2026-07-23-target-debug-skill-enhancement.md b/docs/superpowers/plans/2026-07-23-target-debug-skill-enhancement.md index edace21c7..36a3144c2 100644 --- a/docs/superpowers/plans/2026-07-23-target-debug-skill-enhancement.md +++ b/docs/superpowers/plans/2026-07-23-target-debug-skill-enhancement.md @@ -313,8 +313,8 @@ board back; RISC-V ports have no DEMCR — use a breakpoint on the trap handler. Create the fault build (NOT committed): ```bash -python3 test/hil/board_lock.py hold $OB --reason "skill-enhance verify: vector catch" -cd examples/device/cdc_msc +python3 test/hil/board_lock.py hold $JB --reason "skill-enhance verify: vector catch" +cd examples/device/cdc_msc # executed on $JB (stm32f407disco, ARMv7-M) via JLinkExe — see commit evidence # temporary patch — revert after: fault 5 s after boot python3 - <<'EOF' import pathlib @@ -327,12 +327,13 @@ s = s.replace('led_blinking_task();', 'led_blinking_task(); _fault_after_5s();', p.write_text(s) EOF grep -n '_fault_after_5s' src/main.c # expect 3 hits: definition + call + (none in decl block) -cmake -B build-fault -DBOARD=$OB -G Ninja -DCMAKE_BUILD_TYPE=MinSizeRel && cmake --build build-fault +cmake -B build-fault -DBOARD=$JB -G Ninja -DCMAKE_BUILD_TYPE=MinSizeRel && cmake --build build-fault ``` (If `app_led_task`/`board_millis` anchors differ in the current `main.c`, place the same 3-line helper on whatever per-loop task function exists — the fault line `*(volatile uint32_t*)0xCF000000u = 0;` is the payload.) Flash `build-fault`, then: ```bash +# executed variant: DEMCR armed + autopsy via JLinkExe command file on $JB (see commit c1d2d305f evidence); OpenOCD-native form: openocd $OPENOCD_OPTION -c init -c 'cortex_m vector_catch hard_err bus_err' & timeout 60 arm-none-eabi-gdb -batch -ex 'target remote :3333' -ex 'monitor reset run' \ -ex 'shell sleep 8' -ex 'interrupt' \ @@ -343,7 +344,7 @@ Expected: halted in the fault path, CFSR BusFault bits set, **BFAR = 0xCF000000* - [x] **Step 3: Clean up hardware state** -`git checkout -- src/main.c`, delete `build-fault/`, clear DEMCR bits (`set *(unsigned*)0xE000EDFC &= ~0x7F1` via a final gdb attach or power-cycle note), reflash pristine cdc_msc, `board_lock.py release $OB`. +`git checkout -- src/main.c`, delete `build-fault/`, clear DEMCR bits (`set *(unsigned*)0xE000EDFC &= ~0x7F1` via a final gdb attach or power-cycle note), reflash pristine cdc_msc, `board_lock.py release $JB`. - [x] **Step 4: Commit** From d9268d1e19fe7073f49af411fd9c6fa256099d3a Mon Sep 17 00:00:00 2001 From: hathach Date: Fri, 24 Jul 2026 15:28:44 +0700 Subject: [PATCH 11/11] docs: drop remaining stray .idea modifications (code-review follow-up) misc.xml CMakePythonSetting + vcs.xml Pico-PIO-USB mapping churn were IDE-local and unrelated; .idea now matches master exactly. --- .idea/misc.xml | 5 +---- .idea/vcs.xml | 1 + 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/.idea/misc.xml b/.idea/misc.xml index f0fcd6912..7ed4f1ab9 100644 --- a/.idea/misc.xml +++ b/.idea/misc.xml @@ -1,8 +1,5 @@ - - @@ -11,4 +8,4 @@ - \ No newline at end of file + diff --git a/.idea/vcs.xml b/.idea/vcs.xml index 94a25f7f4..d44b5516f 100644 --- a/.idea/vcs.xml +++ b/.idea/vcs.xml @@ -2,5 +2,6 @@ + \ No newline at end of file