diff options
Diffstat (limited to 'PLAN.md')
| -rw-r--r-- | PLAN.md | 188 |
1 files changed, 188 insertions, 0 deletions
@@ -0,0 +1,188 @@ +# wireframe - Packet Analyzer / Network TUI + +## Overview +Terminal packet capture and analysis tool. Primary goal: learn the C++ +memory model (byte layout, alignment, endianness, `std::span` over +unowned buffers) via a real-world capture pipeline. + +## Stack +- Language: C++ (first of two C++ projects - build this one first) +- Capture: libpcap, or raw `AF_PACKET` with an mmap'd ring buffer to skip + libpcap's copies +- Parsing: hand-rolled L2-L4 decoders over `std::span`, L7 dissectors as + a small interface/vtable so protocols can be added incrementally +- Optional: `aya`-style in-kernel filtering isn't available in C++; if + eBPF filtering is wanted later, that's a separate learning detour +- UI: TUI (library TBD - ftxui or notcurses are the usual C++ options) +- Output format: pcapng (not pcap) so interface metadata survives and + files stay Wireshark-compatible + +## Architecture sketch +- Capture thread (owns the pcap/AF_PACKET handle) -> bounded channel -> + render/analysis thread. A traffic spike should drop packets, not + block the UI. +- Drop privileges immediately after opening the capture handle; use + `CAP_NET_RAW` via file capabilities instead of running as root. + +## Build order +1. [done] Raw capture -> hex dump to stdout +2. [done] Ethernet/IP/TCP/UDP decoders + live packet list in TUI (-t) +3. [done] pcapng read/write +4. [done] Bounded channel + drop-on-backpressure between capture and render +5. [in progress] L7 dissector interface, add protocols incrementally -- + interface + DNS + HTTP + TLS SNI done (wireframe/l7/); more + protocols can still be added incrementally, by design +6. [done] Filtering (-f <expr>, libpcap's own BPF compiler - see Decisions) + +## Open questions +None currently open. + +## Decisions +- TUI library: FTXUI (v7.0.3, fetched via CMake FetchContent). Chosen + over notcurses for pure-C++ portability (no C build-system/dependency + chain to fight on Gentoo/low-spec machines) and genuine native + Windows console support, which notcurses lacks - both matter given + this needs to work everywhere. +- GUI added as a secondary frontend - TUI stays primary (explicit + user direction). Dear ImGui + SDL3 (v1.92.9b / release-3.4.14, both + FetchContent, same approach as FTXUI/doctest - no system-package + dependency, builds the same way everywhere). SDL3 over SDL2: this + ecosystem already carries sdl2-compat as an SDL3-backed shim, so + SDL3 is the live line, not legacy. SDL_Renderer backend, not raw + OpenGL3 - avoids needing a separate GL function loader as another + dependency, which matters more here than raw rendering performance + does. src/gui_main.cpp; parity with the CLI/TUI is structural, not + incidental - all three go through the same wireframe::CaptureSession + (wireframe/capture_session.hpp) for device-open/datalink-validate/ + filter/pcapng/signal-handler setup, so the GUI can't silently skip a + step (e.g. the DLT_RAW check) the way two hand-copied setups would + eventually drift. +- Tests: doctest (v2.5.3, FetchContent), tests/ mirrors include/wireframe/. + Every module gets unit tests as it's built, not backfilled later -- + `cmake --build build && ./build/wireframe_tests` (or `ctest`) should + stay green at every commit. +- Filtering: libpcap's own pcap_compile()/pcap_setfilter() (tcpdump + syntax, kernel-level via BPF), not a hand-rolled parser - the + parser/compiler already exists, is correct, and reimplementing it has + no bearing on this project's actual goal (the C++ memory model). + wireframe/filter.hpp wraps compilation; testable without root via + pcap_open_dead(). Verified live: -f "tcp port N" and -f icmp each + correctly suppressed non-matching traffic that was actually present. +- pcap_stats(): CaptureSession::stats() surfaces kernel/interface-level + drops (ps_recv/ps_drop/ps_ifdrop), shown in CLI/TUI/GUI whenever + nonzero. Distinct from CaptureQueue::dropped() - verified live that + the two really do measure different things: a short capture showed + ps_recv=12 against only 4 packets actually rendered, i.e. packets the + kernel had already received but that were never dispatched to our + callback before shutdown, with queue-side drops at 0 throughout. +- Fuzzing: libFuzzer harnesses (fuzz/, clang + ASan/UBSan, opt-in via + -DWIREFRAME_ENABLE_FUZZING=ON -DCMAKE_CXX_COMPILER=clang++, separate + build-fuzz/ dir) for every hand-rolled decoder plus the pcapng reader + and the full summarize_packet() pipeline - the highest-value tests + in the repo given the project's actual goal (byte layout/alignment/ + UB on parsers over untrusted bytes), not an afterthought. Found and + fixed a real bug on the first run: Reader::next_packet() allocated a + block's claimed size (an untrusted 32-bit field straight from the + file) before validating it, so a corrupted/hostile pcapng file could + OOM the process. Fixed with a 1 MiB body-size cap (reader.hpp is + explicitly scoped to pair with our own writer, whose packets are + capped at a 65535 snaplen, so this is generous, not tight) and locked + in with both a unit test and a passing re-fuzz of the exact crashing + input. ~23M total fuzz executions across all 8 harnesses this + session, one bug found and fixed, zero remaining crashes. +- HTTP L7 dissector (wireframe/l7/http.hpp): best-effort single-segment + request/status-line parse (+ Host: header for requests), same scope + DNS already has - no TCP stream reassembly, so a message split + across packets is only partially visible. This is the first + registered dissector to actually exercise L7Registry's TCP-payload + path; DNS alone never did, since it only runs over UDP. Verified live + against a real HTTP request/response (curl -> python http.server on + port 80): both directions decoded correctly, including the + dst-port-then-src-port fallback in l7_summarize (request matches on + dst_port=80, response matches on src_port=80). Fuzzed separately + (fuzz_http.cpp, 5.3M runs, no crashes) since the string_view request- + line/header scanning is new hand-rolled logic distinct from anything + fuzz_summarize's binary-format parsers already cover. +- TLS SNI L7 dissector (wireframe/l7/tls.hpp): parses a ClientHello's + record/handshake/extensions structure (nested TLVs, every length + bounds-checked against attacker-influenced fields at every level -- + the most structurally complex hand-rolled parser in the project) to + extract the SNI extension. Answers what HTTP alone increasingly + can't: most web traffic is TLS-encrypted, and the server name is the + one thing still readable in cleartext, in every TLS version, before + encryption starts. Same single-segment scope as DNS/HTTP. Verified + against real, unsolicited internet traffic captured live on wlp1s0 + (not loopback/synthetic) - correctly extracted a genuine SNI from a + real ClientHello. Fuzzed the hardest of any target so far given the + nesting depth: fuzz_tls.cpp, 25.7M runs, no crashes. +- IPv6 extension headers: walk_ipv6_extension_headers() (ipv6.hpp) + walks Hop-by-Hop, Routing, Destination Options, Fragment, and AH to + find the real transport protocol underneath them, so e.g. TCP wrapped + in a Hop-by-Hop options header is decoded instead of silently + stopping. ESP is a deliberate hard stop, not an oversight: its own + next-header field lives in a trailer after the encrypted payload, at + an offset unknowable without decrypting first - reported as + "ESP (encrypted)" rather than guessed at. parse_ipv6() itself stays + an unconditional decode of just the fixed 40-byte header; the walk is + a separate, composable function summarize.hpp calls, so parse_ipv6's + existing tests didn't need to change. Verified end-to-end (a + Hop-by-Hop-wrapped TCP frame decodes through to the TCP layer via + summarize_packet, not just the walker in isolation) and fuzzed + (extended fuzz_ipv6.cpp, 6.3M runs; fuzz_summarize.cpp indirectly + covers it too, 4.3M more) - no crashes. This was the last item on + the known-gaps list; none remain. +- Post-capture search: wireframe/search.hpp's matches_search() is a + display filter, deliberately distinct from -f's capture filter -- + -f decides what's captured (and written to -w); search decides what's + shown, without touching either, same distinction Wireshark draws + between a capture filter and a display filter. CLI: -g <term> (only + suppresses what's printed; -w output is unaffected). TUI: '/' opens + live-filtered search (Enter keeps the filter and returns to + browsing, Esc clears it), verified interactively via a real terminal + (tmux capture-pane) - typing, backspace, both Enter and Esc paths, + and confirmed 'q' still quits correctly afterward. GUI: a search box + next to the capture-info line, using io.WantCaptureKeyboard to route + Esc to "clear the search" while the box has focus vs. "quit the app" + otherwise - verified visually via Xvfb, including the focused/ + unfocused Esc distinction actually working both ways. + One real methodology lesson from building this: an initial pty-based + interactive test of the TUI (raw-byte capture, regex-matched against + unparsed ANSI escape sequences) appeared to show a redraw bug -- + typing "abc" only ever displayed "a". Chasing it added an unnecessary + PostEvent "fix" before re-verifying under tmux (which properly + resolves escape sequences via a real terminal emulator) showed the + original code was correct all along; the first test method just + wasn't reliable enough to trust. The PostEvent change was reverted -- + correct code, not narrowly-passing code, was the actual goal. +- Replay mode (-r <file>): reads a previously-saved pcapng file back + through the exact same CaptureQueue/render/search pipeline as a live + capture - the render/consumer side only ever talks to a + CaptureQueue, so it can't tell whether packets are arriving from + pcap_loop or being read back from disk. All in CaptureSession, so + every frontend gets it for free rather than needing a second code + path. Reader gained link_type() (the datalink from the file's IDB, + previously discarded) so replayed packets decode with the correct + DLT_EN10MB/DLT_RAW branch instead of an assumption; CaptureQueue + gained a blocking push() alongside the existing drop-on-full + try_push(), because a live capture thread can't be allowed to stall + but a file has no real-time pressure forcing a drop - dropping from + what's supposed to be a faithful replay of a fixed record would + defeat the point of replaying it. -f (capture filter) is rejected + outright when combined with -r, with an actionable error pointing at + -g, rather than silently ignored. + Verified live end-to-end, not just via unit tests: captured real + traffic with -w on both DLT_EN10MB (lo) and DLT_RAW (tailscale0), + replayed each file with -r with no root/live device needed, and the + output matched the original capture exactly, including L7 dissection + (DNS) surviving the round-trip. The replay thread finishing (not + killed via request_stop()) means the file is exhausted, not that the + user wants to quit - CaptureSession::stop_requested() distinguishes + the two, and TUI/GUI both leave the window open on natural + end-of-file (the point of replaying into an interactive frontend is + browsing/searching afterward, not watching it flash by), closing only + on an explicit 'q'/Esc/window-close or an external signal. Verified + interactively in both: tmux capture-pane confirmed the TUI stays open + with "[replay finished]" shown, search still works against the + now-static list, and 'q' closes it; Xvfb confirmed the same for the + GUI, including a live process check across a multi-second wait to + rule out a delayed auto-close. |