| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
The prerequisite for the plugin ecosystem: any external process - any
language - that can reach the control socket can now tag a history
entry with a short string marker and an opaque JSON data blob, stored
in a new entry_tags table rather than requiring the plugin stay
connected for a later live round-trip. A plugin does its analysis
once; the data it attaches is what a human reviewing the entry later
actually sees.
internal/ipc: new "tag_entry" request (id, tag_plugin, tag, tag_data)
and EntryDetail.Tags (the full record for one entry, populated by
"get"). internal/store: entry_tags table, EntryTag struct, AddEntryTag/
ListEntryTags, a comma-joined Tags aggregate added to List/Search via
a correlated subquery (cheap enough per row that showing a tag badge
in the history list needs no N+1 query), and a new tag: search filter
alongside the existing status:/source:/flagged:.
TUI: a Tags column in the history table (sortable via o/O, the eighth
sort column), T from detail view opens a tag list (mirroring the
WebSocket-messages view's table-then-detail-viewport pattern), enter
on one shows its data - JSON-colorized via the existing jsoncolor.go
if it parses as JSON, sanitized plain text otherwise. Also fixed a
pre-existing gap while touching this: the WebSocket-messages view
never got mouse wheel support when it shipped; wired both it and the
new tags view up together.
PLUGINS.md documents the wire protocol for non-Go plugin authors -
connection model (subscribe vs request/response), the handful of
request types a plugin actually needs, and the trust boundary (the
socket has no auth beyond OS file permissions, same as the TUI's own
access). PLAN.md records the architecture decision (external process
over an embedded scripting language - mirrors the daemon/TUI split
already in place, no interpreter to sandbox, any language) and groups
~20 researched Burp extensions/Pro features into what Phase 1 already
covers (Autorize, Param Miner, Backslash Powered Scanner, Retire.js -
all just subscribe+repeat+tag, no new capability needed), what needs a
second protocol addition (JWT Editor, SAML Raider - live RPC to a
specific connected plugin for interactive actions like re-signing),
and what deserves its own separate project rather than a plugin
(active vulnerability scanning, Collaborator/OAST, a crawler).
Verified live end to end against a real daemon: a throwaway program
simulating a real plugin tagged a captured entry with structured JWT
data over the actual wire protocol; confirmed the tag badge, tag:
search filter, and full tag record all round-tripped correctly through
List/Search/Get. Confirmed in the TUI itself (tmux, real keystrokes):
the Tags column renders, T opens the tag list, entering it shows the
JSON data with real ANSI-verified syntax highlighting (not just
eyeballed), and tag: search filtering works from the history list.
|
|
The last "known limitation": a ws://wss:// connection stops being
one-shot request/response the instant its 101 Switching Protocols
lands, and forward()'s normal write-response-then-record flow has no
way to represent that. Scoped to HTTP/1.1 client legs (HTTP/2 can't be
hijacked for raw post-response access the way HTTP/1.1 can, and
browsers open a dedicated HTTP/1.1 connection for WebSocket regardless
of the surrounding page's protocol, so this isn't a real-world gap).
internal/proxy/websocket.go decodes each RFC 6455 frame's opcode and
payload for capture while relaying the exact same raw bytes it read
unmodified - this is capture, not tampering, matching the rest of the
codebase's raw-bytes-as-source-of-truth stance. One row per frame, not
per reassembled message (fragmentation is rare in real-world
WebSocket traffic; not worth buffering an unbounded number of pending
fragments to handle it). forward() branches on a matching 101 into
handleWebSocketUpgrade, which hijacks the client connection, relays
the handshake response raw, records the upgrade request/response to
history normally, then relays frames bidirectionally into a new
ws_messages table - reachable from the TUI's detail view via `w`.
Found and fixed two real bugs by actually driving a WebSocket
connection through a running daemon, not by reading the code:
stripHopByHop was deleting Connection/Upgrade from every outgoing
request (correct for an ordinary request per RFC 7230, catastrophic
for one asking to upgrade - every WebSocket attempt silently became a
426); and the relay tore the whole connection down the instant either
side saw a close frame, before the peer's own close-frame reply could
be relayed back, producing an abrupt EOF instead of a clean close.
Verified live end to end on both paths a real client uses: ws://
(plain HTTP forward-proxying) against a Python websockets echo
server, and wss:// (CONNECT-tunneled, TLS-intercepted) against the
same server behind TLS - text, binary, and extended-length frames,
plus a full close handshake with both directions' close frames
present, confirmed via the actual bytes captured in ws_messages.
|
|
Adds internal/clientcert: a cert/key pair matched to hosts by the same
substring-or-regex pattern model as scope.Rule, so mitmux can present
a client certificate on an upstream TLS handshake that requires one -
the previous behavior was a hard handshake failure with no way to
authenticate. Wired into both places mitmux dials an https:// upstream
over its own TLS client connection: proxy.go's handleConnect (live
proxied traffic) and repeat.go's dialForRepeat (Repeater/Intruder
resends), both through a new Server.clientCertFor(host) helper.
Stored in a new client_certs table, mirroring the existing scope_rules
persistence pattern. The TUI (`t` from history) is add-only like
scope, for the same reason: delete and re-add covers changing
anything, and it's a rarely-touched, low-cardinality list. The add
form takes cert/key file paths and reads them once at save time - PEM
content, not the path, is what's stored and later presented, so a
cert keeps working even if the original file moves afterward.
Verified live against a real mutual-TLS-requiring origin server:
without a matching cert the handshake correctly fails; with one
configured, the origin receives it and the request succeeds; toggling
it off reproduces the failure, confirming the enable/disable path
works end to end.
|
|
Generalizes Intrude beyond Sniper to all four of Burp's attack modes
(proxy.AttackMode). Sniper and Battering ram only ever need one shared
payload set; Pitchfork and Cluster bomb are inherently per-position, so
they take one payload set per §marked§ position instead.
Request-set generation (intrudeValues) is pure and side-effect free,
so the total request count is validated against the existing 1000
cap before anything is dispatched - Cluster bomb's product is checked
incrementally, one payload set at a time, so a pathological product
bails out before ever trying to enumerate it. This also makes the
combinatorics unit-testable without a live target.
IntrudeResultMsg now reports Values (one substitution per marked
position, in order) instead of a single Position/Payload pair, since
three of the four modes touch multiple positions per request.
TUI: `a` cycles the attack mode. Pitchfork/Cluster bomb reuse the
existing single Payloads pane rather than a new multi-widget editor -
sets are separated by a `---` delimiter line, in position order.
Verified live against a real daemon: all four modes produce the
expected substitution values and request counts, and pitchfork
correctly rejects a payload-set count that doesn't match the
template's marked positions.
|
|
Closes the interop loop HAR export opened - traffic can now move both
directions between mitmux and any other HAR-producing tool (browser
DevTools, Burp, Postman), not just out.
cmd/mitmux/har.go: rawRequestFromHAR/rawResponseFromHAR reconstruct
HTTP/1.1 wire bytes from HAR's structured fields - the mirror image of
harEntryFromDetail on the export side. Deliberately tolerant of a HAR
file that didn't come from mitmux at all: lowercase header names,
"HTTP/2" in httpVersion, missing optional fields like postData, a
redirectURL nobody filled in. Content-Encoding and Transfer-Encoding
headers are stripped from the reconstructed response before writing it
- HAR's content.text is already decoded per spec, so re-emitting those
headers would describe framing the body no longer has and break any
client that tried to decode it again - and a Content-Length is computed
if the HAR didn't carry a consistent one. importEntriesFromHAR converts
a whole document, skipping (not failing on) any entry that fails to
convert, same reasoning as export's own skip-and-continue for a
malformed capture.
Imported entries are always request_exact=false/response_exact=false:
reconstructed from structured HAR fields, the same situation an HTTP/2
capture is already in, never claiming to be the literal bytes that were
actually on the wire for the original request.
internal/ipc: ImportEntry (the slim shape the client sends - the daemon
just stores what it's given, all HAR parsing happens client-side) and
an "import" request type; Client.Import returns how many entries were
actually inserted, a per-entry store failure is skipped rather than
aborting the batch. Server-side, imported entries are tagged
source="import" so source:import finds them in search, same as
source:repeater/source:intruder already work.
TUI: 'I' from the history list prompts for a HAR path (same modal
pattern as export, in reverse - reading instead of writing), then
reloads the list and status once the import completes.
Verified live: exported real captured traffic to HAR, cleared history
entirely, imported the same file back and got both entries with
correct content (byte-different after the round trip - headers get
reordered/reformatted - but semantically identical, correctly labeled
"reconstructed" rather than falsely "exact"); hand-built a HAR mimicking
a real Chrome DevTools export (lowercase headers, HTTP/2, a base64-
encoded binary PNG body, several optional fields omitted) and confirmed
it imports cleanly with the binary body decoded correctly (PNG magic
bytes verified byte-for-byte); confirmed a missing file and invalid
JSON both fail with a clear error and no crash, history left untouched.
go build/vet/gofmt/test/mod tidy all clean.
|
|
The proxy captured and stored literally everything with no way to
exclude unrelated traffic - every CDN asset, analytics beacon, and
third-party tracker request on a real engagement pollutes history and
search right alongside the traffic that actually matters.
internal/scope: Rule{Enabled, Pattern, IsRegex} and InScope(rules,
host). A non-regex pattern matches by case-insensitive substring
against the host - "example.com" matches "example.com",
"www.example.com", and "api.example.com" alike, covering "this domain
and its subdomains" without inventing a separate wildcard syntax.
IsRegex mirrors the same toggle match-and-replace rules already use,
for one consistent mental model across both rule types in this tool.
An empty or all-disabled rule set means everything is in scope - the
behavior before scope existed at all, unchanged, so a fresh install or
a user who never opens the scope view keeps recording everything
rather than silently nothing.
Deliberately a recording filter, not access control: out-of-scope
traffic still proxies completely normally, reaching its destination and
the client exactly as before. internal/proxy's forward() already writes
the response to the client before record() ever runs, so the scope
check (new in record()) can only affect whether the exchange gets
stored, never whether it happens. Blocking out-of-scope traffic outright
would be a materially different, much riskier feature - a wrong scope
pattern could silently break the very traffic someone's trying to test,
which is a far worse failure mode than a noisier history. Repeat/Intrude
(recordRaw, a separate function from record()) deliberately don't go
through the scope check at all: a user explicitly resending or fuzzing
a specific request wants to see the result regardless of scope, which
exists to cut passive-capture noise, not second-guess a deliberate
action.
internal/store: new scope_rules table (CREATE TABLE IF NOT EXISTS, no
migration needed - it's a new table, not a new column on an existing
one) plus List/Add/SetEnabled/Delete, mirroring the existing
match-and-replace rules CRUD exactly. internal/ipc: scope_list/
scope_add/scope_delete/scope_toggle request types and matching Client
methods; scope_add validates a regex pattern compiles before persisting,
same reasoning and same fix as the earlier rules_save validation (an
invalid regex should be rejected up front, not silently never match at
apply time with zero feedback).
TUI: 's' from the history list opens scope management, mirroring the
Rules view's own list+form pattern but simpler (add-only, no
edit-in-place - a pattern and a regex toggle don't need a five-field
form, delete-and-re-add covers changing one).
Verified live in tmux against a running daemon: added a substring scope
rule for one host, sent requests to both a matching and a non-matching
host - the non-matching one proxied successfully (client got its 200)
but was never recorded, the matching one was recorded normally;
confirmed a Repeater resend of the excluded host WAS recorded despite
being out of scope; toggled the rule off and confirmed recording
resumed for everything; added and confirmed a regex-mode rule saves and
displays correctly; deleted a rule and confirmed the list returns to
empty ("no rules means everything is recorded").
go build/vet/gofmt/test/mod tidy all clean.
|
|
Store had full CRUD for match-and-replace rules but no way to delete or
prune history at all - it only ever grew, with no way to remove an
accidental capture or start a new engagement clean short of manually
deleting the DB file outside the tool entirely.
internal/store: DeleteEntry(id) removes one history row and its
history_fts search index row in a transaction. ClearHistory() empties
both tables entirely; rules are untouched. internal/ipc: new
"delete_entry" and "clear_history" request types, Client.DeleteEntry/
ClearHistory methods.
TUI: 'x' deletes the selected history entry, 'X' clears the whole
database. Both gated behind a y/n confirmation - a small reusable
confirmPrompt/confirmYes model state, checked first in the history
list's key handling, so any key other than y/Y safely cancels rather
than falling through to whatever that key normally does elsewhere (this
also means ctrl+c during a pending confirmation cancels the prompt
rather than quitting - a deliberate fail-safe, not an oversight: quick
to dismiss, and a second ctrl+c then quits normally).
'X' is explicitly NOT scoped to an active search filter - it always
clears the true total (read from daemon status, not len(m.entries),
which would understate the count under a filter and make the
confirmation prompt itself misleading about what's about to happen).
Verified live in tmux against a running daemon with real captured
entries: 'x' shows "delete #N? y/n", 'n' cancels with the entry
untouched, 'y' deletes it and the list/count both refresh correctly;
'X' shows "clear all N history entries (not just this view)? y/n" with
the true count, 'y' empties the database (confirmed via direct SQLite
query: both history and history_fts at 0 rows afterward) and the TUI
correctly shows "history (0)" / "0 requests"; 'x'/'X' on an empty list
correctly no-op without crashing.
go build/vet/gofmt/test/mod tidy all clean.
|
|
internal/proxy/tee.go's teeConn silently drops bytes past
maxCaptureBytes (10 MiB) but callers unconditionally marked the result
"exact" anyway. Confirmed live: proxying a 15 MiB response worked
correctly end-to-end (the client got the full, real 15 MiB - proxying
itself is unbounded, only storage is capped), but the stored history
entry was exactly 10485760 bytes with response_exact=1 still set. For a
tool whose core value proposition is "raw bytes are the source of
truth," a silently truncated capture presented as complete could hide
the very evidence a smuggling or parser-differential investigation is
looking for in the tail of a large body - and give false confidence
that it isn't there.
teeConn.Take() now returns (data, truncated) instead of just data;
truncated is true whenever a Read had to drop bytes because the buffer
was already at cap. Every caller (proxy.go's forward() on both the
request and response side, repeat.go's sendRaw for Repeater/Intruder)
now folds truncated into exact - a truncated capture is never marked
exact - and additionally threads a distinct RequestTruncated/
ResponseTruncated bool through store.Entry, ipc.EntryDetail, and the
TUI, since "truncated" and "reconstructed" (HTTP/2, which never had
wire-exact bytes to begin with) are different situations worth telling
apart: a truncated capture is still real wire bytes, just incomplete,
not a synthesized reconstruction. The detail/repeater views now show
"truncated (hit capture size limit)" specifically rather than lumping
it in with "reconstructed", which would have implied more
transformation happened than actually did.
Schema: history gains request_truncated/response_truncated columns via
the same ALTER-TABLE-and-ignore-duplicate-column pattern already used
for source/flagged, so existing databases upgrade in place.
Verified live: a target server returning a 15 MiB body (over the 10 MiB
cap) proxied through cleanly - full body reached the client - while the
stored entry shows length=10485760, response_exact=0,
response_truncated=1 (previously would have shown response_exact=1);
the TUI's Detail view correctly displays "Response (10485760 bytes,
truncated (hit capture size limit))" instead of "exact".
go build/vet/gofmt/test/mod tidy all clean.
|
|
internal/rules/rules.go's apply() treats a regex that fails to compile
exactly the same as "no match" - it silently returns the text unchanged
with no error surfaced anywhere in the call chain. rules_save had no
validation before persisting, so a rule with a typo'd regex would save
successfully, show as Enabled in the UI, and simply never fire on any
traffic - no indication anything was wrong.
internal/ipc/server.go's "rules_save" handler now compiles r.Match with
regexp.Compile before persisting when IsRegex is set, rejecting with a
clear "invalid regex: ..." error otherwise. No TUI changes needed: the
existing ruleWriteDoneMsg error path already surfaces any saveRule
error inline via the status line and - since it only clears ruleForm on
success - keeps the form open with the user's draft intact so they can
fix the pattern without losing their edits. That path already existed
for other error classes (DB errors); this just adds a new one flowing
through it.
Verified live over the real protocol (raw JSON on the control socket):
an unclosed-bracket regex is rejected with the expected error and never
reaches the rules table; a valid regex rule still saves and returns
normally.
go build/vet/gofmt/test/mod tidy all clean.
|
|
Payload processing: an optional case rule (upper/lower) and an optional
encode rule (URL/Base64/Hex/HTML) applied to every payload line before
it's substituted into the request, cycled with 'c'/'e'. Case always
runs before encode - folding an already-encoded value would corrupt it
(e.g. uppercasing Base64 padding). Applied entirely client-side in
startIntrude() (payload_rules.go): a pure string transform with no
proxy-side state, so it needs no protocol changes and reuses the
Decoder's own urlEncodeAll.
Grep-match/grep-extract: two optional Go regexps, edited with 'm'/'v'
using the same modal edit-buffer pattern as the history list's '/'
search (enter validates-and-commits, esc reverts to the last-confirmed
pattern, an unparseable regexp is rejected with an error rather than
silently accepted). Evaluated server-side, in internal/ipc/server.go's
"intrude" handler, against each result's actual entry.ResponseRaw -
that's where the real response bytes already are, and it's how Burp's
own grep options work (matched against the real response, not a
client-refetched copy). Grep-match flags a result (new Match column);
grep-extract captures the first submatch, or the whole match if the
pattern has no capturing group (new Extract column). Both patterns are
compiled once before the attack starts and apply for that run only, not
retroactively if changed mid-attack.
All four new keys (c/e/m/v) are gated to normal mode, checked in the
view's outer key switch before ever reaching the template/payloads
vi-textareas - otherwise they'd be either untypeable letters or steal
keystrokes mid-edit. Same discipline as the Repeater tab keys.
internal/ipc: Request gained GrepMatch/GrepExtract string fields (for
"intrude"), IntrudeResultMsg gained GrepMatch bool/GrepExtract string,
and the client Intrude() helper takes the two pattern strings as new
trailing parameters.
Verified live in tmux against a running daemon and real httpbin.org
traffic: built a template with a §marked§ query param, payloads 1/2/3,
grep-match `"id": "2"` and grep-extract `"id": "([0-9]+)"`, ran the
attack and confirmed the Match column flagged only the payload=2 row
and Extract correctly pulled 1/2/3 from each response respectively;
cycled case/encode through all states; confirmed an invalid regexp
(`[abc`) is rejected with a visible error and esc correctly reverts to
the last-confirmed pattern instead of committing the invalid one.
(Also confirmed, incidentally: a batch of vi normal-mode two-key
commands like "gg"/"dd" sent as one multi-character tmux send-keys
argument doesn't reliably reach the app as separate keystrokes - a
tmux scripting artifact, not a bug in the vi-mode implementation, which
works correctly when each key is sent as its own event, as any real
keypress would be.)
go build/vet/gofmt/test/mod tidy all clean.
|
|
Last of the "should build soon" items from the Burp/ZAP/Caido gap
research - Burp's row highlighting and Caido's Findings both serve
the same real workflow: mark something interesting mid-engagement,
revisit later. Scoped to a boolean flag (★) rather than full free-text
notes/comments, which would need their own text-input overlay for
comparatively modest extra value over a simple marker - tracked as a
real follow-up in PLAN.md, not dropped silently.
internal/store: history gains a flagged column (migrated in for
existing databases the same way source was) plus Store.SetFlagged and
Summary/Entry.Flagged. Search's structured-filter layer (added last
commit for status:/source:) gains flagged:true/false alongside them -
extractStructured already existed for exactly this kind of "pull it out
before it reaches FTS5" filter. internal/ipc gains a "set_flagged"
request. cmd/mitmux: 'f' toggles the flag on the selected history row
(applied optimistically to local state, persisted async - a drift
between local and server state on failure is an acceptable trade-off
for a marker this low-stakes), shown as a ★ column in the list and in
the detail view's title.
store_test.go covers the flagged: parsing (true/false spellings, and
a "looks like it but isn't" case - flagged:maybe - falling through as
literal search text, matching the existing pattern for status:).
Verified live: toggling 'f' shows the star immediately, flagged:true
correctly filtered to just that entry, and a direct SQLite check
confirmed the flag actually persisted to the database (flagged=1),
not just reflected in local UI state.
|
|
Baseline TUI UX that should exist regardless of feature parity -
flagged directly by the Burp/ZAP/Caido comparison research as missing.
A persistent one-line status bar (proxy address, live request count,
current view) is now appended to every screen. Fetched once at startup
via a new "status" IPC request (internal/store gains Store.Count();
internal/ipc gains StatusMsg plus a daemon-side handler reading
proxy.Server.Addr through ipc.NewServer's new proxyAddr parameter), then
kept approximately live by incrementing locally on each "new" subscribe
push rather than re-querying every time.
'?' opens a full keybinding reference from every view, gated so it
never shadows literal text entry - it's a no-op while typing in the
search box, a rule form field, or (checked via viTextarea.Mode())
insert-mode text in Repeater/Intruder, where a URL query string
literally starting with '?' is completely ordinary input. Any key
dismisses it and returns to whichever view opened it.
Every existing height calculation (table, viewport, textarea panes)
had to shrink by one line to make room for the status bar without
pushing content off-screen - done once via a shared `h := msg.Height-1`
in the WindowSizeMsg handler rather than touching each call site
individually.
Verified live: status bar shows the real proxy address and updates its
count after a live-captured request; '?' renders the full reference
from the history list; dismissing returns to the correct prior view;
and specifically confirmed '?' still types literally (tested typing
"?foo=bar" into a Repeater request body in insert mode) rather than
being swallowed by the help shortcut.
|
|
Implements build-order step 7, the last (optional) item. Scoped to
Sniper only - one payload set, one §-marked position fuzzed at a time,
others held at their base value - since that covers most real Intruder
usage; battering ram / pitchfork / cluster bomb aren't implemented.
Sequential sending, capped at 1000 generated requests as a fixed safety
limit.
internal/proxy: repeat.go's Repeat() is refactored into a shared
sendRaw(..., source) primitive so Intrude can reuse the exact same
raw-byte send/record path with source="intruder" instead of
duplicating it. intrude.go adds ParseMarkers/buildRequest (marker
parsing and payload substitution, covered by intrude_test.go - this is
fiddly byte-splicing logic, worth locking down with real tests rather
than trusting it by inspection) and Intrude(), which walks positions ×
payloads calling sendRaw and streaming each result through a callback.
internal/ipc gains a dedicated streaming "intrude" connection (same
shape as Subscribe, but blocking sends rather than drop-on-slow-
consumer - each result is the attack's actual data, not a
notification). cmd/mitmux gains an Intruder view: editable request
template (ctrl+p inserts a § marker at the cursor - typing § directly
also works, ctrl+p just doesn't require a keyboard layout that can
produce it), editable payload list, and a live results table wired to
the existing detail view (selecting a row and hitting enter opens the
full request/response for that specific attack request).
Verified live against real external traffic: a Sniper attack against
httpbin.org/status/§200§ with payloads 200/404/500 produced exactly the
three corresponding real status codes back (not a canned/local result),
confirmed the three requests landed in history tagged source="intruder"
with the § markers correctly stripped from what was actually sent, and
confirmed opening a result row's full detail from the results table.
This closes out the full build order from PLAN.md (steps 1-7).
|
|
Implements build-order step 6, scoped to headers only for this pass -
see PLAN.md for why bodies are a separate problem (request-body capture
currently depends on streaming straight through, which a body-rewriting
rule would have to interrupt; deciding what "exact" means for a
rule-modified request needs its own pass, not a rushed add-on to this
one).
internal/rules: Rule type and ApplyHeaders, which serializes a Header
map to a raw "Name: value\r\n" block, runs enabled rules' match/replace
over that text, and reparses it - operating on text rather than
per-value substitution is what lets a rule add or remove a header, not
just rewrite one, matching how Burp's header match/replace works.
Invalid rule output (bad regex, unparseable result) leaves the header
map untouched rather than corrupting the request.
internal/store: rules table + CRUD. internal/proxy: forward() fetches
enabled rules for each scope and applies them to outReq.Header /
resp.Header, positioned so the existing capture/history pipeline is
untouched - request_raw keeps showing what the client actually sent and
response_raw what the origin actually sent, while the wire itself
reflects the rules. Deliberate split: match-and-replace transforms
traffic, it doesn't rewrite the audit trail. internal/ipc gains
rules_list/rules_save/rules_delete/rules_toggle. cmd/mitmux gains a
rules view ('m' from history) with add/edit/delete/toggle and a small
form (name, match, replace, scope, regex).
Verified live against real external traffic, not just local echoes:
a request-scope rule rewriting User-Agent, confirmed via httpbin.org's
own header echo that the origin received the rewritten value while curl
sent the real one; a response-scope rule rewriting the Server header,
confirmed the client actually received the rewritten value; disabling a
rule confirmed via a follow-up request that it stops applying; and
throughout, history continued showing the pre-rule original on both
sides, confirming the capture/transform split holds.
|
|
Audited every file in the proxy/store/ipc/TUI stack before starting
step 6, per request. Found and fixed three real bugs in already-shipped
code, all confirmed with live tests (including a race-detector build)
rather than just read:
1. No timeout covered the write-request/read-response phase of an
upstream exchange, in either the main proxy path (roundTripH1/
roundTripH2) or Repeater - only the dial itself was bounded. A
server that accepted the connection and then never finished
responding hung the request forever. Fixed with conn.SetDeadline
after a successful dial in both forward() and Repeat() (new
upstreamTimeout constant, 60s). Verified against a real hung TCP
listener: the daemon returned a clean "i/o timeout" error at exactly
60s instead of hanging.
2. ipc.Client shared one connection/encoder/decoder with no locking.
Bubble Tea dispatches each request as its own goroutine, and
viewList's 'r' key doesn't change mode while its loadDetail call is
in flight - pressing it again (or 'enter' on another row) before the
first response arrives calls Get/List/Repeat concurrently on the same
connection, which can interleave JSON on the wire or hand one call
another's response. Fixed with a mutex serializing round trips.
Stress-tested with rapid overlapping key input against a -race build
of both binaries: no warnings, no corruption.
3. The IPC "subscribe" handler only noticed a disconnected client when
the next broadcast's Encode failed - a subscriber that quit while the
daemon was otherwise idle leaked its goroutine and channel
indefinitely. Fixed by reading the connection in the background too,
so disconnection is detected immediately regardless of traffic.
Also removed a dead, misleading parameter: captureResponse took a *teeConn
it was never actually called with (the exact-capture path is handled
directly in forward()), so the branch using it was unreachable.
Re-verified all five prior steps end-to-end against a fresh build:
plain HTTP, HTTPS H1.1/H2/untrusted-CA-rejection, exact vs reconstructed
capture flags cross-checked directly in SQLite, Repeater over both HTTP
and HTTPS, and search (plain text, dotted domains, hyphenated terms,
column filters) - all correct.
|
|
Implements build-order step 5. internal/store gains an FTS5 virtual
table (history_fts) kept in sync with every Insert in the same
transaction, indexing method/host/path plus the full raw request and
response text - so search covers headers and bodies, not just metadata.
Store.Search ranks by bm25 relevance. internal/ipc's existing "list"
request grows an optional query field rather than a new message type.
cmd/mitmux gets an inline '/' filter on the history view (bubbles/
textinput), esc to clear; live entries arriving while a filter is
active are held back with a "+N new" indicator rather than guessed at,
since FTS match can't be evaluated against a bare Summary.
Two real bugs found via testing against the actual sqlite3 CLI, not
assumed from docs:
1. This SQLite build doesn't support MATCH/bm25() against an aliased
FTS5 table ("no such column") - only the literal table name resolves.
Fixed by leaving history_fts unaliased in the JOIN.
2. FTS5's query grammar treats a wide range of punctuation as syntax,
not literal characters - confirmed '.', '-', '/', '@', '(', ')' all
produce parse errors (or worse, silently different results, as
hyphens get misparsed as column-filter syntax) in an unquoted
bareword. Since that covers the most common things people search
proxy history for (domains, paths, hyphenated headers, IPs), this
would have made the feature fail by default for its primary use
case. Fixed with prepareFTSQuery: quote every plain token as an FTS5
phrase (syntactically valid regardless of content) while still
recognizing AND/OR/NOT and column:value filters.
Also caught, mid-testing, that a query fix wasn't taking effect - traced
to the daemon still running an old `go run` build from before the fix
while only the TUI had been restarted; not a code bug, but a reminder to
restart both.
Verified live end-to-end: plain-text search matching header/body/JSON
content, a previously-failing dotted-domain search now returning exactly
the right single match, a hyphen/host:-filter case, boolean-free numeric
search, filter-clear returning to the unfiltered list, and the pending-
count indicator when new traffic arrives mid-filter.
|
|
Implements build-order step 4, the feature the plan calls out as used
daily. internal/proxy/repeat.go adds Server.Repeat(scheme, host, raw):
dials fresh (HTTP/1.1-only - raw edited text has no equivalent in
HTTP/2's binary framing), writes raw exactly as given with no framing
correction or header injection, and captures the exact response bytes.
This is deliberately separate from forward()'s parsed-*http.Request path
since Repeater's entire point is letting a malformed/edited request
reach the wire unmodified.
Repeater sends are recorded to the same history table as proxy traffic
(added a "source" column: "proxy" vs "repeater") so they show up in the
unified history view and the live subscribe stream, not a separate silo.
internal/ipc gains a "repeat" request/response pair; cmd/mitmuxd wires
proxy.Server into ipc.NewServer via a small Repeater interface so the
daemon keeps owning all network I/O and the TUI stays a thin client.
cmd/mitmux gains a repeater view (bubbles/textarea for the editable raw
request, a read-only viewport for the response), reachable with 'r' from
either the list or detail view, ctrl+r to send.
One real bug found via testing: bubbles/textarea only understands LF,
but HTTP/1.1 requires CRLF, so loading raw bytes straight into it split
each line in two on render. Fixed by normalizing CRLF<->LF at the editor
boundary only (load: strip \r; send: restore it) - documented as a
narrow, known trade-off for bodies with their own embedded LF line
breaks, which is the cost of being able to edit raw HTTP as text at all.
Verified live: edited and sent a plain-HTTP repeater request (confirmed
in SQLite that the edit - including an intentional extra blank line from
imprecise cursor navigation during testing - went out completely
unmodified, which is the correct behavior: mitmux must never "fix" what
the user typed), and sent an HTTPS repeater request against a freshly
captured entry, both getting real 200 responses with exact response
bytes back.
|
|
Implements build-order step 3. Adds:
- internal/store: SQLite (WAL, single-writer) history table, raw
request/response blobs plus metadata for the list view.
- internal/proxy: request/response capture wired into forward(). HTTP/1.1
legs are captured byte-exact via a teeConn that records wire bytes as
they're read, taken right after the message is fully drained (so no
manual re-reading/replaying is needed - RoundTrip's own streaming does
the draining). HTTP/2 legs (no meaningful "raw bytes" of their own -
multiplexed, HPACK-compressed framing) are reconstructed instead, and
marked as such in storage.
- internal/ipc: JSON-over-Unix-socket protocol between mitmuxd (owns the
proxy and the DB) and any client - list/get for queries, subscribe for
a live push stream of newly captured entries. Keeps the proxy engine
independent of the UI, per the architecture sketch.
- cmd/mitmux: Bubble Tea TUI - a live-updating history table and a
request/response detail view with raw bytes.
Two real bugs surfaced during testing and got fixed before commit:
1. http.Transport's HTTP/2 auto-dispatch does a literal *tls.Conn type
assertion on the dialed connection; wrapping it in a capturing teeConn
broke that silently, and HTTP/2 framing got parsed as HTTP/1.1 text.
Fixed by dropping http.Transport for the upstream leg entirely in
favor of an explicit per-protocol round trip (see PLAN.md stack note).
2. singleConnListener wrapped the client teeConn *inside* a
closeSignalConn, so ConnContext's type assertion for it silently
failed and HTTP/1.1 client-side capture never activated. Fixed the
wrap order; verified via direct SQLite inspection that request_exact
flips back to 1 and the stored bytes are genuinely wire-exact
(preserved chunked-encoding framing, original header casing/order).
Verified live: plain HTTP, HTTPS H1.1, HTTPS H2, and a POST with a body,
checked against the raw stored bytes directly in SQLite; IPC list/get/
subscribe against a throwaway client; and the TUI driven end-to-end in a
tmux session (list, detail view, tab between request/response, live
update on a new request while sitting on the list).
|