| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Last of the "should build soon" items from the Burp/ZAP/Caido gap
research - Burp's row highlighting and Caido's Findings both serve
the same real workflow: mark something interesting mid-engagement,
revisit later. Scoped to a boolean flag (★) rather than full free-text
notes/comments, which would need their own text-input overlay for
comparatively modest extra value over a simple marker - tracked as a
real follow-up in PLAN.md, not dropped silently.
internal/store: history gains a flagged column (migrated in for
existing databases the same way source was) plus Store.SetFlagged and
Summary/Entry.Flagged. Search's structured-filter layer (added last
commit for status:/source:) gains flagged:true/false alongside them -
extractStructured already existed for exactly this kind of "pull it out
before it reaches FTS5" filter. internal/ipc gains a "set_flagged"
request. cmd/mitmux: 'f' toggles the flag on the selected history row
(applied optimistically to local state, persisted async - a drift
between local and server state on failure is an acceptable trade-off
for a marker this low-stakes), shown as a ★ column in the list and in
the detail view's title.
store_test.go covers the flagged: parsing (true/false spellings, and
a "looks like it but isn't" case - flagged:maybe - falling through as
literal search text, matching the existing pattern for status:).
Verified live: toggling 'f' shows the star immediately, flagged:true
correctly filtered to just that entry, and a direct SQLite check
confirmed the flag actually persisted to the database (flagged=1),
not just reflected in local UI state.
|
|
Closes another top item from the Burp/ZAP/Caido gap research: status-
code and MIME/type filtering alongside free text is used constantly in
practice (Caido's HTTPQL, Burp's proxy history filter). Scoped to
status and source for now - method: already works today via FTS5's own
method column (a plain text match on "POST" is effectively exact for a
short alphanumeric token), so it didn't need special handling.
status_code isn't a text column FTS5 can index, and doesn't benefit
from full-text matching anyway (it's a numeric comparison, not a word
search), so extractStructured pulls status:/source: tokens out of the
query before it reaches FTS5 and turns them into real parameterized SQL
predicates against history's typed columns: status:404 (exact),
status:>=400 / status:!=200 (comparison operators), status:4xx (also
2xx/3xx/5xx - the shorthand people actually reach for: "show me the
errors"), source:repeater/intruder/proxy. Whatever text remains after
extraction still goes through the existing FTS5 path, so "admin
status:200" correctly ANDs a real full-text match with a real status
predicate in one query. When nothing remains (pure "status:4xx"),
Search skips the FTS5 join entirely and queries history directly.
store_test.go covers the parsing (exact/operator/range/source,
combined with free text, and two "looks like it but isn't" cases -
status:banana and the malformed 4-digit status:4004 - to confirm they
fall through as literal search text instead of being misparsed).
Verified live against real varied traffic (status 200/404/500 requests
plus a POST with an "admin" body) - status:4xx matched only the 404;
status:>=400 matched both 404 and 500; "admin status:200" correctly
matched only the POST and excluded the other unrelated 200; source:proxy
matched everything captured so far. All against the actual SQL execution
path, not just the pure parsing function.
|
|
Another item off the Burp/ZAP/Caido gap list: reading raw JSON
responses without any formatting is real daily friction.
pretty.go parses raw response bytes via net/http (reusing its tested
chunked-transfer-encoding and gzip content-encoding handling rather
than reimplementing either) and, if the decoded body is valid JSON,
returns it indented. Framing headers that no longer describe the
reformatted body (Transfer-Encoding, Content-Encoding, Content-Length)
are dropped from the displayed header block since keeping them would
be actively misleading. Falls back to raw on anything that doesn't
parse cleanly.
This is deliberately display-only and off by default: 'p' toggles it
in the detail view's response tab, refreshing the viewport in place;
the underlying raw bytes (what's stored, what would be resent) are
never touched. Not wired into Repeater's response pane or into either
tool's editable request buffer - the whole point of this tool is byte-
exact control, so nothing that could be sent anywhere gets silently
reformatted, only a read-only view a user explicitly asked to reformat.
Verified live against a real response with a known formatting quirk:
httpbin.org's own JSON output uses Python's json.dumps with ", "
separators, leaving a trailing space before each newline (confirmed
directly in the stored raw bytes: "7B 7D 2C 20 0A" - "{}, \n"). Toggling
pretty mode replaced it with Go's canonical json.Indent output, and
toggling back returned the original raw bytes - proving the
reformatting is real, not just passing through the origin's own
formatting.
|
|
Baseline TUI UX that should exist regardless of feature parity -
flagged directly by the Burp/ZAP/Caido comparison research as missing.
A persistent one-line status bar (proxy address, live request count,
current view) is now appended to every screen. Fetched once at startup
via a new "status" IPC request (internal/store gains Store.Count();
internal/ipc gains StatusMsg plus a daemon-side handler reading
proxy.Server.Addr through ipc.NewServer's new proxyAddr parameter), then
kept approximately live by incrementing locally on each "new" subscribe
push rather than re-querying every time.
'?' opens a full keybinding reference from every view, gated so it
never shadows literal text entry - it's a no-op while typing in the
search box, a rule form field, or (checked via viTextarea.Mode())
insert-mode text in Repeater/Intruder, where a URL query string
literally starting with '?' is completely ordinary input. Any key
dismisses it and returns to whichever view opened it.
Every existing height calculation (table, viewport, textarea panes)
had to shrink by one line to make room for the status bar without
pushing content off-screen - done once via a shared `h := msg.Height-1`
in the WindowSizeMsg handler rather than touching each call site
individually.
Verified live: status bar shows the real proxy address and updates its
count after a live-captured request; '?' renders the full reference
from the history list; dismissing returns to the correct prior view;
and specifically confirmed '?' still types literally (tested typing
"?foo=bar" into a Repeater request body in insert mode) rather than
being swallowed by the help shortcut.
|
|
Started a broader pass to close the gap with Burp/ZAP/Caido (feature
comparison researched, tracked in PLAN.md) and to fully support vi
bindings as asked. This commit is the vi-bindings piece.
Checked the actual bubbles library before writing anything: table and
viewport already ship full vi navigation by default (h/j/k/l, ctrl+u/
ctrl+d, g/G) - nothing to build there. textarea and textinput are
Emacs-style with no vi support at all, and textarea's own ctrl+p
("previous line") was silently shadowed by the ctrl+p shortcut I'd
bound for Intruder's marker insertion - a real bug from the last
session, fixed here by moving it to ctrl+g.
vimode.go adds viTextarea, wrapping textarea.Model with a normal/insert
modal layer. Vi commands are translated into the underlying textarea's
own existing keybindings (a synthesized "alt+right" for `w`, "ctrl+k"
for `d$`, etc.) and reuse its tested cursor/line logic rather than
reimplementing text manipulation - this is a front-end over textarea,
not a parser. Covers what's actually used constantly: h/j/k/l, 0/$,
w/b, x, i/a/I/A/o/O, dd/yy/p/P, dw/d$/d0, gg/G, esc-to-normal (esc
never leaves the view while still in insert mode, matching real vi).
Not attempted: registers beyond one yank slot, visual mode, ex
commands, macros, counts. No undo, since textarea itself has none.
Starts in normal mode on focus, per vi convention - not insert.
Replaces the three textarea.Model fields (Repeater's request editor,
Intruder's template and payload editors) with viTextarea, and adds a
vim-style mode indicator ("-- NORMAL --" / "-- INSERT --") to both
views, without which modal editing is unusable - there'd be no way to
tell which mode you're in.
Verified live via a real session: normal-mode letters don't leak into
the buffer as text, gg/x/dd/yy/p all produce the correct edits in
sequence on a real captured request, i enters insert mode and typing
works, esc from insert correctly drops to normal without leaving the
view, a second esc then backs out, and ctrl+g inserts a marker without
colliding with anything in the vi command set.
|
|
Implements build-order step 7, the last (optional) item. Scoped to
Sniper only - one payload set, one §-marked position fuzzed at a time,
others held at their base value - since that covers most real Intruder
usage; battering ram / pitchfork / cluster bomb aren't implemented.
Sequential sending, capped at 1000 generated requests as a fixed safety
limit.
internal/proxy: repeat.go's Repeat() is refactored into a shared
sendRaw(..., source) primitive so Intrude can reuse the exact same
raw-byte send/record path with source="intruder" instead of
duplicating it. intrude.go adds ParseMarkers/buildRequest (marker
parsing and payload substitution, covered by intrude_test.go - this is
fiddly byte-splicing logic, worth locking down with real tests rather
than trusting it by inspection) and Intrude(), which walks positions ×
payloads calling sendRaw and streaming each result through a callback.
internal/ipc gains a dedicated streaming "intrude" connection (same
shape as Subscribe, but blocking sends rather than drop-on-slow-
consumer - each result is the attack's actual data, not a
notification). cmd/mitmux gains an Intruder view: editable request
template (ctrl+p inserts a § marker at the cursor - typing § directly
also works, ctrl+p just doesn't require a keyboard layout that can
produce it), editable payload list, and a live results table wired to
the existing detail view (selecting a row and hitting enter opens the
full request/response for that specific attack request).
Verified live against real external traffic: a Sniper attack against
httpbin.org/status/§200§ with payloads 200/404/500 produced exactly the
three corresponding real status codes back (not a canned/local result),
confirmed the three requests landed in history tagged source="intruder"
with the § markers correctly stripped from what was actually sent, and
confirmed opening a result row's full detail from the results table.
This closes out the full build order from PLAN.md (steps 1-7).
|
|
Implements build-order step 6, scoped to headers only for this pass -
see PLAN.md for why bodies are a separate problem (request-body capture
currently depends on streaming straight through, which a body-rewriting
rule would have to interrupt; deciding what "exact" means for a
rule-modified request needs its own pass, not a rushed add-on to this
one).
internal/rules: Rule type and ApplyHeaders, which serializes a Header
map to a raw "Name: value\r\n" block, runs enabled rules' match/replace
over that text, and reparses it - operating on text rather than
per-value substitution is what lets a rule add or remove a header, not
just rewrite one, matching how Burp's header match/replace works.
Invalid rule output (bad regex, unparseable result) leaves the header
map untouched rather than corrupting the request.
internal/store: rules table + CRUD. internal/proxy: forward() fetches
enabled rules for each scope and applies them to outReq.Header /
resp.Header, positioned so the existing capture/history pipeline is
untouched - request_raw keeps showing what the client actually sent and
response_raw what the origin actually sent, while the wire itself
reflects the rules. Deliberate split: match-and-replace transforms
traffic, it doesn't rewrite the audit trail. internal/ipc gains
rules_list/rules_save/rules_delete/rules_toggle. cmd/mitmux gains a
rules view ('m' from history) with add/edit/delete/toggle and a small
form (name, match, replace, scope, regex).
Verified live against real external traffic, not just local echoes:
a request-scope rule rewriting User-Agent, confirmed via httpbin.org's
own header echo that the origin received the rewritten value while curl
sent the real one; a response-scope rule rewriting the Server header,
confirmed the client actually received the rewritten value; disabling a
rule confirmed via a follow-up request that it stops applying; and
throughout, history continued showing the pre-rule original on both
sides, confirming the capture/transform split holds.
|
|
Implements build-order step 5. internal/store gains an FTS5 virtual
table (history_fts) kept in sync with every Insert in the same
transaction, indexing method/host/path plus the full raw request and
response text - so search covers headers and bodies, not just metadata.
Store.Search ranks by bm25 relevance. internal/ipc's existing "list"
request grows an optional query field rather than a new message type.
cmd/mitmux gets an inline '/' filter on the history view (bubbles/
textinput), esc to clear; live entries arriving while a filter is
active are held back with a "+N new" indicator rather than guessed at,
since FTS match can't be evaluated against a bare Summary.
Two real bugs found via testing against the actual sqlite3 CLI, not
assumed from docs:
1. This SQLite build doesn't support MATCH/bm25() against an aliased
FTS5 table ("no such column") - only the literal table name resolves.
Fixed by leaving history_fts unaliased in the JOIN.
2. FTS5's query grammar treats a wide range of punctuation as syntax,
not literal characters - confirmed '.', '-', '/', '@', '(', ')' all
produce parse errors (or worse, silently different results, as
hyphens get misparsed as column-filter syntax) in an unquoted
bareword. Since that covers the most common things people search
proxy history for (domains, paths, hyphenated headers, IPs), this
would have made the feature fail by default for its primary use
case. Fixed with prepareFTSQuery: quote every plain token as an FTS5
phrase (syntactically valid regardless of content) while still
recognizing AND/OR/NOT and column:value filters.
Also caught, mid-testing, that a query fix wasn't taking effect - traced
to the daemon still running an old `go run` build from before the fix
while only the TUI had been restarted; not a code bug, but a reminder to
restart both.
Verified live end-to-end: plain-text search matching header/body/JSON
content, a previously-failing dotted-domain search now returning exactly
the right single match, a hyphen/host:-filter case, boolean-free numeric
search, filter-clear returning to the unfiltered list, and the pending-
count indicator when new traffic arrives mid-filter.
|
|
Implements build-order step 4, the feature the plan calls out as used
daily. internal/proxy/repeat.go adds Server.Repeat(scheme, host, raw):
dials fresh (HTTP/1.1-only - raw edited text has no equivalent in
HTTP/2's binary framing), writes raw exactly as given with no framing
correction or header injection, and captures the exact response bytes.
This is deliberately separate from forward()'s parsed-*http.Request path
since Repeater's entire point is letting a malformed/edited request
reach the wire unmodified.
Repeater sends are recorded to the same history table as proxy traffic
(added a "source" column: "proxy" vs "repeater") so they show up in the
unified history view and the live subscribe stream, not a separate silo.
internal/ipc gains a "repeat" request/response pair; cmd/mitmuxd wires
proxy.Server into ipc.NewServer via a small Repeater interface so the
daemon keeps owning all network I/O and the TUI stays a thin client.
cmd/mitmux gains a repeater view (bubbles/textarea for the editable raw
request, a read-only viewport for the response), reachable with 'r' from
either the list or detail view, ctrl+r to send.
One real bug found via testing: bubbles/textarea only understands LF,
but HTTP/1.1 requires CRLF, so loading raw bytes straight into it split
each line in two on render. Fixed by normalizing CRLF<->LF at the editor
boundary only (load: strip \r; send: restore it) - documented as a
narrow, known trade-off for bodies with their own embedded LF line
breaks, which is the cost of being able to edit raw HTTP as text at all.
Verified live: edited and sent a plain-HTTP repeater request (confirmed
in SQLite that the edit - including an intentional extra blank line from
imprecise cursor navigation during testing - went out completely
unmodified, which is the correct behavior: mitmux must never "fix" what
the user typed), and sent an HTTPS repeater request against a freshly
captured entry, both getting real 200 responses with exact response
bytes back.
|
|
Implements build-order step 3. Adds:
- internal/store: SQLite (WAL, single-writer) history table, raw
request/response blobs plus metadata for the list view.
- internal/proxy: request/response capture wired into forward(). HTTP/1.1
legs are captured byte-exact via a teeConn that records wire bytes as
they're read, taken right after the message is fully drained (so no
manual re-reading/replaying is needed - RoundTrip's own streaming does
the draining). HTTP/2 legs (no meaningful "raw bytes" of their own -
multiplexed, HPACK-compressed framing) are reconstructed instead, and
marked as such in storage.
- internal/ipc: JSON-over-Unix-socket protocol between mitmuxd (owns the
proxy and the DB) and any client - list/get for queries, subscribe for
a live push stream of newly captured entries. Keeps the proxy engine
independent of the UI, per the architecture sketch.
- cmd/mitmux: Bubble Tea TUI - a live-updating history table and a
request/response detail view with raw bytes.
Two real bugs surfaced during testing and got fixed before commit:
1. http.Transport's HTTP/2 auto-dispatch does a literal *tls.Conn type
assertion on the dialed connection; wrapping it in a capturing teeConn
broke that silently, and HTTP/2 framing got parsed as HTTP/1.1 text.
Fixed by dropping http.Transport for the upstream leg entirely in
favor of an explicit per-protocol round trip (see PLAN.md stack note).
2. singleConnListener wrapped the client teeConn *inside* a
closeSignalConn, so ConnContext's type assertion for it silently
failed and HTTP/1.1 client-side capture never activated. Fixed the
wrap order; verified via direct SQLite inspection that request_exact
flips back to 1 and the stored bytes are genuinely wire-exact
(preserved chunked-encoding framing, original header casing/order).
Verified live: plain HTTP, HTTPS H1.1, HTTPS H2, and a POST with a body,
checked against the raw stored bytes directly in SQLite; IPC list/get/
subscribe against a throwaway client; and the TUI driven end-to-end in a
tmux session (list, detail view, tab between request/response, live
update on a new request while sitting on the list).
|
|
Implements build-order step 2. CA gains LeafFor(host), signing and
caching per-host leaf certificates on demand. The proxy's CONNECT
handler now terminates TLS with the client using a matching leaf cert
instead of tunneling raw bytes, and forwards each request upstream
over its own independently negotiated TLS connection.
Client-side and upstream-side ALPN are negotiated separately rather
than one being forced to mirror the other: an http.Transport configured
via http2.ConfigureTransport auto-bridges HTTP/1.1 and HTTP/2 on each
side independently, so e.g. an HTTP/1.1-only client reaching an
HTTP/2-preferring origin still works instead of failing the handshake
(caught by testing curl --http1.1 against example.com before this fix).
Verified live: plain HTTP passthrough, HTTPS with default (H2) and
forced HTTP/1.1 clients, and that requests without the mitmux CA
trusted are correctly rejected.
|
|
Implements build-order step 1: headless proxy daemon (mitmuxd) with
plaintext HTTP passthrough and raw CONNECT tunneling, plus root CA
generation/persistence for later TLS interception. Verified live
against real HTTP and HTTPS requests through the proxy.
|