diff options
| -rw-r--r-- | PLAN.md | 71 | ||||
| -rw-r--r-- | PLUGINS.md | 8 | ||||
| -rw-r--r-- | README.md | 4 | ||||
| -rw-r--r-- | internal/clientcert/clientcert.go | 14 | ||||
| -rw-r--r-- | internal/rules/rules.go | 18 | ||||
| -rw-r--r-- | internal/scope/scope.go | 8 | ||||
| -rw-r--r-- | internal/store/store.go | 91 | ||||
| -rw-r--r-- | plugins/authcheck/main.go | 265 |
8 files changed, 410 insertions, 69 deletions
@@ -864,7 +864,70 @@ because it came from a plugin rather than raw traffic. core-engine change to how requests get dispatched, not something a plugin protocol should be shaped around. -Next: Phase 1's plugins themselves (Autorize- and Param-Miner- -equivalents first, since they validate the simple subscribe-act-tag -path before anything gets built on top of it), then the live-RPC -protocol addition for JWT Editor/SAML Raider. +### Wire-format consistency fix (found while building the first plugin) + +Writing PLUGINS.md as an authoritative external spec surfaced a real, +pre-existing inconsistency: `store.Summary`/`Entry`/`EntryTag`/ +`WSMessage`, `rules.Rule`, `scope.Rule`, and `clientcert.Cert` had no +JSON struct tags at all, so Go's default marshaling serialized them as +PascalCase (`"ID"`, `"StartedAt"`) while the rest of the protocol +(`EntryDetail`'s own fields, every `Request`/`Response` wrapper field) +uses snake_case. Confirmed live against a real daemon before touching +anything: a raw socket `list` request came back with `"ID"`, +`"StartedAt"`, `"StatusCode"` - exactly the mismatch suspected. Nothing +outside this repo's own Go code consumes this wire format yet, so +adding tags now (rather than documenting the wart) was a free, purely +additive fix - every affected struct now tags snake_case throughout, +consistent with the rest of the protocol, verified with a full +build/vet/test pass afterward. + +### First plugin shipped: `plugins/authcheck` + +An Autorize-style authorization checker, and the reference +implementation `PLUGINS.md` points to. Deliberately speaks the wire +protocol directly - its own local `request`/`response`/`summary`/ +`entryDetail` structs mirroring the real ones field-for-field, not +imported - rather than reaching into `internal/ipc`, even though nothing +stops a Go plugin from doing that. The point isn't style: it's proof +that the documented protocol is actually sufficient on its own, since +that's the only thing a non-Go plugin author has to work with, and the +best available check against silently depending on some Go-internal +convenience that never made it into the docs. + +For every proxied (not `source: "repeater"` - see below) request +carrying an `Authorization` or `Cookie` header, resends it via `repeat` +with that header stripped and compares status classes: if the original +succeeded (2xx/3xx) and the anonymous resend *also* succeeded, that's a +likely missing-function-level-access-control bug, tagged +`authcheck:bypass` with the two status codes and which header was +stripped as the tag's JSON data. Matches Burp's own Autorize default of +only surfacing likely findings, not logging every check performed. +Explicitly guards against reprocessing its own resends (`Repeat` +records `source: "repeater"`, filtered out on the subscribe feed) - +without that, the plugin would try to authcheck its own control-group +requests, which is at best wasted work and at worst misattributes a +finding to the wrong entry. + +Simplified relative to real Autorize: Autorize's other mode swaps in a +SECOND, lower-privileged identity's session rather than going fully +anonymous, which catches cross-account IDORs an anonymous-only check +can't (an endpoint might correctly reject "no credentials" while still +leaking another user's data to a validly-authenticated-but-wrong-user +request). That needs a second credential as input this reference +version doesn't take - a natural, small extension (a `-low-priv-cookie` +flag, one more resend variant, one more tag) once wanted. + +Verified live end to end against a real daemon, a real Python origin +server with one endpoint that looks like it enforces auth but doesn't +(vulnerable by design) and one that actually does (the control case): +the broken endpoint was correctly tagged, the properly-secured one was +correctly left untouched - no false positive - confirmed both via the +stored tag data (`go run` against the daemon directly) and visually in +the TUI (tmux, real keystrokes): the `Tags` column badge, `T`'s tag +list, and the tag detail view's JSON-colorized data, ANSI-verified, all +showing the plugin's actual findings. + +Next: the remaining Phase 1 plugins (Param Miner, Backslash Powered +Scanner, Retire.js - no new protocol capability needed, same pattern +`authcheck` already validates), then the live-RPC protocol addition for +JWT Editor/SAML Raider. @@ -11,6 +11,14 @@ Go), and means a plugin can be developed and tested against the exact same socket the TUI is already using, with `mitmux` itself open in another terminal watching what happens in real time. +`plugins/authcheck` is a real, working reference implementation - an +Autorize-style authorization checker (resends a captured request with +its auth header stripped, tags the entry if the response still +succeeds) - written to only ever exercise what's documented on this +page, not any of mitmux's own internal Go packages, specifically so it +proves this protocol is sufficient on its own. Worth reading alongside +this document, or just copying as a starting point. + ## Connecting The socket path is the same one `mitmux -socket` and `mitmuxd -socket` @@ -69,7 +69,9 @@ the same socket the TUI itself uses - see [`PLUGINS.md`](PLUGINS.md). JWT decoder, an authorization-bypass checker, anything. Tags show as a badge in the history list, searchable via `tag:name`, viewable (`T` from detail view) with JSON data syntax-highlighted the same way - a pretty-printed response is. See [`PLUGINS.md`](PLUGINS.md). + a pretty-printed response is. See [`PLUGINS.md`](PLUGINS.md) and + `plugins/authcheck` for a real, working one (an Autorize-style + authorization checker). - **Comparer**: mark one entry (`c`), then `c` on a different entry to see a colored unified diff of either side's request or response. - **Decoder**: standalone URL/Base64/Hex/HTML encode and decode (`d`), diff --git a/internal/clientcert/clientcert.go b/internal/clientcert/clientcert.go index 4bba835..1d94879 100644 --- a/internal/clientcert/clientcert.go +++ b/internal/clientcert/clientcert.go @@ -15,13 +15,13 @@ import ( // Cert is one client certificate, scoped to hosts matching Pattern. type Cert struct { - ID int64 - Enabled bool - Name string - Pattern string - IsRegex bool - CertPEM []byte - KeyPEM []byte + ID int64 `json:"id"` + Enabled bool `json:"enabled"` + Name string `json:"name"` + Pattern string `json:"pattern"` + IsRegex bool `json:"is_regex"` + CertPEM []byte `json:"cert_pem"` + KeyPEM []byte `json:"key_pem"` } func (c Cert) matches(host string) bool { diff --git a/internal/rules/rules.go b/internal/rules/rules.go index e0547db..76d2ce7 100644 --- a/internal/rules/rules.go +++ b/internal/rules/rules.go @@ -14,17 +14,17 @@ import ( // Rule is one match-and-replace rule. type Rule struct { - ID int64 - Enabled bool - Name string - Scope string // "request" or "response" - Part string // "header" or "body" - Match string - Replace string - IsRegex bool + ID int64 `json:"id"` + Enabled bool `json:"enabled"` + Name string `json:"name"` + Scope string `json:"scope"` // "request" or "response" + Part string `json:"part"` // "header" or "body" + Match string `json:"match"` + Replace string `json:"replace"` + IsRegex bool `json:"is_regex"` // Position orders rule application (ascending) when several rules // could touch the same text. - Position int + Position int `json:"position"` } // ApplyHeaders rewrites h in place by serializing it to a raw diff --git a/internal/scope/scope.go b/internal/scope/scope.go index d8d2e80..a259801 100644 --- a/internal/scope/scope.go +++ b/internal/scope/scope.go @@ -22,10 +22,10 @@ import ( // match-and-replace rules already use, for the same reason: one // consistent mental model across both rule types in this tool. type Rule struct { - ID int64 - Enabled bool - Pattern string - IsRegex bool + ID int64 `json:"id"` + Enabled bool `json:"enabled"` + Pattern string `json:"pattern"` + IsRegex bool `json:"is_regex"` } // InScope reports whether host should be recorded, given rules. diff --git a/internal/store/store.go b/internal/store/store.go index ecc3854..8e9ab2b 100644 --- a/internal/store/store.go +++ b/internal/store/store.go @@ -159,66 +159,69 @@ func (s *Store) Close() error { return s.db.Close() } -// Entry is one captured request/response pair. +// Entry is one captured request/response pair. Never itself put on the +// IPC wire (see EntryDetail, which is) - tagged anyway for consistency +// with the rest of this package's now-uniformly-snake_case convention, +// and in case that ever changes. type Entry struct { - ID int64 - StartedAt time.Time - Duration time.Duration - Method string - Scheme string - Host string - Path string - StatusCode int // 0 if no response was received - RequestRaw []byte - ResponseRaw []byte // nil if no response was received - RequestExact bool // true if RequestRaw is wire-exact, false if reconstructed (e.g. HTTP/2) - ResponseExact bool + ID int64 `json:"id"` + StartedAt time.Time `json:"started_at"` + Duration time.Duration `json:"duration"` + Method string `json:"method"` + Scheme string `json:"scheme"` + Host string `json:"host"` + Path string `json:"path"` + StatusCode int `json:"status_code"` // 0 if no response was received + RequestRaw []byte `json:"request_raw"` + ResponseRaw []byte `json:"response_raw"` // nil if no response was received + RequestExact bool `json:"request_exact"` // true if RequestRaw is wire-exact, false if reconstructed (e.g. HTTP/2) + ResponseExact bool `json:"response_exact"` // Truncated is true when the corresponding *Raw field would have // been an exact capture but hit maxCaptureBytes and had bytes // dropped off the end - distinct from a false *Exact, which also // covers HTTP/2's inherently-reconstructed (never wire-exact to // begin with) captures. Only meaningful when the matching *Exact // field is false; a capture can't be both exact and truncated. - RequestTruncated bool - ResponseTruncated bool - Error string // network/transport error, if the request never got a response - Source string // "proxy" or "repeater" - Flagged bool + RequestTruncated bool `json:"request_truncated"` + ResponseTruncated bool `json:"response_truncated"` + Error string `json:"error"` // network/transport error, if the request never got a response + Source string `json:"source"` // "proxy" or "repeater" + Flagged bool `json:"flagged"` } // Summary is the lightweight metadata used for the history list view - // no request/response bodies. type Summary struct { - ID int64 - StartedAt time.Time - Duration time.Duration - Method string - Scheme string - Host string - Path string - StatusCode int - ReqSize int - RespSize int - Error string - Source string - Flagged bool + ID int64 `json:"id"` + StartedAt time.Time `json:"started_at"` + Duration time.Duration `json:"duration"` + Method string `json:"method"` + Scheme string `json:"scheme"` + Host string `json:"host"` + Path string `json:"path"` + StatusCode int `json:"status_code"` + ReqSize int `json:"req_size"` + RespSize int `json:"resp_size"` + Error string `json:"error"` + Source string `json:"source"` + Flagged bool `json:"flagged"` // Tags is every distinct plugin tag on this entry, comma-joined - // cheap enough to compute per row (a correlated subquery, see List/ // Search) that a plugin-tagged entry shows a badge in the history // list itself, not just in its detail view. - Tags string + Tags string `json:"tags"` } // EntryTag is one plugin-contributed marker on a history entry - see // the "tag_entry" IPC request and entry_tags' own schema comment for // what Plugin/Data mean. type EntryTag struct { - ID int64 - EntryID int64 - Plugin string - Tag string - Data string - CreatedAt time.Time + ID int64 `json:"id"` + EntryID int64 `json:"entry_id"` + Plugin string `json:"plugin"` + Tag string `json:"tag"` + Data string `json:"data"` + CreatedAt time.Time `json:"created_at"` } // AddEntryTag stores t and returns its assigned ID. @@ -785,12 +788,12 @@ func (s *Store) DeleteClientCert(id int64) error { // internal/proxy/websocket.go for why it's one row per frame rather than // per reassembled logical message. type WSMessage struct { - ID int64 - EntryID int64 - StartedAt time.Time - Direction string // "client_to_server" or "server_to_client" - Opcode int // RFC 6455 opcode: 1 text, 2 binary, 8 close, 9 ping, 10 pong - Payload []byte + ID int64 `json:"id"` + EntryID int64 `json:"entry_id"` + StartedAt time.Time `json:"started_at"` + Direction string `json:"direction"` // "client_to_server" or "server_to_client" + Opcode int `json:"opcode"` // RFC 6455 opcode: 1 text, 2 binary, 8 close, 9 ping, 10 pong + Payload []byte `json:"payload"` } // AddWSMessage stores one captured frame and returns its assigned ID. diff --git a/plugins/authcheck/main.go b/plugins/authcheck/main.go new file mode 100644 index 0000000..8f2a5ec --- /dev/null +++ b/plugins/authcheck/main.go @@ -0,0 +1,265 @@ +// Command authcheck is a reference mitmux plugin - an Autorize-style +// authorization checker - and, deliberately, a template: it speaks +// mitmux's plugin wire protocol directly (raw JSON over the control +// socket, see PLUGINS.md) rather than importing mitmux's own internal +// Go packages, the same way a plugin written in any other language +// would have to. That's not a style preference - it's what actually +// proves PLUGINS.md's documented protocol is sufficient on its own, +// rather than silently depending on Go-internal conveniences a +// non-Go plugin author wouldn't have access to. +// +// What it does: for every captured request that carries an +// Authorization or Cookie header, resends the exact same request with +// that header stripped and compares the result. A resend that still +// succeeds where the original also succeeded means the endpoint +// doesn't actually enforce the authentication it appears to require - +// a missing-function-level-access-control bug, one of the more common +// real findings this class of check turns up. Matches are tagged +// "authcheck:bypass" on the original entry, with structured detail (the +// original status vs. the anonymous resend's) for the TUI's tag panel. +// +// This is the simplified half of what Burp's Autorize does: Autorize +// additionally supports swapping in a SECOND, lower-privileged +// identity's session and comparing against that - useful for catching +// cross-account IDORs a fully-anonymous check can't see. That needs a +// second credential as input, which this reference version doesn't +// take; a -low-priv-cookie flag doing that is a natural, small +// extension of the same pattern used here. +package main + +import ( + "bufio" + "bytes" + "encoding/json" + "flag" + "fmt" + "log" + "net" + "net/http" + "os" + "path/filepath" +) + +// request/response mirror internal/ipc's wire structs field-for-field +// (see PLUGINS.md) - defined fresh here, not imported, so this file +// only ever exercises what's actually documented as the public +// protocol. +type request struct { + Type string `json:"type"` + ID int64 `json:"id,omitempty"` + Scheme string `json:"scheme,omitempty"` + Host string `json:"host,omitempty"` + Raw []byte `json:"raw,omitempty"` + TagPlugin string `json:"tag_plugin,omitempty"` + Tag string `json:"tag,omitempty"` + TagData string `json:"tag_data,omitempty"` +} + +type summary struct { + ID int64 `json:"id"` + Method string `json:"method"` + Scheme string `json:"scheme"` + Host string `json:"host"` + Path string `json:"path"` + Source string `json:"source"` +} + +type entryDetail struct { + summary + StatusCode int `json:"status_code"` + RequestRaw []byte `json:"request_raw"` +} + +type response struct { + Type string `json:"type"` + Entries []summary `json:"entries,omitempty"` + New *summary `json:"new,omitempty"` + Detail *entryDetail `json:"detail,omitempty"` + TagID int64 `json:"tag_id,omitempty"` + Error string `json:"error,omitempty"` +} + +// client is a minimal request/response connection - send one request, +// read back one response, repeat. A plugin also needs a second, +// separate connection for "subscribe" (see main): that one is only +// ever written to once and then just read from continuously, so it +// doesn't need this type's request/response pairing at all. +type client struct { + conn net.Conn + enc *json.Encoder + dec *json.Decoder +} + +func dial(path string) (*client, error) { + conn, err := net.Dial("unix", path) + if err != nil { + return nil, err + } + return &client{conn: conn, enc: json.NewEncoder(conn), dec: json.NewDecoder(conn)}, nil +} + +func (c *client) call(req request) (response, error) { + if err := c.enc.Encode(req); err != nil { + return response{}, err + } + var resp response + if err := c.dec.Decode(&resp); err != nil { + return response{}, err + } + if resp.Type == "error" { + return response{}, fmt.Errorf("%s", resp.Error) + } + return resp, nil +} + +func defaultSocketPath() string { + if rt := os.Getenv("XDG_RUNTIME_DIR"); rt != "" { + return filepath.Join(rt, "mitmux.sock") + } + // Matches internal/ca.Dir() without importing it - see this file's + // package doc for why plugins shouldn't reach into mitmux's own Go + // internals even when it would be more convenient. + cfg, err := os.UserConfigDir() + if err != nil { + return "mitmux.sock" + } + return filepath.Join(cfg, "mitmux", "mitmux.sock") +} + +// authHeaders are checked in order; the first one present on a request +// is what gets stripped for the anonymous resend. Checking more than +// one matters: an API might authenticate via a bearer token while a +// browser-driven flow on the same host uses a session cookie, and both +// are worth checking independently rather than only ever picking one. +var authHeaders = []string{"Authorization", "Cookie"} + +type result struct { + OriginalStatus int `json:"original_status"` + ResendStatus int `json:"resend_status"` + StrippedHeader string `json:"stripped_header"` + Verdict string `json:"verdict"` +} + +func main() { + socketPath := flag.String("socket", "", "daemon control socket path (default: same as mitmux itself)") + pluginName := flag.String("name", "authcheck", "name this plugin tags entries as") + flag.Parse() + + path := *socketPath + if path == "" { + path = defaultSocketPath() + } + + actor, err := dial(path) + if err != nil { + log.Fatalf("dial %s: %v", path, err) + } + defer actor.conn.Close() + + subConn, err := net.Dial("unix", path) + if err != nil { + log.Fatalf("dial %s (subscribe): %v", path, err) + } + defer subConn.Close() + if err := json.NewEncoder(subConn).Encode(request{Type: "subscribe"}); err != nil { + log.Fatalf("subscribe: %v", err) + } + + log.Printf("authcheck: watching live traffic on %s", path) + dec := json.NewDecoder(subConn) + for { + var resp response + if err := dec.Decode(&resp); err != nil { + log.Fatalf("subscribe feed closed: %v", err) + } + if resp.Type != "new" || resp.New == nil { + continue + } + sum := *resp.New + // Never touch our own resends - the daemon records a Repeat() + // as source="repeater", and reprocessing it would misfile the + // very control-group requests this check depends on (it also + // wouldn't loop: a resend already has its auth header removed, + // so it would never match authHeaders again - but it's still + // pointless work and pointless noise to try). + if sum.Source != "proxy" { + continue + } + if err := checkEntry(actor, *pluginName, sum.ID); err != nil { + log.Printf("entry #%d: %v", sum.ID, err) + } + } +} + +func checkEntry(c *client, pluginName string, id int64) error { + resp, err := c.call(request{Type: "get", ID: id}) + if err != nil { + return fmt.Errorf("get: %w", err) + } + if resp.Detail == nil { + return fmt.Errorf("get: no detail in response") + } + detail := *resp.Detail + + req, err := http.ReadRequest(bufio.NewReader(bytes.NewReader(detail.RequestRaw))) + if err != nil { + return nil // not a well-formed request we can safely reparse - skip, not fatal + } + + var strippedHeader string + for _, h := range authHeaders { + if req.Header.Get(h) != "" { + strippedHeader = h + break + } + } + if strippedHeader == "" { + return nil // nothing to check + } + + req2, err := http.ReadRequest(bufio.NewReader(bytes.NewReader(detail.RequestRaw))) + if err != nil { + return nil + } + req2.Header.Del(strippedHeader) + + var buf bytes.Buffer + if err := req2.Write(&buf); err != nil { + return fmt.Errorf("rebuild request: %w", err) + } + + repeatResp, err := c.call(request{Type: "repeat", Scheme: detail.Scheme, Host: detail.Host, Raw: buf.Bytes()}) + if err != nil { + return fmt.Errorf("repeat: %w", err) + } + if repeatResp.Detail == nil { + return fmt.Errorf("repeat: no detail in response") + } + resentStatus := repeatResp.Detail.StatusCode + + suspicious := successClass(detail.StatusCode) && successClass(resentStatus) + if !suspicious { + return nil // matches Burp's own default: only surface likely findings, not every check performed + } + + data, err := json.Marshal(result{ + OriginalStatus: detail.StatusCode, + ResendStatus: resentStatus, + StrippedHeader: strippedHeader, + Verdict: "bypass", + }) + if err != nil { + return fmt.Errorf("marshal result: %w", err) + } + + if _, err := c.call(request{Type: "tag_entry", ID: id, TagPlugin: pluginName, Tag: "authcheck:bypass", TagData: string(data)}); err != nil { + return fmt.Errorf("tag_entry: %w", err) + } + log.Printf("#%d %s %s -> possible auth bypass: %d without %s still got %d", + id, detail.Method, detail.Path, detail.StatusCode, strippedHeader, resentStatus) + return nil +} + +func successClass(status int) bool { + return status >= 200 && status < 400 +} |