diff options
| author | srdusr <[email protected]> | 2024-09-14 22:16:00 +0200 |
|---|---|---|
| committer | srdusr <[email protected]> | 2024-09-14 22:16:00 +0200 |
| commit | a02d612ba5efb5f1e79a2f9a0778890e8c3e6acf (patch) | |
| tree | 4d1da9141392237bfa4e132c04191e12c85ef992 /internal/ipc | |
| parent | 70f675e59ac4bb0e469ca0073f8cc0506b17677e (diff) | |
| download | mitmux-a02d612ba5efb5f1e79a2f9a0778890e8c3e6acf.tar.gz mitmux-a02d612ba5efb5f1e79a2f9a0778890e8c3e6acf.zip | |
Search/filter: FTS5 index over history
Implements build-order step 5. internal/store gains an FTS5 virtual
table (history_fts) kept in sync with every Insert in the same
transaction, indexing method/host/path plus the full raw request and
response text - so search covers headers and bodies, not just metadata.
Store.Search ranks by bm25 relevance. internal/ipc's existing "list"
request grows an optional query field rather than a new message type.
cmd/mitmux gets an inline '/' filter on the history view (bubbles/
textinput), esc to clear; live entries arriving while a filter is
active are held back with a "+N new" indicator rather than guessed at,
since FTS match can't be evaluated against a bare Summary.
Two real bugs found via testing against the actual sqlite3 CLI, not
assumed from docs:
1. This SQLite build doesn't support MATCH/bm25() against an aliased
FTS5 table ("no such column") - only the literal table name resolves.
Fixed by leaving history_fts unaliased in the JOIN.
2. FTS5's query grammar treats a wide range of punctuation as syntax,
not literal characters - confirmed '.', '-', '/', '@', '(', ')' all
produce parse errors (or worse, silently different results, as
hyphens get misparsed as column-filter syntax) in an unquoted
bareword. Since that covers the most common things people search
proxy history for (domains, paths, hyphenated headers, IPs), this
would have made the feature fail by default for its primary use
case. Fixed with prepareFTSQuery: quote every plain token as an FTS5
phrase (syntactically valid regardless of content) while still
recognizing AND/OR/NOT and column:value filters.
Also caught, mid-testing, that a query fix wasn't taking effect - traced
to the daemon still running an old `go run` build from before the fix
while only the TUI had been restarted; not a code bug, but a reminder to
restart both.
Verified live end-to-end: plain-text search matching header/body/JSON
content, a previously-failing dotted-domain search now returning exactly
the right single match, a hyphen/host:-filter case, boolean-free numeric
search, filter-clear returning to the unfiltered list, and the pending-
count indicator when new traffic arrives mid-filter.
Diffstat (limited to 'internal/ipc')
| -rw-r--r-- | internal/ipc/ipc.go | 17 | ||||
| -rw-r--r-- | internal/ipc/server.go | 8 |
2 files changed, 23 insertions, 2 deletions
diff --git a/internal/ipc/ipc.go b/internal/ipc/ipc.go index b1a997a..691dadf 100644 --- a/internal/ipc/ipc.go +++ b/internal/ipc/ipc.go @@ -20,6 +20,11 @@ type Request struct { BeforeID int64 `json:"before_id,omitempty"` ID int64 `json:"id,omitempty"` + // For "list": a non-empty Query switches from most-recent-first to + // an FTS5 search (see store.Store.Search for syntax), ranked by + // relevance. + Query string `json:"query,omitempty"` + // For "repeat": send Raw to scheme://host exactly as given. Scheme string `json:"scheme,omitempty"` Host string `json:"host,omitempty"` @@ -69,7 +74,17 @@ func (c *Client) Close() error { // List returns up to limit history summaries older than beforeID (0 for // the most recent), newest first. func (c *Client) List(limit int, beforeID int64) ([]store.Summary, error) { - if err := c.enc.Encode(Request{Type: "list", Limit: limit, BeforeID: beforeID}); err != nil { + return c.list(Request{Type: "list", Limit: limit, BeforeID: beforeID}) +} + +// Search returns up to limit history summaries matching an FTS5 query +// (see store.Store.Search for syntax), ranked by relevance. +func (c *Client) Search(query string, limit int, beforeID int64) ([]store.Summary, error) { + return c.list(Request{Type: "list", Query: query, Limit: limit, BeforeID: beforeID}) +} + +func (c *Client) list(req Request) ([]store.Summary, error) { + if err := c.enc.Encode(req); err != nil { return nil, err } var resp Response diff --git a/internal/ipc/server.go b/internal/ipc/server.go index e3b2a29..0e839fb 100644 --- a/internal/ipc/server.go +++ b/internal/ipc/server.go @@ -93,7 +93,13 @@ func (s *Server) handleConn(conn net.Conn) { switch req.Type { case "list": - entries, err := s.db.List(req.Limit, req.BeforeID) + var entries []store.Summary + var err error + if req.Query != "" { + entries, err = s.db.Search(req.Query, req.Limit, req.BeforeID) + } else { + entries, err = s.db.List(req.Limit, req.BeforeID) + } if err != nil { enc.Encode(Response{Type: "error", Error: err.Error()}) continue |