I built labelswap so I stop paying a frontier model to pick a label

A support ticket comes in. A frontier model, a large paid LLM (large language model), picks one of twenty labels, and I pay every time.

Two Amazon engineers replaced Claude Sonnet 4.6 with Strands Decider to label internal tickets. They tuned a confidence cutoff, then ran both side by side. labelswap is my simpler, clean-room take: a Naive Bayes classifier with a confidence gate, not a decision model like Strands Decider. It learns from your LLM's past labels, answers confident cases locally in microseconds, and sends the rest back.

Here is how I am testing it:

1. Clone the repo and run pip install -e ..
2. Train on a labeled log with a target agreement rate.
3. Run labelswap shadow next to fresh LLM labels.
4. Switch to labelswap decide once agreement holds.

On held-out rows of a made-up demo log, it answered 136 of 141 tickets locally and matched the LLM 98.5% of the time. The demo is easy, so real logs will see less. Naive Bayes can be confidently wrong: one demo mistake scored 1.0, so shadow mode comes first.

Takeaway: let the expensive model teach a cheap one, then check it.

Coverage: Hacker News.