OPERATE: CUT THE CLOUD
Context
The frontier capability of a language model is no longer the only useful thing about it. A 70B-parameter open-weight model, running on a laptop with enough memory, will answer most of the questions a working professional needs to ask in a day. The cloud is still faster for some tasks and strictly necessary for some others. It is no longer the default.
Local Frontier is a project that documents the practical edge of that shift. It runs the current generation of open-weight models, exposes them through API endpoints that match the cloud incumbents byte for byte, and keeps a running log of what works, what does not, and what the gap to the frontier actually looks like at a given week.
The thesis is not anti-cloud. It is that the choice between cloud and local should be made per task, not per stack.
What was built
LM Studio hosts the models locally. A small Node service sits in front of LM Studio and re-exposes the inference endpoints in two shapes: the OpenAI Chat Completions shape, and the Anthropic Messages shape. Any existing client that targets one of those two APIs can swap its base URL and run against a 70B local model with no other code change.
The interesting design decision was the surface. Both incumbents have quirks the spec sheet does not describe: streaming chunk shapes, tool-use event ordering, content-block delimiters. The local service emulates those quirks even when the underlying model does not need them, because the goal is drop-in compatibility, not a new API. A test suite walks through about thirty named edge cases and asserts byte-equivalent streaming output for the most common workloads.
A separate concern was honesty. The service exposes a /capabilities
endpoint that returns the actual model parameters, context window, and
known limitations, so a client that wants to gate behavior on the real
capability can do so without pretending the local model is something it
is not.
What was learned
The hard part was not the models. It was the API emulation. Cloud APIs look stable from the outside and are surprisingly under-documented in their details. Half the work on this project was reading network captures.