What I am working my way through, from steering a single model to running whole systems. Written to be understood, not to sound difficult.
Comfortable
Models know a lot and can do little, until you hand them tools
Search, calculate, run code, call an API. The model decides for itself when to reach for a tool, and folds the result back into its answer. That single ability is what turns a text generator into something that can act.
Comfortable
The model fills in the form; you are the one who executes it
The mechanism behind tool use. You describe your functions as a JSON schema; instead of prose, the model answers with a structured call like get_weather({"city": "Stuttgart"}). Nothing runs on the model's side. It hands you a filled-in request, and your code decides what to do with it.
Comfortable
When an answer goes wrong, you want to see why, not just that
Every request is recorded as a trace: prompts, tool calls, intermediate steps, latency, tokens, cost. This is the part of the job I already do for infrastructure. The object under the microscope is new, the discipline is not.
Learning now
One API in front of every provider
The central junction between your applications and all model vendors: routing and fallbacks, caching, rate limits, key management, a single view of cost (LiteLLM, Portkey). If one provider goes down, traffic simply carries on somewhere else.
Learning now
Every token is on the bill
So you save systematically: cache answers, route easy work to small models in a cascade, trim prompts, batch requests. The goal is not a cheaper product but the same quality for a fraction of the invoice, and knowing which lever costs you quality.
What I set up, what it cost, and what broke. Dated, with the measured value beside it.
No entries yet. The log starts when there is something running to measure — from January, when inference for Kotoba moves onto a Mac Mini in Tokyo. An empty log that says so is worth more than one written from memory.