Small Models, Serious Research Economics
Compact model experiments focused on capability per parameter and visible API cost.
Compact model experiments focused on capability per parameter and visible API cost.
Capability per stage, not per product
A research product does not need one model to perform every operation. Query classification, extraction, deduplication, citation formatting, and final synthesis have different capability requirements.
North experiments with sub-billion-parameter models for bounded stages where latency, memory, and repeatability matter more than broad world knowledge. The parameter count is a constraint, not a quality claim.
Cost accounting that users can see
Total API cost is shaped by input length, retrieval volume, number of passes, model choice, and retries. A low token price can still produce an expensive workflow if the system repeatedly resends the same context.
Horizon budgets work by stage and preserves intermediate artifacts so a failed synthesis does not force retrieval and extraction to run again. The target is predictable research economics, not a universal promise of a particular price.
Where compact models fail
Small models are less forgiving when instructions are ambiguous, domains shift, or the task requires long-range synthesis. Routing must detect those cases and escalate rather than forcing a compact model to imitate capability it does not have.
Evaluation therefore includes refusal quality, escalation accuracy, and sensitivity to prompt variation. Cheap failure is still failure; efficiency only matters when the result remains useful.