--- title: "If It's Useful, It Won't Stay Closed | Daedalus" url: "https://ddls.co/blog/if-its-useful-it-wont-stay-closed/" description: "Scoped and describable model capabilities tend to get open versions quickly, which changes how much dependence a closed small model deserves." author: "Hunter Warburton" published: "2026-10-13" modified: "2026-10-13" --- # If It's Useful, It Won't Stay Closed By Hunter Warburton & muse-wonder-full-1.0 October 13, 2026 A version of the same decision keeps showing up in our work. A new closed model or API does some scoped job clearly better (or just much more cheaply) than whatever we were using before, and we have to decide whether to build on it now or wait. More and more often the answer depends less on how good the closed version is and more on how copyable the job is. Anything genuinely useful at a scoped size tends to get an open version fairly quickly these days, either from the company that made it or from somebody who rebuilds the useful behavior on top of an open base model. ![Candlelit workshop hall where one finished brass balance in the foreground is being copied on rows of benches receding into darkness.](https://ddls.co/images/blog-if-its-useful-it-wont-stay-closed-inline.webp) The clearest cases share a few properties that are easy to check from the outside. The job can be described as a function, with inputs you can assemble and outputs you can test. The output is usually much smaller than the input (a label, a score, a transcription, an embedding, fields extracted from a document). Most of the hard capability in cases like that is in reading and understanding the input, and open base models already do that part well enough for a lot of scoped work. Reproducing the closed product often comes down to reading information the base model already represents, then doing calibration and serving work around it, and sometimes fine-tuning at a size where fine-tuning is cheap. None of that requires access to the original company's weights. It does not require the original company's business model either (an individual can publish an implementation for reputation and an open model lab benefits when its base becomes the thing everyone builds on and a platform company can give a model away because it makes money from hosting and surrounding services). Secrecy still buys the first company some time, but that time is measured against how quickly motivated strangers can rebuild a describable behavior, and the interval keeps getting shorter. Whether waiting is worth it depends a lot on what you would have to send the closed service in the meantime. For a decision or classification workflow, the inputs can look harmless one at a time, while the full stream describes how a business operates. It contains the leads being sorted and the tickets being prioritized and the documents and claims and customers being routed down one path or another. We run small open models locally for work in that shape already (a Qwen vision model pulling data off faxes for a medical client is one example), and we self-host most of our software in general. AI has lowered the cost of building and operating your own systems enough that avoiding an external dependency is often the practical option rather than an expensive preference. For cheap scoped work, our default is to run it ourselves unless a closed option is clearly better on our own examples, or the data involved does not matter much. Decision models are a recent example where the copyable properties lined up at once. [TypeSafe's Jev](https://typesafe.ai/blog/introducing-system-one-models-and-jev) takes state (an email or a ticket or structured JSON from one of your systems) together with typed questions and returns a typed answer instead of generating text. That answer might be a choice from labels you defined or a score or a probability for a yes-or-no question. Classifiers in various forms have existed for a long time. A general version that accepts new labels at request time and runs far more cheaply than a large language model is still a useful improvement. I wrote internally when it appeared that open models would probably do a similar thing within a couple of months if it was actually good. Open implementations based on Qwen (including [Kev](https://github.com/jaredpalmer/kev) and Cloudflare's Apache-2.0 [Clef](https://blog.cloudflare.com/clef-decision-models/)) appeared within days to weeks, several of them compatible with Jev's own API. The example matters here mostly because the timeline was so compressed, and because the copies copied the interface along with the behavior. Switching between a hosted model and a local one became a configuration change rather than a rewrite. We have actually run members of that open family on our own work (along with Perplexity's larger open decider) because Ari needs exactly this kind of model to tag meeting transcripts automatically. A mistag matters much more than a missed tag in our system because the wrong tag can make one client's meeting visible where it should not be. So we backtested the models against transcripts we had tagged by hand and only let a model apply a tag when it was confident (and when our existing embedding matcher picked the same tag). Eighteen small open models got scored on a first batch of 101 meetings and the five leaders went on to all 407 hand-tagged transcripts. The honest result is that none of the small ones stayed at zero mistags across the full set. The best of them (a 4B model called JevK5) tagged 113 of the 407 with three mistags under the agreement rule, while our simple embedding baseline tagged fewer at 62 and made no mistakes on the same data. Clean results on the first 101 meetings did not survive the larger set, which is a pretty normal outcome when thresholds get tuned on the same examples being scored. We kept JevK5 for suggested tags that a person can confirm and deleted the rest. Published benchmark wins on short classification tasks told us very little about long transcripts and our own tag definitions, and running our own examples first is what kept an interesting open model from quietly becoming an unsound auto-tagging system. The same properties describe where an open version should not be expected quickly. Frontier models need very large amounts of compute and data, and their behavior is open-ended, so reproducing them is not a matter of a readout trick and a fine-tune. We still use models from established frontier labs every day for difficult work under normal commercial terms. Hosted services can also be worth paying for (convenience and support are real things, and sometimes the hosted model is just better on your examples). An open implementation existing is not the same as that implementation being good enough for a particular workflow. Expecting a scoped capability to be copied is a planning assumption rather than a reason to refuse a tool that solves a problem today. It is most useful before a dependency is wired in (how describable is the job and how small is the output and do capable open base models already cover the hard part and how much of your operation would the input stream reveal if somebody saw all of it). ![Craftsman's hands weighing plain metal tokens against master brass weights at a dark assay table, with sorted trays on one side and discarded brass frames on the other.](https://ddls.co/images/blog-if-its-useful-it-wont-stay-closed-inline-2.webp) Planning with that assumption mostly means keeping the parts around a model in your own hands. Labels and their definitions come from your workflow. A set of real examples with correct answers is what lets you compare any two models on decisions you care about. The thresholds where a person reviews an output before anything consequential happens are judgments your business has to make either way. Keeping those pieces behind a simple API boundary leaves the model itself replaceable when a cheaper, better, or local version appears, and it avoids a predictable migration later. Models at this size now change faster than the workflows around them, so the workflow and its evaluation data are the parts worth treating as durable. I would not add a closed dependency from a new company for important and potentially sensitive decision-making when the model is small enough (and the job describable enough) that an open base model or somebody building on one is likely to cover it soon after the idea proves itself. Waiting briefly or starting with an open implementation and measuring it honestly against the closed one usually costs less than integrating first and moving later. If a scoped capability turns out not to be useful, the wait costs very little. If it turns out to be useful, recent experience suggests the open version will not be far behind. Contact: [Contact Daedalus](https://ddls.co/#contact). Email addresses are omitted from this export.