We Tested Two Small Local AI Models. They Still Need Supervision
We tested K2 Horizon 7B and Ornith 1.5 9B as local AI agents. Both could use tools, but failures exposed risks businesses should understand.
An AI supplier shows a compact model running on equipment your company controls. It follows instructions, uses software tools and completes a prepared task without sending company data to a large cloud service. The obvious conclusion is that a private AI agent is ready to work for your business.
A demonstration covers the easy part. The buying decision depends on what happens when progress stops, information is missing or the model should hand the task to a person.
We tested two recent small models, K2 Horizon 7B and Ornith 1.5 9B, inside local agent systems. Both showed useful capabilities. Both also exposed a gap between being able to take actions and being reliable enough to take responsibility for a business process. K2 performed better overall in our runs, but neither model was ready to work without firm limits and supervision.
Quick answer
Small local models can already understand an objective, choose tools and complete short, predictable work. Their weak point appears when the expected route fails. In our tests, the models repeated actions without making progress, continued after a stop instruction and, in one case, added unsupported details after correctly admitting that the required information was unavailable.
Local AI agents can still be useful, provided the model is not the only control mechanism. A company also needs software that limits attempts, checks important facts, records what happened and sends uncertain cases to an employee. A supplier selling “agentic AI” should be able to show those controls as well as a successful demonstration.
Why we tested K2 and Ornith
K2 Horizon 7B and Ornith 1.5 9B belong to a new group of models small enough for local use but promoted for reasoning, software tools and multi-step work. That combination is attractive to companies that want more control over data and operating costs without giving up automation.
The two models are part of a broader market direction. Qwen3.5-9B, LFM2.5-8B-A1B, Ling 3.0 tiny and Granite 4.2 8B have also been released or positioned for local reasoning and agent work during 2026. These names provide market context rather than recommendations: capable local agents are moving quickly into mainstream product marketing.
Our goal was practical. We checked whether the models could follow an objective, use the available tools, recognise when progress had stopped, respect a clear stopping rule and keep unverified information out of the final answer.
What the models could already do
Both models understood the instructions and could use tools correctly when the work was short and the conditions were predictable. They retained the overall objective and produced sensible next steps. These abilities are real, and they explain why a polished demonstration can be convincing.
K2 was the more disciplined of the two in our observed runs. It generally reached useful actions faster and used fewer resources. Ornith took longer and produced more activity without a clean result when the work became difficult. The different tasks and conditions prevent a general model ranking. The comparison still shows that choosing the stronger model does not remove the need to manage failure.
Where reliability broke
When the normal route stopped working, the models kept the objective in mind and continued producing actions that looked reasonable one at a time. The sequence as a whole no longer moved towards a verified result.
This behaviour can be difficult to spot during a sales demonstration. The agent remains active and explains what it is doing, so it appears to be working through the problem. A company may instead receive a delayed task, a larger computing bill and an employee who still has to determine whether anything useful was achieved.
Businesses should measure verified progress rather than visible activity. An agent should either produce a result supported by evidence or stop with a clear explanation that another person can act on.
One weakness concerned the ability to stop. In a K2 run, we instructed the agent to report the obstacle and end the task after the second failed attempt. It continued trying.
Many AI proposals describe rules written in the prompt as safety controls. Those rules can influence behaviour, but the model still interprets them every time it chooses the next action. A request to stop is therefore weaker than a fixed limit in the surrounding software.
For a business, the difference is practical. If a supplier promises a maximum number of attempts, the system should end the run when that number is reached. If an action needs approval, the software should withhold permission until an authorised person provides it. Important limits need to work even when the model makes a poor decision.
Another K2 run revealed a more serious problem. The information needed to finish the work was unavailable. The agent recognised the obstacle and said so clearly, then added details it had not verified. Some were wrong.
The answer sounded responsible because it openly acknowledged the limitation. That tone made the unsupported details easier to trust. A manager reading a long report could reasonably assume that anything following the warning had still been checked.
For business use, admitting uncertainty is only the first step. The system must preserve that boundary through the whole answer. When evidence is missing, the corresponding fields should remain empty or be marked for review. A polite disclaimer does not protect the company from a confident error elsewhere in the same output.
What a benchmark does not settle
The official K2 release reports strong results in coding, reasoning and agent benchmarks. IFM also publishes more supporting material than many model makers, including weights, code, training information and detailed evaluations. The reported progress is credible, and our controlled runs confirmed that a small model can complete bounded work cleanly.
The same release acknowledges that extensive exploration and repeated recovery after errors remain difficult for the smallest models. That limitation matters because business processes rarely stay inside a prepared demonstration.
A benchmark is a standardised test under defined conditions. It can show that a model has a capability, but a buyer still needs evidence that the complete system behaves properly when conditions change.
| A product claim may say | A business still needs to verify |
|---|---|
| Strong performance on agent tasks | Whether the system recognises a dead end and stops |
| Can complete multi-step work | Whether later actions still contribute to the result |
| Can use software tools | Whether it checks the information those tools return |
| Follows instructions | Whether important limits are enforced outside the model |
| Can explain a failure | Whether the final answer remains free of invented details |
The score does not have to be false for the sales conclusion to be too broad. A model can perform well in the published test and still need substantial supervision in a company process.
What this means for a business
K2 and Ornith are capable components. Neither should be treated as a finished autonomous employee. Small local models make the most sense when the work has clear boundaries, access is limited, important facts can be checked automatically and a person handles exceptions.
Risk rises when an agent receives broad permissions, an open-ended objective and no reliable definition of success or failure. In that situation, a fluent model can keep working long after useful progress has stopped. It can also turn missing information into an answer that looks complete.
The supplier is therefore responsible for more than selecting a model. The proposal should explain how the system stops repeated attempts, protects important data, records the source of its answers, requests approval and returns unfinished work to an employee. These controls are part of the product and part of its cost.
Before approving a local AI agent, ask the supplier:
- What happens when the model cannot complete the task?
- Which limits are enforced by software rather than written only in the prompt?
- Can an employee trace important facts back to their source?
- Which actions always require human approval?
- Who maintains these controls when the model or connected software changes?
The answers are more useful than another perfect demonstration. They show whether the supplier has built a dependable business system or only connected a capable model to a set of tools.
Our broader guide explains whether an SME needs an AI agent at all. If the proposal includes running the model on company equipment, review the cost and operating trade-offs of local LLMs. Our AI and process automation work starts with the process, its risks and the people responsible for it before any model receives access.
Our conclusion after testing both models
K2 and Ornith showed that small local models can already do meaningful work with software tools. They also showed that action and autonomy are different levels of capability. K2 was stronger in our trials, yet it still continued past a stopping rule and produced unsupported information when evidence was missing.
Businesses can use small local models with accurate expectations. Treat the model as one part of a controlled process, test failure before granting wider access and make a person responsible for exceptions. We ran those tests because the difficult part of an AI agent appears after the prepared demonstration ends.
Sources and method
- IFM: Introducing K2 Horizon, published 3 September 2026.
- K2 Horizon 7B model card, checked 7 September 2026.
- Ornith 1.5 9B model card, checked 7 September 2026.
- Qwen3.5-9B, LFM2.5-8B-A1B, Ling 3.0 tiny and Granite 4.2 8B model cards, checked 7 September 2026.
The observations come from Taronja Nova implementation trials with one observed run per condition. K2 was tested under more conditions than Ornith, and the tasks differed. The findings are therefore qualitative and do not establish a general failure rate for either model.