Teaching small models to find the evidence that matters

“Can we approve this supplier under our current procurement policy?”

A search might return the supplier’s proposal, an onboarding checklist and several procurement guidelines. All are related to the question. The decisive passage could be a short exception covering suppliers that process customer data.

Answering well requires identifying which evidence determines the decision.

A small model trained to judge relevance

In Distillation and Refinement of Reasoning in Small Language Models for Document Re-ranking, Chris Samarinas and Hamed Zamani train a three-billion-parameter model using teacher-generated relevance explanations and further reward-based refinement.

Their model achieves strong results on the reasoning-intensive BRIGHT benchmark, outperforming some much larger models. Generating explanations during inference improves its ranking performance compared with directly predicting relevance scores.

For business search, this makes task-specific post-training particularly interesting.

Give each search stage a clear job

A practical search system can first retrieve a broad set of promising passages, then examine a smaller selection more closely.

In our procurement example, that second stage would assess whether each passage contains a relevant requirement, exception or approval condition. A document that mentions the supplier repeatedly may be less useful than a policy paragraph describing the exact situation.

Training for this judgment could use examples of business questions paired with helpful and misleading passages. Useful distinctions include current versus superseded policies, binding requirements versus suggestions, and customer-specific terms versus general templates.

These are concrete behaviours we can evaluate.

Reasoning needs a budget

A compact model still consumes time and compute when it generates explanations for multiple candidates. Its parameter count alone does not tell us the cost of a completed search.

For Traize, the useful engineering question is how much additional assessment a query needs. Straightforward requests may already produce clear results. Ambiguous questions involving several conditions may justify closer examination.

A meaningful evaluation would measure whether reranking improves the final evidence selection, how much latency it adds and how often it prevents another search. We would also account for the cost of preparing training data and updating the model.

Train for the decisions users actually make

Company search contains recurring judgments: which policy applies, which version is current, which passage supports a conclusion.

These judgments offer a focused target for specialised models. Better evidence selection could give the answering model a shorter, more useful set of sources and help employees reach an answer with fewer follow-up questions.

That is the opportunity we see for Traize: applying post-training to a specific part of the search process where improved judgment can translate directly into a better everyday experience.