Build task-specific models from enterprise data and evaluate agent behavior
Ashr develops custom AI models for organizations whose important tasks depend on knowledge and judgments that are not captured well by a general-purpose model alone. Its current offering combines data preparation, post-training and task-specific evaluation, with engineers working in the customer's environment. The company also provides an agent-testing platform that can be used independently through documented SDKs.
The process starts with material such as documents, transcripts, logs and corrections made by reviewers. Ashr works with the organization to define what a correct result means, using an output schema, rubric or known answers. That definition becomes the basis for fine-tuning and reinforcement-learning work. The approach is task-specific: an extraction workflow, a support process and a research agent can require different data and acceptance criteria even when they use similar underlying language-model technology.
Ashr describes comparing a candidate model with the customer's existing API on a frozen set of the customer's own examples. Keeping that evaluation set separate from routine changes makes a comparison easier to interpret. Reviewer corrections can then feed later training iterations. The official offering emphasizes running the work in the customer's cloud and delivering model weights, but the exact deployment and ownership commitments should be included in the commercial agreement rather than inferred from a marketing summary.
The separate evaluation SDK supports scenario datasets, agent runs, expected-versus-actual comparisons and server-side grading. Python and TypeScript integrations are documented, including adapters for common agent providers. Production tracing is a distinct feature and requires the appropriate tenant enablement; it should not be assumed to be active simply because evaluations work. A team can therefore begin by defining one measurable failure pattern, build a representative test set and use the resulting evidence to decide whether prompt changes, tool changes or custom training are justified. Public prices for a custom model engagement are not specified, so scope and success criteria need discussion with the provider.
