Evaluating Multilingual LLMs at Scale
For Microsoft Research, Karya completed one of the largest multilingual human evaluations of LLMs within three weeks.
Our client is building a tool for agriculture helplines to understand conversations between farmers and agents. This tool will identify crop names, specific issues, and suggested solutions to improve communication and support.
Ensuring accurate transcription, especially with regional accents and background noise. Developing a robust system for recognizing different agricultural terms and entities in various linguistic contexts. Teaching the model to understand nuanced language and identify specific solutions proposed by farmers and tele-agents.
We have trained 50 Hindi speakers to transcribe and recognize agriculture-related terms. Using advanced tools for accurate understanding of spoken words. Improving the model's skill in identifying agricultural terms across different languages.
For Microsoft Research, Karya completed one of the largest multilingual human evaluations of LLMs within three weeks.
Collecting and validating conversational speech data across multiple domains in 11 languages for Microsoft