Subjective evaluation for your conversational AI models
Defined.ai offers a suite of subjective tests to help you evaluate your Text-to-Speech (TTS) models, utilizing live, human-in-the-loop assessment. Integrate any or all of our select EoE services into your AI pipeline via API, or as a managed service, to ensure that your conversational AI models reflect customer reality. Leverage insights and metrics towards tuning and iterating your AI models to maximize customer satisfaction.
Mean opinion score
Our MOS testing service benchmarks and evaluates your TTS voice performance and output to give you actionable insights for model tuning and retraining.
Comprehensive evaluation
Obtain a comprehensive and fair evaluation of the voice output of your speech models with our unique synthesis of automated and live human assessments. 
Meaningful metrics
Mean scores for model outputs reflect reality better than single evaluations by providing numeric representations of evaluator agreement and overall assessment trend rates. 
A quality crowd
Our vetted community of crowd workers—500k strong data veterans and growing—guaranteeing quality and guarding against novice evaluators and outlier opinion endemic to competing platforms. ABx
Compare outputs between AI model iterations with direct, head-to-head preference testing to validate statistically significant improvements between versions, and to determine user preferences between models.
Actionable insights
ABx testing between model versions is straightforward with open and clear results. 
Robust to bias
Randomized presentation of AB pairs (A à B vs B à A) across users reduces ordering bias 
Ideal for benchmarking
Benchmark and compare which model iterations are best or how they stack up against popular, renowned models. Pronunciation validation
Create and evaluate best-in-class speech recognition models by conducting pronunciation and fluency assessments.
Native fluency
Native speakers judge pronunciation and fluency, providing realistic and authentic expertise, meaning your AI speech models are understandable and natural in their target languages. 
Expand into new markets
Expand into new markets with confidence that your models will sound natural thanks to training data vetted by fluent, native language evaluators. Looking to get started right away?
Try our API templates
Leverage our reliable and scalable REST APIs to submit common requests like MOS/CSAT surveys, A/B tests, and PRV evaluations. Oversee the entire process in a precise and time-efficient setting.
Evaluation of Experience in action
How can we help?
Looking for a custom workflow to evaluate your TTS voice models?
Contact us. We’d love to hear from you.
Contact us. We’d love to hear from you.
