Evaluating speech quality: WER is not enough

Named entities, numbers and intent success matter more to users.

Late to the thread, but evaluating beyond WER changed how I work. No exaggeration.

Good questions. Answering one by one below.

Thanks everyone, this is really helpful!

Thanks everyone, this is really helpful!

Maya_Brooks, ssato, dmenon and 10 others like this.

Great points. I'll try the suggestions tonight and report back.

172

What breaks first when you push evaluating beyond WER hard? 🎉

Elif_D35, noor-m, cokafor and 13 others like this.

@imran-k Code snippet, should work with minimal changes:

async def stream(frames):
    async for chunk in frames:
        yield vad(chunk)

Yui_Tanaka likes this.

Clean explanation. I'll try evaluating beyond WER tonight and report back.

XIRO_Team likes this.

Update: solved it. Thanks all 🙌

@julesM Evaluating beyond WER: five stars from me ⭐ would use again

Facts.

oliverH, oreed, lsilva and 5 others like this.

@sungminP Agreed 100%