AI voicemail detection for cold calling
- Twilio Programmable Voice
- OpenAI Whisper
- OpenAI
- CallHippo
- CRM
- Before
- ~150 dials a day · 15–20% picked up
- Action
- Verification call → Whisper transcript → small LLM: human or voicemail; queue of 8
- After
- 45–50% picked up
- Verified by
- Dialer pickup rate (self-reported)
The problem
SDRs worked their call lists from the CRM in a CallHippo dialer, making about 150 dials a day each. Only 15–20% of those calls were picked up by a person. The rest rang out or went to voicemail. That meant most of an SDR's day was spent listening to rings and voicemail greetings instead of talking to owners. I built the fix end to end, and it was still running when I left.
Constraints
- The SDR's time is the expensive part. Every dial that ends in voicemail costs a rep the ring time plus the greeting. The fix had to remove those dials before they reached a person.
- A human answer has to be recognised from a few words. An owner picking up says "hello" and not much else. A voicemail greeting says "please leave a message". The system has to tell the two apart from a few seconds of audio.
- Freshness matters. A number that someone has just answered is the best bet for a live conversation right now. The same number checked hours earlier is much less likely to be answered again.
How it works
- A short verification call first. The system places a call through Twilio Programmable Voice and records only the first few seconds of the answer. The SDR is not on this call.
- No answer means rejected. If nobody picks up, the number is dropped straight away and never reaches an SDR.
- Transcribe what answered. For an answered call, OpenAI's Whisper turns those few seconds into a transcript.
- Let a small LLM decide. A small, cheap ChatGPT model reads the transcript and labels it human or voicemail. "Hello? Hello?" is a person. "Please leave a message after the tone" is a voicemail.
- Route only people. A number that a person answered goes into the SDR's dialer queue. Voicemails don't.
- Keep the queue short and fresh. Each SDR holds at most 8 verified leads at a time. Every time an SDR dials one, the system qualifies another and tops the queue back up. Leads go to the SDR minutes after a person answered, not hours later.
Each lead gets a short verification call, recording only the first few seconds; unanswered calls are rejected, answered calls are transcribed and classified by a small LLM as human or voicemail, and only humans join an SDR queue capped at eight, which refills as the SDR dials.
Receipts
- Before: about 150 dials a day per SDR, with 15–20% picked up by a person.
- After: a 45–50% pickup rate on the calls SDRs made.
- These are self-reported figures from the dialer's pickup rate. There is no public dataset behind them.
What broke, and what I changed
Bulk batches went stale. The first version qualified leads in bulk: it verified 100 numbers, then sent the whole batch across to the SDRs. That's where I realised recency matters as much as the human-or-voicemail check. A number someone answered a minute ago is likely to be answered again now. By the time an SDR reached the bottom of a 100-lead batch, those answers were old.
So the queue became small and just-in-time. Each SDR now holds at most 8 verified leads. Every time they dial one, the system verifies another and tops the queue back up. An SDR is only ever routed leads with a high chance of being picked up, because every number they see was answered by a person moments earlier.