Multi-System · Real-Time · Priority Routing
Voicemail-to-Slack Notification Pipeline
Customer Service Operations · Active-Client Prioritization · Principal Architect
- Python
- Gmail API + OAuth
- CRM cross-reference
- Slack Block Kit + audio uploads
- Self-healing backfill
- Atomic JSON state writes
- Asterisk PBX integration
- Multi-subsystem cycle resilience
Problem
The firm's CRM-integrated dialer had no native voicemail surface. Voicemails dropped into a generic Asterisk PBX inbox as email attachments, where they sat unread for hours — sometimes days. Worst case: a missed call from an active policy holder calling about coverage rolling into a no-policy day-old summary, costing the firm a renewal and a relationship. The customer service floor needed live-time visibility into every voicemail AND a way to instantly know which callers were existing clients vs. cold leads.
Architectural Solution
A polling pipeline that reaps PBX-generated voicemail emails from Gmail (OAuth-scoped read-only), extracts the caller's phone number, cross-references the CRM to find the matching lead, then queries policy state to determine priority. Active-policy callers trigger live Slack alerts with the actual voicemail audio auto-uploaded as a threaded reply — the recording plays inline in the channel, no inbox-diving, no link-clicking, no app-switching required. No-policy callers are silenced during business hours and rolled into a single morning digest the next day; long-form messages in the digest still surface their audio as standalone posts so meaningful voicemails aren't lost, while hang-ups under 10 seconds (duration parsed directly from RIFF/WAVE headers, including PCM, μ-law, and A-law encodings) are auto-suppressed. A self-healing auto-backfill pass runs at the end of every cycle: it scans recent channel history, identifies any alert thread missing its audio attachment, and re-uploads the recording from Gmail — so a transient Slack upload failure never costs the team a voicemail. A separate CLI flag (`--backfill --backfill-days N`) lets ops manually catch up larger windows after extended outages. Two delivery modes selectable by config: webhook-only for simpler text-only deployments, or bot-token mode for full audio attachment + threaded replies. Every external system is wrapped in retry logic with exponential backoff, and the three cycle subsystems (live alerts, morning summary, auto-backfill) run in independent try blocks so one failing doesn't take down the others.
Tangible Result
Customer service team gets actionable voicemail visibility in near-real-time. Priority callers' audio recordings play directly inside Slack — agents listen to the voicemail in the same window where they're already working, then call back the same hour rather than the next morning. The morning digest replaces what used to be a manual inbox-triage ritual: one Slack thread, sorted, deduped, hang-ups silently suppressed, long-form messages surfaced individually with their audio. The self-healing pass means alert threads end up with their recording attached even when an upstream service hiccups, and the manual backfill mode lets ops recover from any extended outage in a single command. The CRM didn't gain a voicemail feature — but operationally, it's like it did.
Where it scales
Polling cadence (15 min default), Gmail page size (500 per call, 20-page ceiling), and business-window logic (Monday auto-pulls weekend gaps) are all configuration. Both Gmail and CRM API calls are wrapped in retry-with-jitter; transient failures don't drop voicemails and aren't marked processed until they succeed. Atomic JSON writes on the processed-IDs ledger guarantee state is never corrupted by a mid-write process kill. The self-healing backfill pass auto-detects missing OAuth scopes and degrades quietly (warns once per session instead of spamming), so deployment misconfigurations are visible but don't drown the logs.
Where it mutates
Slack delivery has two modes selectable by env var: webhook-only (text alerts, simpler deployments) or bot-token mode (audio attachments + threaded replies, full-fidelity delivery). The CRM phone-column lookup is a list — adding a new field to the search is a one-line change. Hang-up threshold, summary post time, active-policy criteria, and backfill window size are all top-of-file constants editable in seconds. CLI flags toggle one-shot backfill mode for ad-hoc recovery without redeployment.
voicemail_pipeline.py
# Cross-reference caller against CRM, route by active-policy statusdef route_voicemail(caller_phone, message_id, audio_blob): lead = find_lead_by_phone(caller_phone) if not lead: # No matching lead — defer to morning summary return queue_for_summary(message_id, audio_blob) active_policies = find_active_policies(lead["lead_id"]) if active_policies: # Priority caller — post live with audio attached return post_priority_alert(lead, active_policies, audio_blob) # Cold lead — defer to summary, audio dropped if hang-up return queue_for_summary(message_id, audio_blob) # Retry with jitter — transient failures don't lose voicemailsdef tld_get_with_retry(endpoint, payload, max_retries=4): for attempt in range(max_retries): try: return requests.get(endpoint, ...) except (RequestException, HttpError) as e: if attempt == max_retries - 1: raise time.sleep(BASE_DELAY * 2**attempt + random.uniform(0, 0.5))