FLEXPERTO — FINTECH / INSURTECH / VIDEO CONSULTATION — 2022–2025
150% Throughput on Real-Time Transcription
I extended open source libraries with batching support and optimized it for NVIDIA H100 clusters using vLLM and custom CUDA kernels, achieving 150% higher throughput than the baseline.
I unified distributed state using Redis as the central source-of-truth, and decoupled the real-time transcription pipeline via async messaging. Supported by full-stack Grafana observability, this fault-tolerant distributed messaging eliminated REST bottlenecks and ensured zero data loss during high-load WebSocket streaming.
Leveraging my background in data science and low-level systems, I overhauled the applied AI pipeline. By integrating sub-symbolic models and clustering algorithms for live speaker diarization, and extending open-source components with Rust and custom CUDA optimizations for NVIDIA H100 clusters, I achieved 150% higher throughput than the baseline, due to a custom batching strategy.
For strict regulatory compliance, I engineered GDPR-aligned recording consent flows alongside cryptographic client-side watermarking and tamper protection.
APPLIED AI · CLUSTERING · DISTRIBUTED MESSAGING · REDIS · NVIDIA H100