Block's new AI model Dessa was trained entirely on human speech to catch emotions text-based models miss

Block, the company behind Square and Cash App, has introduced Dessa, a speech model that processes audio directly instead of converting it to text first. Dessa is designed to recognize emotions, sarcasm, and pauses in real time. The model runs locally on a MacBook Pro M4 Max but isn't publicly available yet. The article Block's new AI model Dessa was trained

The top 3

  1. Architectural Shift: Audio-Native Processing vs. Text-to-Speech: Unlike conventional models that convert speech to text to perform inference, Dessa processes audio directly. By bypassing the text intermediate, the model is designed to preserve and interpret raw acoustic features like laughter, sarcasm, and rhythm that are typically lost during transcription.
  2. Training Methodology and Hardware Constraints: Dessa was built from scratch without the use of text data, relying on an audio-only training paradigm. The model was trained on an NVIDIA GB200 NVL72 and demonstrates sufficient efficiency to run locally on hardware as consumer-accessible as the MacBook Pro M4 Max, enabling real-time full-duplex interaction.

Sources

Open the full topic