oto

oto builds full-duplex conversational speech datasets for voice AI and humanoids, in English, Japanese, and Korean. Each speaker is recorded on a separate channel at the source, so overlap, backchannels, and interruptions stay in the audio instead of being mixed into one track.

Open datasets

The 141-hour set draws partly on the 280-hour set. The 104.94-hour set is hand-annotated with 17 turn-taking event labels.

Licensed datasets

otoSpeech 1,125h can be bought online. The other sets are available on request. Audio samples are on oto.earth.

• English: otoSpeech 1,125h, 1,125 hours from 3,878 sessions of casual two-speaker conversation. Sold in packs of 100, 250, 500, 750, 1,125 hours, from $10 per hour
• Japanese: free conversation, role-play, and scripted dialogue, 48 kHz
• Korean: free conversation on a given topic, 44.1 kHz, with ASR transcripts for both speakers
• Multi-speaker: English conversation among three to five people
• Video: two-speaker English conversation with one camera per speaker
• Egocentric: first-person video and conversation of two to four people folding towels, tidying objects, and playing Jenga

oto also collects datasets to specification: languages, task designs, turn-taking labels, multi-party, in-person, and video-grounded setups.

Where otoSpeech is used

• TurnBench: otoSpeech is the training set for TurnBench, Sesame's open turn-taking benchmark. oto researchers co-authored the paper.
• Research: papers including DualTurn, DuplexAct-Bench, and MultiPartyTurnBench train or evaluate on otoSpeech.
• Open models and tools: Pipecat's backchannel detector, LiveKit's eot-bench, and a Lhotse recipe for loading the data.

The full list has 19 entries, each with a link to its source.

Contact

oto.earth | [email protected]