💡 The paper introduces OmniFlatten, a novel end-to-end GPT model that enables real-time natural full-duplex spoken dialogue. The goal is to achieve low latency and natural interactions in full-duplex dialogue systems, which is a significant challenge due to human conversation dynamics such as interruptions, backchannels, and overlapping speech. To address this, the authors propose a multi-stage post-training technique that integrates speech and text without altering the original model's architecture. The training process consists of three stages: modality alignment, half-duplex dialogue learning, and full-duplex dialogue learning. A flattening operation is used to standardize the data, allowing for unified training methods and model architecture across different modalities and tasks. The OmniFlatten model can generate text and speech in real-time, effectively modeling complex behaviors inherent to natural conversations. The approach offers a straightforward modeling technique and a promising research direction for developing efficient and natural end-to-end full-duplex spoken dialogue systems. The results are demonstrated through audio samples of dialogues generated by OmniFlatten, which can be found online. Overall, the paper contributes to the development of full-duplex spoken dialogue systems that can mimic human-human interactions, with potential applications in various areas such as virtual assistants, customer service, and more.