💡 The paper introduces Continuous Audio Language Models, a new approach to audio generation that addresses the limitations of traditional discrete audio language models. Discrete models represent audio as sequences of discrete tokens, which are extracted from lossy codecs with limited bitrate, resulting in a trade-off between audio quality and computational cost. To overcome this issue, the authors propose Continuous Audio Language Models, which instantiate a large Transformer backbone that produces a contextual embedding at every time step. This sequential information then conditions a multilayer perceptron to generate the next continuous frame of an audio Variational Autoencoder through consistency modeling. By avoiding lossy compression, Continuous Audio Language Models achieve higher quality at lower computational cost than their discrete counterparts. Experiments on speech and music demonstrate improved efficiency and fidelity over state-of-the-art discrete audio language models, facilitating lightweight, high-quality audio generation. The approach enables the generation of high-quality audio samples, which are made available for demonstration purposes. Overall, the paper contributes a novel method for continuous audio language modeling, which has the potential to improve the efficiency and quality of audio generation tasks.