The "tl;dr" on a few notable transformer papers (pre-2022). - will-thompson-k/tldr-transformers

Implementation of Fast Transformer in Pytorch. Contribute to lucidrains/fast-transformer-pytorch development by creating an account on GitHub.

Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch - lucidrains/RETRO-pytorch

Implementation of the specific Transformer architecture from PaLM - Scaling Language Modeling with Pathways - lucidrains/PaLM-pytorch

Implementation of 🦩 Flamingo, state-of-the-art few-shot visual question answering attention net out of Deepmind, in Pytorch - lucidrains/flamingo-pytorch

Implementation of CoCa, Contrastive Captioners are Image-Text Foundation Models, in Pytorch - lucidrains/CoCa-pytorch

Implementation of Parti, Google's pure attention-based text-to-image neural network, in Pytorch - lucidrains/parti-pytorch

Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Research, in Pytorch - lucidrains/audiolm-pytorch

Implementation of Make-A-Video, new SOTA text to video generator from Meta AI, in Pytorch - lucidrains/make-a-video-pytorch

Implementation of RT1 (Robotic Transformer) in Pytorch - lucidrains/robotic-transformer-pytorch

Implementation of Muse: Text-to-Image Generation via Masked Generative Transformers, in Pytorch - lucidrains/muse-maskgit-pytorch

Implementation of MusicLM, Google's new SOTA model for music generation using attention networks, in Pytorch - lucidrains/musiclm-pytorch

Implementation of Toolformer, Language Models That Can Use Tools, by MetaAI - lucidrains/toolformer-pytorch

Implementation of Recurrent Memory Transformer, Neurips 2022 paper, in Pytorch - lucidrains/recurrent-memory-transformer-pytorch

Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch - lucidrains/MEGABYTE-pytorch

Implementation of SoundStorm, Efficient Parallel Audio Generation from Google Deepmind, in Pytorch - lucidrains/soundstorm-pytorch

Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens" - kyegomez/LongNet

Implementation of MeshGPT, SOTA Mesh generation using Attention, in Pytorch - lucidrains/meshgpt-pytorch

A novel implementation of fusing ViT with Mamba into a fast, agile, and high performance Multi-Modal Model. Powered by Zeta, the simplest AI framework ever. - kyegomez/MultiModalMamba

Quantized Attention that achieves speedups of 2.1-3.1x and 2.7-5.1x compared to FlashAttention2 and xformers, respectively, without lossing end-to-end metrics across various models. - thu-ml/SageAt...