Conferences

AAAI

• Byte-level BPE, 2020. Neural Machine Translation with Byte-Level Subwords

ACL

• BPE, 2016. Neural Machine Translation of Rare Words with Subword Units
• Subword regularisation, 2018. Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates
• ULMFiT, 2018. Universal Language Model Fine-tuning for Text Classification
• Transformer-XL, 2019. Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
• Pruning heads, 2019. Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned
• BART, 2020. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
• Self-Instruct, 2023. Self-Instruct: Aligning Language Models with Self-Generated Instructions
• OLMo, 2024. OLMo: Accelerating the Science of Language Models
• DeepSeekMoE, 2024. DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
• NSA, 2025. Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention

AISTATS

• Xavier init, 2010. Understanding the Difficulty of Training Deep Feedforward Neural Networks

CVPR

• Contrastive loss, 2006. Dimensionality Reduction by Learning an Invariant Mapping
• FaceNet, 2015. FaceNet: A Unified Embedding for Face Recognition and Clustering
• ResNet, 2016. Deep Residual Learning for Image Recognition
• Latent Diffusion, 2022. High-Resolution Image Synthesis with Latent Diffusion Models
• MAE, 2022. Masked Autoencoders Are Scalable Vision Learners

EMNLP

• GRU encoder-decoder, 2014. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• GloVe, 2014. GloVe: Global Vectors for Word Representation
• Luong attention, 2015. Effective Approaches to Attention-based Neural Machine Translation
• SentencePiece, 2018. SentencePiece: A Simple and Language Independent Subword Tokenizer and Detokenizer for Neural Text Processing
• Sentence-BERT, 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
• How contextual?, 2019. How Contextual are Contextualized Word Representations?
• QK-norm, 2020. Query-Key Normalization for Transformers
• DPR, 2020. Dense Passage Retrieval for Open-Domain Question Answering
• Prompt tuning, 2021. The Power of Scale for Parameter-Efficient Prompt Tuning
• SimCSE, 2021. SimCSE: Simple Contrastive Learning of Sentence Embeddings
• GQA, 2023. GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
• Vec2Text, 2023. Text Embeddings Reveal (Almost) As Much As Text

ICCV

• He init, 2015. Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
• Swin Transformer, 2021. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

ICLR

• VAE, 2014. Auto-Encoding Variational Bayes
• VGG, 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition
• Adam, 2015. Adam: A Method for Stochastic Optimization
• Bahdanau attention, 2015. Neural Machine Translation by Jointly Learning to Align and Translate
• SGDR, 2017. SGDR: Stochastic Gradient Descent with Warm Restarts
• Sparsely-gated MoE, 2017. Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
• Sharp minima, 2017. On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima
• Mixed precision, 2018. Mixed Precision Training
• Don’t decay the LR, 2018. Don’t Decay the Learning Rate, Increase the Batch Size
• AdamW, 2019. Decoupled Weight Decay Regularization
• Performer, 2021. Rethinking Attention with Performers
• ViT, 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• ALiBi, 2022. Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation
• S4, 2022. Efficiently Modeling Long Sequences with Structured State Spaces
• FLAN, 2022. Finetuned Language Models Are Zero-Shot Learners
• LoRA, 2022. LoRA: Low-Rank Adaptation of Large Language Models
• BEiT, 2022. BEiT: BERT Pre-Training of Image Transformers
• Self-consistency, 2023. Self-Consistency Improves Chain of Thought Reasoning in Language Models
• GPTQ, 2023. GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
• ReAct, 2023. ReAct: Synergizing Reasoning and Acting in Language Models
• YaRN, 2024. YaRN: Efficient Context Window Extension of Large Language Models
• FlashAttention-2, 2024. FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning
• Gated DeltaNet, 2025. Gated Delta Networks: Improving Mamba2 with Delta Rule

ICML

• Gradient clipping, 2013. On the Difficulty of Training Recurrent Neural Networks
• BatchNorm, 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Adapters, 2019. Parameter-Efficient Transfer Learning for NLP
• Pre-LN, 2020. On Layer Normalization in the Transformer Architecture
• Linear attention, 2020. Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention
• SimCLR, 2020. A Simple Framework for Contrastive Learning of Visual Representations
• Alignment & uniformity, 2020. Understanding Contrastive Learning through Alignment and Uniformity on the Hypersphere
• CLIP, 2021. Learning Transferable Visual Models From Natural Language Supervision
• DALL·E, 2021. Zero-Shot Text-to-Image Generation
• Speculative decoding, 2023. Fast Inference from Transformers via Speculative Decoding
• Platonic Representation Hypothesis, 2024. The Platonic Representation Hypothesis

MLSys

• AWQ, 2024. AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

NAACL

• Relative position, 2018. Self-Attention with Relative Position Representations
• ELMo, 2018. Deep Contextualized Word Representations
• BERT, 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• Augmented SBERT, 2021. Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Tasks

NeurIPS

• AlexNet, 2012. ImageNet Classification with Deep Convolutional Neural Networks
• word2vec, 2013. Distributed Representations of Words and Phrases and their Compositionality
• seq2seq, 2014. Sequence to Sequence Learning with Neural Networks
• GAN, 2014. Generative Adversarial Nets
• Transformer, 2017. Attention Is All You Need
• Loss landscape, 2018. Visualizing the Loss Landscape of Neural Nets
• RMSNorm, 2019. Root Mean Square Layer Normalization
• DDPM, 2020. Denoising Diffusion Probabilistic Models
• GPT-3, 2020. Language Models are Few-Shot Learners
• RAG, 2020. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• μP, 2021. Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer
• Chinchilla, 2022. Training Compute-Optimal Large Language Models
• InstructGPT, 2022. Training Language Models to Follow Instructions with Human Feedback
• FlashAttention, 2022. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
• LLM.int8(), 2022. LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale
• Chain-of-Thought, 2022. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
• Zero-shot CoT, 2022. Large Language Models are Zero-Shot Reasoners
• QLoRA, 2023. QLoRA: Efficient Finetuning of Quantized LLMs
• Unfaithful CoT, 2023. Language Models Don’t Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting
• Tree of Thoughts, 2023. Tree of Thoughts: Deliberate Problem Solving with Large Language Models
• Reflexion, 2023. Reflexion: Language Agents with Verbal Reinforcement Learning
• LLM-as-a-Judge, 2023. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
• Toolformer, 2023. Toolformer: Language Models Can Teach Themselves to Use Tools
• LLaVA, 2023. Visual Instruction Tuning
• FineWeb, 2024. The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale
• Beyond fixed durations, 2024. Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations
• SGLang, 2024. SGLang: Efficient Execution of Structured Language Model Programs
• SWE-agent, 2024. SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering

OSDI

• Orca, 2022. Orca: A Distributed Serving System for Transformer-Based Generative Models

SC

• ZeRO, 2020. ZeRO: Memory Optimizations Toward Training Trillion Parameter Models

SIGIR

• ColBERT, 2020. ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT

SOSP

• PagedAttention, 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention

STOC

• LSH, 1998. Approximate Nearest Neighbors: Towards Removing the Curse of Dimensionality

Journals

Found. Trends IR

• BM25, 2009. The Probabilistic Relevance Framework: BM25 and Beyond

IEEE TBD

• Faiss, 2019. Billion-Scale Similarity Search with GPUs

IEEE TPAMI

• PQ, 2011. Product Quantization for Nearest Neighbor Search
• HNSW, 2018. Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs

JMLR

• Neural LM, 2003. A Neural Probabilistic Language Model
• Adagrad, 2011. Adaptive Subgradient Methods for Online Learning and Stochastic Optimization
• Dropout, 2014. Dropout: A Simple Way to Prevent Neural Networks from Overfitting
• T5, 2020. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• Switch Transformer, 2022. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
• PaLM, 2023. PaLM: Scaling Language Modeling with Pathways

Neural Computation

• LSTM, 1997. Long Short-Term Memory

Proc. IEEE

• LeNet, 1998. Gradient-Based Learning Applied to Document Recognition

TACL

• Lost in the middle, 2024. Lost in the Middle: How Language Models Use Long Contexts

Preprints & reports

arXiv

• Distillation, 2015. Distilling the Knowledge in a Neural Network
• LayerNorm, 2016. Layer Normalization
• GELU, 2016. Gaussian Error Linear Units (GELUs)
• Large minibatch SGD, 2017. Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• Online softmax, 2018. Online normalizer calculation for softmax
• CPC, 2018. Representation Learning with Contrastive Predictive Coding
• Sparse Transformer, 2019. Generating Long Sequences with Sparse Transformers
• MQA, 2019. Fast Transformer Decoding: One Write-Head is All You Need
• RoBERTa, 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach
• Megatron-LM, 2019. Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
• SwiGLU, 2020. GLU Variants Improve Transformer
• Scaling laws, 2020. Scaling Laws for Neural Language Models
• RoPE, 2021. RoFormer: Enhanced Transformer with Rotary Position Embedding
• Induction Heads, 2022. In-context Learning and Induction Heads
• Constitutional AI, 2022. Constitutional AI: Harmlessness from AI Feedback
• E5, 2022. Text Embeddings by Weakly-Supervised Contrastive Pre-training
• Mamba, 2023. Mamba: Linear-Time Sequence Modeling with Selective State Spaces
• Llama 2, 2023. Llama 2: Open Foundation and Fine-Tuned Chat Models
• LLaMA, 2023. LLaMA: Open and Efficient Foundation Language Models
• Mistral 7B, 2023. Mistral 7B
• u-μP, 2024. u-μP: The Unit-Scaled Maximal Update Parametrization
• Llama 3, 2024. The Llama 3 Herd of Models
• DeepSeek-V2, 2024. DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
• DeepSeek-V3, 2024. DeepSeek-V3 Technical Report
• Test-time compute, 2024. Scaling LLM Test-Time Compute Optimally Can Be More Effective than Scaling Model Parameters
• Mixtral, 2024. Mixtral of Experts
• τ-bench, 2024. τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
• Qwen3, 2025. Qwen3 Technical Report
• Kimi K2, 2025. Kimi K2: Open Agentic Intelligence
• Muon, 2025. Muon is Scalable for LLM Training
• s1, 2025. s1: Simple Test-Time Scaling
• DeepSeek-R1, 2025. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
• vec2vec, 2025. Harnessing the Universal Geometry of Embeddings

OpenAI report

• GPT-1, 2018. Improving Language Understanding by Generative Pre-Training
• GPT-2, 2019. Language Models are Unsupervised Multitask Learners
• GPT-4, 2023. GPT-4 Technical Report