Reading Log

Library

A reading log of the AI papers, books, and resources that shaped how I build and research, each with a short note on what it introduced.

Paper / Transformers / Read

Attention Is All You Need

Vaswani et al. (2017)

Introduces the Transformer: self-attention without recurrence or convolution. The backbone of nearly every modern LLM.

Paper / Pretraining / Read

BERT: Pre-training of Deep Bidirectional Transformers

Devlin et al. (2018)

Masked-language-model pretraining for bidirectional context. Reframed NLP around pretrain-then-finetune.

Paper / Scaling / Read

Language Models are Few-Shot Learners (GPT-3)

Brown et al. (2020)

Showed that scale alone unlocks in-context / few-shot learning without gradient updates at inference.

Paper / RAG / Reference

Retrieval-Augmented Generation for Knowledge-Intensive NLP

Lewis et al. (2020)

The original RAG formulation: combine a parametric LLM with a non-parametric retrieval index. Directly relevant to the chatbots I ship.

Paper / Scaling / Read

Training Compute-Optimal Large Language Models (Chinchilla)

Hoffmann et al. (2022)

Corrected the scaling laws: most large models were undertrained on data relative to their parameter count.

Paper / Fine-tuning / Reading

LoRA: Low-Rank Adaptation of Large Language Models

Hu et al. (2021)

Freezes pretrained weights and injects trainable low-rank matrices, so you can fine-tune huge models on modest hardware.

Paper / GNNs / Reference

A Comprehensive Survey on Graph Neural Networks

Wu et al. (2019)

Map of the GNN landscape, and background for my UNSAT-core detection research on bipartite clause graphs.

Book / Foundations / Reference

Deep Learning

Goodfellow, Bengio, Courville (2016)

The standard reference for the math and intuition behind deep nets: optimization, regularization, representation.