Constitutional AI: Harmlessness from AI Feedback

Why this one

Read this after Training language models to follow instructions with human feedback. RLHF says, roughly, collect human preferences and optimize the model toward them. Constitutional AI asks what happens when the expensive part is not the optimizer but the supervision itself. The key idea is not that a written constitution magically makes a model moral. It is that rules can become a reusable interface for generating critiques, revisions, and preference data at scale. People often talk about this as self alignment, which hides the engineering choice: humans still choose the principles, the data distribution, the evaluation, and the failure budget. The paper is worth your evening because it turns alignment from a vague values debate into a pipeline you can inspect. If you are fine tuning assistants, this is where harmlessness becomes something you can build, test, and break.