Chain-of-Thought (CoT) Concepts & Latent Reasoning

Language models cannot pause to 'think' during the generation of a single token; every token generation involves the exact same fixed amount of compute (a forward pass through the transformer weights).

Consequently, when a model is asked a complex logic or mathematical question in one shot, it is forced to guess the answer token immediately.

The Power of Token Scratchpads By prompting the model to: ```markdown Think step by step before outputting your final answer. First, list all constraints and verify each candidate solution in a <scratchpad> block. ``` We grant the model extra forward passes (more tokens) to encode intermediate calculations into the autoregressive context. With reasoning models like DeepSeek R1 and OpenAI o1/o3, this chain-of-thought is native, but applying explicit CoT scaffolds remains paramount for non-reasoning foundation models.