What is reinforcement learning?
Learn from what happens next.

An agent learns a policy by acting in an environment.

At each step, the agent receives an observation, chooses an action, and gets feedback from the environment: a new observation and a reward signal. A policy is the rule or model used to choose actions. The learning objective is often to improve the total reward accumulated over time.

1. Observe. The system receives the information available about its current situation.
2. Choose. Its policy selects an action from the options the environment permits.
3. Experience. The environment changes and returns a reward signal with a new observation.
4. Update. A learning algorithm uses experience to improve later decisions.

Rewards can arrive long after an action. In a maze, a learner might receive feedback only when it reaches a destination; it then has to estimate which earlier turns helped. This is the credit-assignment problem: connect delayed outcomes to the decisions that contributed to them.

When the objective values outcomes over time, the learning target is a return: accumulated rewards across later steps. Algorithms can discount future rewards, which gives earlier outcomes more weight, or use another objective suited to the task.

There may be no label telling the learner which action was right.

A static dataset may be enough for a prediction task; interactive learning is useful when decisions affect what happens next. It also requires a safe, meaningful environment for collecting experience.

Different algorithms learn from the loop in different ways.

Deep Q-Networks (DQN) estimate the expected future return for possible actions, then use those estimates to choose among them. Mnih and colleagues demonstrated this approach on Atari games, where the agent received screen pixels and selected joystick actions. Proximal Policy Optimization (PPO) instead updates a policy directly from experience collected in the environment. Both are reinforcement-learning algorithms; they differ in what they learn and how they update it.

In robotics and control, the observations, action space, and environment can look very different from a game. A simulator can provide many low-cost trials, but a simulated score only describes the simulated conditions until transfer to a real setting is tested.

Reinforcement learning from human feedback (RLHF) applies related ideas to language-model post-training. In one common pipeline, human comparisons train a reward model, then an optimization stage uses that signal to update the assistant. This does not mean a person approves every generated response, and preference feedback is not a complete measure of truth, safety, or usefulness.

For the steps, alternatives such as DPO, and the limits of preference data, read how human feedback shapes AI assistants ↗. For the InstructGPT research example, see the original paper ↗. For sequential decisions in product workflows, continue with what an AI agent is ↗.

The agent does not always need to explore while it learns.

In online reinforcement learning, the policy gathers new experience by interacting with an environment during training. That can be useful when the agent can safely try actions and learn from their outcomes, but it may be expensive or unsafe to explore on a real system.

In offline reinforcement learning, a policy is learned from a fixed dataset of earlier interactions, such as logs, demonstrations, or experience from several policies. D4RL is a research benchmark built to test this setting with datasets from hand-designed controllers, human demonstrations, and mixed policies. A central challenge is coverage: the new policy may propose actions the dataset barely represents, so its value is difficult to estimate from those records alone. Read the D4RL paper ↗

Offline data can reduce the need to experiment directly in a costly or safety-critical environment, but it does not remove the need for evaluation. Check the data’s source and coverage, compare the policy with appropriate baselines, and test its behavior under conditions that differ from the training records.

For an overview of environment tooling used in reproducible experiments, see the Gymnasium documentation ↗.

Optimizing the signal can expose a gap in the goal.

A reward function translates a goal into a measurable signal. If the signal captures only a proxy, a learner may find actions that score well while missing the intended outcome. It can also exploit bugs in a simulator or fail when a real environment differs from training.

1. Define outcomes with care. State whose interests the reward is meant to represent and what it leaves out.
2. Test edge cases. Look for ways a policy can maximize the signal without doing the intended task.
3. Limit the environment. Use safe boundaries and a clear stop condition during training and evaluation.
4. Test outside training. Evaluate different conditions, rare events, and shifts between simulation and use.
5. Keep human oversight meaningful. Make consequential actions reviewable and provide a way to interrupt or correct them.

A strong reward score is evidence about the chosen objective and environment. It is not, on its own, evidence of broad competence or safe behavior. For the wider question of testing intended behavior, see AI alignment and evaluation ↗.

Connect learning methods to the system around them.

Reinforcement learning changes a policy through feedback. In a deployed system, permissions, interfaces, monitoring, user correction, and the definition of success still matter. Explore the AI field guide ↗, or compare learning with retrieval in RAG or fine-tuning ↗.