---
title: Build RL environments with REA
url: https://rea.tools/guides/rl-environments/
---

[Home](https://rea.tools/) / [Guides](https://rea.tools/guides/) / RL environments

# Build RL environments.

Turn a recorded web app session into a runnable environment for
training and testing AI agents.

[Watch the announcement video on X](https://x.com/reatools/status/2112574822584918075?s=20)

## What REA builds

An agent can practice finding an issue, creating a folder, or
editing a record without running the original app. REA builds the
tools the agent calls, the local data those tools use, and a basic
test that checks the environment starts and responds.

1. 01 · Record

   Use a test app

   Capture requests, responses, and the results of
   changes.
2. 02 · Build with REA

   Create tools and local state

   Export a runnable AgentEnv folder with the recording and a
   test.
3. 03 · Run with AgentEnv

   Give the environment to an agent

   Add task goals and reward checks for evaluation or RL
   training.

The recording determines which operations the environment can
support. Changes are supported when the recording confirms their
effects.

## Before you start

[Set up REA with your coding agent](https://rea.tools/get-started/). To record an app, have a Chrome-family browser and a running
test instance where the agent may create, edit, and delete data.
REA records the app you provide; you start it and prepare its test
data.

To run the exported environment, install the
[AgentEnv framework](https://github.com/scaleapi/agentenv-framework#install)
and start Docker. REA's compatibility check uses AgentEnv version
`0.9.1334` with Python 3.12.

## Record and build with your agent

Tell your agent which app and operations to cover. For an app with
folders, adapt this example:

Your coding agent

Use REA to build an AgentEnv environment from my test web app
at `http://127.0.0.1:3000`. Cover listing,
creating, renaming, and deleting folders. You may create and
remove test folders; leave existing data unchanged. Sign in as
admin using the password in `APP_PASSWORD`. Use the
browser at `/absolute/path/to/chromium` and export
to `/absolute/path/to/my-env`, which does not exist
yet. Check which operations are supported and report any
gaps.

Example prompt. Replace the URL, operations, account, and paths.
Set `APP_PASSWORD` in the environment that starts REA's
MCP server so the password stays out of the prompt.

Your agent inspects the app, records a session, checks the
supported operations, and exports the environment. After export,
continue with [running the environment](https://rea.tools/guides/rl-environments/#run).

## Or use a saved recording

A HAR file records a browser's network requests and responses.
Export one with API response bodies from the browser's network
panel, then build into a new or empty folder:

Build the environment

```
npx -y rea-agents@latest environment-build ./session.har ./my-env --name my-env --json
```

You can also pass capture Evidence JSON saved by REA's
`capture-browser-scenario` or HAR inspection. Evidence
is the saved recording with its source information. The build
reads local data and writes the folder without contacting the app.
Add `--dry-run` to review the report without writing
files.

The report lists read operations, supported changes, and
operations withheld because the recording did not establish their
effects. Each withheld operation has a reason and a hint about
what to record next.

## Run the environment

Whether your agent exported the folder or you built it from a
recording, run it with AgentEnv:

Run the environment test

```
agent-env run ./my-env
```

Use the path your agent reported if it differs. The first run
builds the container image and needs access to its image and
package sources.

The included smoke task is a basic test. It checks the tool count
and, when a suitable read tool exists, calls it and checks its
response status. Passing it shows the environment starts and
responds in that setup.

## Add tasks and reward checks

Use the environment with your RL framework to define goals, score
results, and run training. Your coding agent can help write the
tasks and checks. Start with one operation the environment
supports:

Your coding agent

Use this exported environment to add an AgentEnv task that
asks an agent to create a folder named Practice. Check the
environment state to confirm the folder exists with that name,
and use that check to score the result. Reset the environment
before each run. Use the AgentEnv task and verifier
formats.

Example prompt for an environment that supports folder creation. A
task that changes data should check the resulting state; a
retrieval task can check an answer against a recorded value.

The export includes the runnable environment and its basic test.
Task-specific grading is added in the evaluation workflow. See the
[AgentEnv documentation](https://www.agentenvframework.com/docs)
for tasks, verifiers, and agent connections.

## Check what works

For changes to be supported, record the results as well as the
action: read an item, change it, then read the item and its list
again before the next change. Repeat with different values. For
deletes, record the missing-item response and the updated list.

A passing startup test, an agent completing a task, and matching
the original app are separate checks. For the last one, use
`environment-compare` with a separate recording of
calls made against the app from the same starting data.

Before sharing the folder, review its retained response bodies and
local paths. Custom headers, query values, and body tokens can
remain. Credential-name warnings identify places to review; they
do not remove those values.

Follow the
[full recording and validation guide](https://github.com/morluto/rea/blob/main/docs/agent-environments-quickstart.md)
for coverage reports and comparison commands. The
[environment reference](https://github.com/morluto/rea/blob/main/docs/agent-environments.md)
covers direct MCP serving, state controls, and limits.