LEARN/RL ENVIRONMENT

RL environment

A Gymnasium environment where the market answers back. Every seed is a fresh independent episode.

from tradefloor.gym import TradingEnv

env = TradingEnv(
    universe=u, seed=42, days=20)

obs, info = env.reset(seed=42)
obs, reward, terminated, truncated, info = \
    env.step(action)

Three things to know

It passes env_checker Gymnasium's own conformance check, so your existing training code works without a wrapper.

Actions are target weights One weight per name, in [-1, 1]. The harness turns them into orders against the book.

Reward includes your footprint It is measured after the market moves, so the cost of your own size is already in it.

Why a tape can not teach size

On historical data

A policy can buy a million shares of a name that trades ten thousand a day. The tape carries on exactly as it did in 2019. What the policy learns about size is then fiction.

Here

The order takes the levels above it and the price moves. Reward is measured after that move, so the cost of the footprint is already in the number the policy optimises.

Actions are target weights in [-1, 1]. A policy does not spend its first million steps to learn that one name trades at 40 and another at 400.

One seed, one episode

Training needs one independent episode per seed. A seed gives you one, and there is no limit on how many you draw. Two things follow.

Variance is measurable

Hold the universe fixed and change the simulation seed. The spread across seeds tells you how much of a score was the market rather than the policy.

A result stays checkable

Each episode is defined by package version, preset, universe fingerprint and seed. Someone else can replay the exact episode your policy trained on.

for seed in range(2048):
    obs, info = env.reset(seed=seed)   # a new market, same rules
    done = False
    while not done:
        action = policy(obs)
        obs, reward, terminated, truncated, info = env.step(action)
        done = terminated or truncated

Read this before you trust a policy

A policy can learn the model instead of the market

The price process comes from a known model. A policy is very good at finding the structure of that model. A high score can mean it found the herding term rather than a real edge. Test the trained policy against the checks on the agents page, and read the envelope before you make a claim about a real market.

Volatility memory does not transfer past a month

The model forgets a volatile period faster than a real market. The log-log slope over lags 1 to 20 reads -0.953 against a real -0.436. A policy that sizes on a one-month volatility estimate is learning a decay shape that real markets do not have.