A reproducible evaluation environment for financial AI agents
Run controlled, reproducible experiments on trading agents in a market with realistic execution - not a backtest over historical prices. A deterministic market simulator with order-book execution, macro dynamics, causal ground truth and agent-native interfaces.
pip install tradefloor
cargo add tradefloorimport tradefloor as tf
# you name the companies yourself
u = tf.Universe([
tf.Instrument("AMZN", "technology",
initial_price=397.17, eps=12.55,
book_value_per_share=20.19,
shares_outstanding=5.1e9),
# GOOGL, MSFT, AAPL, NVDA alike
])
# or Universe.random(5, seed=7) to
# generate one: AAA, AAB, AAC...
e = tf.Engine(seed=776644, universe=u)
e.run_days(30)
bars = e.bars(grain="day") # prices
truth = e.truth() # and why
print(u.fingerprint, e.model_fingerprint)
Reproducible
A universe of companies, a macro state for day zero, and a seed. The same three inputs produce the same market on every supported platform, so a run is reproducible by anyone who has them.
Execution-aware
Agents trade through an order book and pay the spread, the queue and their own impact. Run the same seed without the orders and the difference is the cost of trading.
Inspectable
The simulator computed each price, so it can say why. The nine factor contributions sum to the move - an identity, not a fitted explanation. The residual is near 1e-16.
Agent-native
A Python object with an act method, a Gymnasium environment, or a model calling the simulator as a tool over MCP. The harness never asks what is inside.
Git for market experiments
A market here is an explicit configuration and a seed, so an experiment can be recorded, re-run, compared and branched without depending on a market nobody controls. It is the working practice of version control applied to an experiment rather than to code: share the configuration, share the seed, re-run the experiment and get the same market back.
What a RunManifest pins
reproduce() refuses on a mismatch rather than returning a number that looks fine. A checkpoint branches a run at a decision point, so two hypotheses share one past instead of being statistically similar.
Same world. Same conditions. Different agent. Two agents run against the identical market path, so a difference between them is the agent and not a different roll of the market.
Same experiment. Different hypothesis. Change one component, re-run, and attribute the difference to the change. Determinism is what makes the attribution mean anything.
Fork an experiment without changing its past
mark.branch(2) returns two engines identical to that state, in under a millisecond. Checkpoint.resume() is the slower form that replays the order log and outlives the process.This is a fork of an experiment, not of a repository. Both branches carry the same past bit for bit, so they do not start from similar conditions - they start from the same one, and the difference between the futures is attributable to the input you changed rather than to a different roll of the market.
The simulator knows why the price moved
Every price came out of a computation, so the move decomposes into the nine contributions that produced it and the accounting closes on the realised move. A backtest cannot do this: nobody recorded why the market did what it did, so an attribution over historical prices is a model fitted after the fact.
Questions it makes answerable
- Did the agent trade on momentum, the macro state, liquidity, or noise?
- Did its stated reason match the state the market was actually in?
- What happens to the result when one causal channel is removed?
- Did two agents reach the same P&L for different reasons?
What it is not
Ground truth about this market, not about any real one. The decomposition is exact because the simulator did the arithmetic, which is also the reason it says nothing about why a real price moved.
The seed fixes every random number
The same universe, macro state and seed give the same market on Linux, macOS and Windows, digit for digit. Every release does a check of this. It builds five wheel targets, runs one fixed simulation in each, and compares the digests. A disagreement stops the release. Determinism also makes a fork possible: checkpoint at day 100, then branch, and the two branches share one past. The two runs below are the repository's reference runs, and they have a different seed and a different roster.
The nine factors that sum to every move
One row per instrument per tick. The nine contributions sum to the total log move. The residual is near 1e-16. An explanation here is therefore an identity, not a model that fits after the fact.
Three of them are the model's own dynamics, not shocks. The answer can therefore be "nothing happened, the price drifted back toward fair value". That is the correct answer most of the time.
Choose who exists
A universe is the roster and its fundamentals. The generator fills twelve sectors round-robin, so a generated roster is more balanced than any real index. Set one up here and take the code.
import tradefloor as tf
u = tf.Universe.random(40, seed=7)
len(u) # 40
u.fingerprint # sha256, order included
u[0].ticker, u[0].sector
# heaviest sectors: technology 4, financial_services 4
macro = tf.Macro(federal_funds_rate=0.025,
corporate_bond_yield=0.052, vix=16.0)
e = tf.Engine(seed=42, universe=u, macro_state=macro)
e.run_days(252) # one trading year
Reorder the roster and you get a different market from the same seed. Sort your tickers alphabetically upstream and you change the world by accident. Do a check with universe.fingerprint.
Write the strategy down
A StrategySpec is JSON with a sha256, so someone else can check your result with the package and nothing else of yours. Compose one here and take it.
{
"spec_version": 1,
"signal": {
"kind": "momentum",
"lookback_days": 1
},
"portfolio": {
"gross": 1,
"top_k": 5
},
"execution": {
"cadence": "step",
"max_participation": 0.02
},
"seed": null
}
spec = tf.StrategySpec.momentum(
lookback_days=1.0,
top_k=5)
scores = tf.evaluate(
{"mine": spec}, seed=7,
universe=u, days=10)
scores["mine"].return_pct
scores["mine"].impact_bps
scores["mine"].strategy_fingerprint
# sha256 -- cite this
How an order fills
Price-time priority, one book per name, no slippage formula anywhere. Step through the engine's own reference program.
market buy sweeps several levels
Three ways to use it
Three modes of agent interaction: observe the market, trade inside it, or train against it. A model over MCP, a reinforcement-learning policy, or trading logic you wrote.
Agents over MCP
No simulator integration code. Claude or another model calls tradefloor through MCP, observes the market and submits trades as tool calls. Eleven tools, everything composed as data.
pip install "tradefloor[mcp]" claude mcp add tradefloor \ -- tradefloor-mcp # then just ask it a question
Reinforcement learning
Passes gymnasium's env_checker. Actions are target weights in [-1, 1]. Reward is measured after the market moves, so it already carries the cost of the agent's own footprint.
import tradefloor.gym as ptg
env = ptg.TradingEnv(universe=u,
seed=42, days=20)
obs, info = env.reset(seed=42)
obs, r, done, trunc, i = env.step(a)
Your own trade logic
A TWAP, a VWAP, an iceberg, or your own logic. Any object with an act method will do. The observation is narrow on purpose. It exposes only information available to a trader inside the simulated market.
class Mine:
def act(self, obs):
return {"AAA": 0.2}
tf.tca.analyse(Mine(), seed=42,
universe=u, days=5)
Ten agents, one market each
Hand evaluate a dict of ten agents and each one runs in its own copy of an identical world: same seed, same roster, same macro path. Your algorithm, a trained policy, an LLM agent and the five baselines land in one table.
Then rank repeats that across seeds and separates them with a paired sign test. The Oracle reads the true mispricing and says so. Read it as a reference point, not a ceiling: it spends the same gross on a naive rule, and mean-reversion beats it on 5 of the 12 markets.
Ten agents, ten identical markets. Each agent runs in its own copy of the same deterministic market, so nobody eats another agent's depth and the comparison stays clean and reproducible. A single shared market with many agents is the extension described below.
agents = tf.reference_agents(seed=3)
agents["mine"] = Mine()
agents["policy"] = TrainedPolicy(net)
scores = tf.evaluate(
agents, seed=7, universe=u, days=10)
ranking = tf.rank(
lambda: dict(agents), seeds=range(12),
universe=u, days=10, workers=4)
Coming next: competitive multi-agent markets
The next step is to move from evaluating agents independently to evaluating them together. A shared-book arena will let several agents interact in one market, compete for the same liquidity, respond to each other, and be scored under identical conditions - which turns a reproducible environment for single-agent evaluation into somewhere to study how a market of agents behaves. It is not shipped. The primitives are public and composable in the meantime, and this is what building one takes.
What an arena takes
| COMPONENT | PRETIUM GIVES YOU | YOU WRITE |
|---|---|---|
| agent | act(obs). The harness never asks what is inside an agent. | The model call behind it. |
| merge | engine.run_session takes an order_flow dict, so many agents can push orders into the same tick. | Combining every agent's pending flow into one dict per tick. |
| observation | The book, the position and the truth table, every tick. | What each agent sees, and how you serialise it into a prompt a model can answer. |
| model budget | Nothing: this one is entirely yours. | One call per agent per decision. Ten agents over 20 days at three decisions a day is 600 calls. |
| execution | Portfolio turns target weights into orders and reports pending_flow(). One book per name, price-time priority, so the agent that arrives first fills first. | The participation cap and the order sizing. |
| scoreboard | tf.rank compares agents across seeds, not against each other in one book. | Ranking the cohort within the shared book. |
| counterfactual | The same seed with no flow at all. | The comparison that prices the whole cohort's footprint. |
# 1. an agent is a model behind act()
class LLMTrader:
def __init__(self, model, style):
self.model = model
self.style = style # "aggressive"
def act(self, obs):
reply = self.model.ask(
system=self.style,
book=obs.book,
position=obs.position)
return reply["weights"]
agents = {
"claude-momentum": LLMTrader(m1, "momentum"),
"gpt-value": LLMTrader(m2, "value"),
"twap": TWAP(minutes=30),
}
# 2. one engine, one book, all of them
engine = tf.Engine(seed=7, universe=u)
books = {n: tf.Portfolio(cash=1e6) for n in agents}
for day in range(days):
engine.open_market()
for step in range(steps_per_day):
flow = {}
for name, agent in agents.items():
obs = observe(engine, books[name])
orders = tf.baselines.rebalance(
obs, agent.act(obs),
max_participation=0.02)
books[name].execute(orders)
merge(flow, books[name].pending_flow())
engine.run_session(*clock(step), ticks,
order_flow=flow)
engine.close_market()
What the arena would give you
Agents that pay for each other's size. In a shared book agents do not only compete on returns, they compete for liquidity: one agent's sweep consumes the depth another expected to trade against, and every fill is priced by whoever else was in the book that tick. The counterfactual still works - same seed, no flow - so you can price the whole cohort's footprint.
What it would not give you
A market maker that learns the cohort. The maker and the aggregate flow do not adapt, so the only strategic behaviour in the arena is the behaviour you brought. The realism envelope is also silent here: it was measured on one agent, so a cohort result sits outside what has been certified.
How closely it reproduces measured behaviour
Real markets have habits. Prices jump by a typical amount. A calm week often comes after a calm week. Names move together in a panic. This page measures fourteen of those habits, each against the range that real equities hold. For the shipped preset, the thirty-seed median of each of the fourteen falls inside its range at 252 days, on the forty-name sector-balanced roster the certification was measured on and again on a held-out sixty-name universe. A median is not every seed: check the intervals before relying on one. There is no single score, because one number travels further than its caveats.
This is the exact roster the certification was measured on: twelve sectors, filled round-robin, forty names. The thirty-seed median of all fourteen landed in range on it.
Run a crisis through it
A scenario drives the macro path. Without one, the economy finds its own path. Pick a scenario and see what the market did.
import tradefloor as tf
u = tf.Universe.random(40, seed=7)
shock = tf.Scenario.rate_shock(
start=0.045, end=0.016, over=13)
pt.evaluate(agents, seed=7,
universe=u, days=83,
scenario=shock)
# leave corporate_bond_yield free:
# pinning it severs the channel
# the shock is travelling down.
Good results here do not predict real returns
The price process comes from a known model. A strategy that fits the structure of that model looks excellent and teaches you nothing. A strategy that fails here tells you more, because it broke against a live order book under honest impact costs.
One market is one sample
On the twelve-market grid the docs measure, mean-reversion pools at +0.783 capture and momentum at +0.259, so mean-reversion leads by three times. Paired across the same twelve markets it wins 9 to 3, at p = 0.15. One seed picks the pooled leader eight times in twelve, and what a seed says momentum is worth runs from -0.503 to +0.909.
Built for financial AI research
LLM agents
Evaluate agents that reason, call tools, read state and make a sequence of trading decisions. The MCP server exposes the simulator as eleven tools.
Reinforcement learning
A Gymnasium environment with explicit rewards, execution costs and market state, and as many independent episodes as you want.
Quant and execution
Queue position, partial fills, honest impact, and macro scenarios to run a strategy through. Test under controlled dynamics rather than one historical path.
Benchmark builders
A manifest that names the package version, preset, seed, universe, macro, scenario and strategy, and a reproduce() that refuses on mismatch.