Launch TapeYou wrote a bot, maybe with ChatGPT, Claude or Cursor doing most of the typing. You ran it over some history and it made money. That is the most dangerous moment in the whole project.
A first backtest usually tests a bot that could never exist: one that buys at the exact price it saw, pays nothing to trade, and never moves the market. Real bots do none of that. Below are the five fixes we make before we believe any result, our own included.
1. Fill at the next block, not at the signal price
Your bot sees a price, decides to buy, signs a transaction, and that transaction lands in a later block. By then the price has moved. On fresh memecoins it usually moves against you, because other bots saw the same thing you did.
So the honest rule is simple: a signal at time T fills at the end of the first block after T. Never at the price that triggered it. The same goes for sells and stops: a stop that "triggers at -20%" fills wherever the price is one block later.
Takeaway: if your backtest uses the signal's own price as the fill, it is grading you on a trade you could not have made.
# wrong: fill = price_at(signal_time)
# right: fill at the first block after the signal
fill_slot = next(s for s in slots if s.time > signal_time)
fill_price = fill_slot.last_price2. Charge fees on both sides
Every swap pays the pool's fee. On top of that you pay a priority fee to land quickly, and you lose a bit more to slippage. All of that happens on the way in and on the way out.
It sounds small per trade. But a strategy that aims for small, quick wins lives or dies on those few percent, because they come off every single round trip, win or lose.
Takeaway: put a cost on the buy and a cost on the sell, and use real numbers from your own transactions, not a guess.
3. Your size moves the pool
A new coin's pool is shallow. If you buy with a meaningful amount of SOL, you push the price up as you buy, so your average fill is worse than the price you saw. When you sell, you push it down.
Most backtests ignore this and assume you can buy any amount at the quoted price. Use the pool's own maths instead: for a constant-product pool, the tokens you get for x SOL depend on the pool's reserves at that moment.
Takeaway: test at the size you will actually trade. A rule that works at 0.01 SOL can fail at 1 SOL on the same coins.
4. Count the coins that never filled
This one is sneaky. If your entry is "buy 15 seconds after graduation", some coins will have no trades at all after 15 seconds, or the pool will be gone. If your code quietly skips those, fine. But if it skips coins because they went badly (no data after a crash, a missing price after a rug), your results only include the survivors.
The same thing happens when you test on a list of coins you picked later, like "coins that hit the trending page". Those coins already survived something.
Takeaway: decide the list of coins before you look at outcomes, and report how many never reached your entry, as a separate number.
5. Look at the gap, not just the result
Once you have fixed the four things above, run your rules twice: once the naive way (signal price, no fees, no size impact) and once the honest way. The difference between the two runs is the part of your "edge" that was never really there.
Here is what that looked like for us. We took one simple rule set (buy 15 seconds after a pump.fun graduation, sell at 3 minutes or at -20%, 0.1 SOL a trade) and ran it on 67 coins from October 10, 2026, 14:00 to 16:00 UTC. The naive backtest showed +2.3% a trade. The same rules, filled a block late, with fees on both sides and the size moving the pool: -0.9% a trade. Same coins, same rules, opposite answer. The full method is on our Test your bot page.
That is two hours of data, so it does not tell you the rule is bad forever. It tells you that the first number was not a measurement of the rule at all.
Takeaway: if your naive and honest runs are far apart, trust the honest one, and be suspicious of any idea that only works in the naive one.
Why bother
None of this is meant to talk you out of building. Better rules than ours might well exist. The point is to find out cheaply, on data, before the market finds out for you with real SOL.
A good habit: every time your AI writes or changes the backtest, ask it to list exactly how fills, fees, size and missing coins are handled. If it cannot tell you, the backtest is not done yet.
If you want to try this on real graduations, the free one-coin file has every trade of one graduate, a full day ($10) has every trade of every graduate, and Test your bot has a prompt that runs the honest version for you.