Backtesting
Description
Steer's Uniswap backtesting tool is a CLI tool designed for simulating wasm apps to be run using Steer's infrastructure. The tool is designed to simulate all transactions that would be executed while running the app on mainnet. Based on a time period set by the user, the tool will fork mainnet at the begining of the period and pull all transactions related to the pool being used. It will then prompt inputs for the test and needed parameters from the wasm bundle. Once everything is set up, the backtesting tool will simulate the behavior of the UniLiquidityManager exactly as it would behave in reality. Liquidity positions will be set and moved based on the results of the app wasm bundle. During the test, a UI in the browser will open, showing the price movement measured in ticks. The positions made and added will be shown here as well. At the end of the backtesting period, the client will generate a series of metrics which show the results of the test.
@steerprotocol/steer-toolset is in beta, changes may occur.
How to Install
Run one of the following commands to install @steerprotocol/steer-toolset globally.
- NPM
- Yarn
npm install -g @steerprotocol/steer-toolset
yarn global add @steerprotocol/steer-toolset
How to Use
You will need to have an app file built to wasm, see our App Guide for making your own or to download the example.
To initiate a backtest run the following command:
steer-toolset app backtest path_to_bundle/app.wasm
Try out the backtesting tool with a pre-built strategy app! Download the Steer Protocol Keltner Band app here: Keltner Band app. To understand how this strategy works, read more about keltner channels.
Upon running the backtest command you should see a new browser window open:
Hint: You can use the --help anywhere in the CLI for more available commands and parameters.
Simulation Configuration
You will see a number of areas requiring your attention, all of these parameters are required to run a backtest. If you don't know what to put in, don't worry, we'll provide some examples below.
Protocol
The first thing to pick is the CLAMM protocol you would like to backtest on. We will have live support for these procotocols if they are listed here generally. Don't see what you are looking for? Reach out and we will see if the protocol is viable.
Chain
After picking a protocol, you will be able to select which chain you would like to run the backtest on. Recognize that pools with the same asset pairs and fee tiers will have varying TVL and volume between chains.
Picking pools with low liquidity or volume can lead to innacurate results due to assumptions of how the introduction of the app would affect overall behavior within the pool. Refer to the Assumptions page for more info.
To find pools with high TVL or volume check on Uniswap's Info Page or the associated protocols info or analytics page.
Dataset
Next you will need a dataset from the pool which you want to test the app against. This dataset consists of all of the swaps, liquidity minting and burning over the duration you pick. Provide the name or symbol of a token in the pool you want. Once chosen, a subset will be provided to narrow down to the specific pool you want. Once you've set a pool, the CLI will display a summary of that pool. And you will be able to pick a timeframe to simulate. You will need to wait for the download to complete before starting the backtest.
Chains and pools with lots of activity will take longer to download. If you are having issues timing out, try testing shorter time periods.
All historical data which is consumed by the CLI will be cached locally. This allows for subsequent runs to be faster, as the data doesn't need to be pulled from the network.
Target a timeframe that is at least a month after a pool has been created. This will allow the tool to identify accounts which will be used for the simulation. If you have issues with the tool, try changing the timeframe to a later date or report the issue to the development team.
Candle Size
Once the dataset has been retrieved, the candle size which is required by the app should be provided. This is provided in the following formats: 30m,4h,3d. Depending on the logic in your strategy, this may have an impact on performance.
Strategy Configuration
The bundle you provide should have a config function which provides the necessary parameters for a given app. The backtesting tool will read this configuration from the bundle and requests and validate inputs from the user.
Simulation Configuration
Once the timeframe, candle size, and app params have been entered the simulator can require its final parameters. These include how often the app should run its logic, the initial token balances for the wallet managing the app, and the gas fee for the duration of the simulation. This parameter is used to help in the calculations of the approximate total gas costs over the timeframe the simulator will test.
The amount of tokens can have an impact on behaviour of the pool. Large amounts of liquidity can change the price and impact user decisions. Refer to the Assumptions page.
Running a Test
After running the command your browser should open a tab to http://localhost:8012 (example below). Please wait while the backend warms up, this can take a few minutes. This page will update live!
Additionally you can set a gas price that will apply for the entire period and will affect the calculated ETH used.
The backtesting is designed to be flexible if the app fails to execute (a log will tell you when it happens). If multiple runs consecutively fail with no success, then stop the test and debug your app.
Results
After the test is finished running, you will see a series of metrics of how your app performed on the command line.
General Run Info
- Backtest Runtime - how long the test ran for in minutes
- TX Count - how many transactions the toolkit attempted to simulate
- Failed TXs - how many transactions failed to be simulated
- Success Rate - how many historical transactions were successfully simulated (over 95% is best)
Initial Balance Info
- Initial Native Balance token0 - how much token0 principal was provided
- Initial Native Balance token1 - how much token1 principal was provided
- Total principal in Native token1 - full value of the principal measured in terms of token1
- Initial Value in USD - the value of principal measured in USD. This price is accurate to the day the simulation started.
App Info
- App Runs - How many times the app executed in the time period
- Gas Spent - how much gas was used in running the app for this duration
- Eth spent on gas - how much Eth was used for gas, using the user-provided gas price
- Gas Spent (USD) - the amount of gas spent in terms of USD with eth prices at the last day of the back-tested period
Final Balance Info
- Final Native Balance token0 - how much token0 principal was held at the end of test
- Change in token0 Holdings - the percent change of holdings from the start to end of the test
- Final Native Balance token1 - how much token1 principal was held at the end of test
- Change in token1 Holdings - the percent change of holdings from the start to end of the test
- Final Total in Native token1 - full value of ending holdings measure in terms of token1
- Final Value in USD - the value of holdings measured in USD, this price is accurate to the day
Results
- Return from backtest - total increase from initial holdings measured in token1 to final holdings
- Net USD Adjusted for Gas - not perfectly adjusted due to gas price, but takes the net change of all USD amounts. This value might be high while the return % is low due to increase of the assets compared to USD
- Compounded APY - compounded for backtesting durations with period return rather than epoch lengths to smooth out the return (Most accurate), does not factor gas expenses
- Linearized APY - the return for the period continued linearly for a year
- Hold Change in value - USD value of the principal of the tokens from the begining of the period compared to the end