Skip to main content

Backtesting

Description

Steer's Uniswap backtesting tool is a CLI tool designed for simulating wasm apps to be run using Steer's infrastructure. The tool is designed to simulate all transactions that would be executed while running the app on mainnet. Based on a time period set by the user, the tool will fork mainnet at the begining of the period and pull all transactions related to the pool being used. It will then prompt inputs for the test and needed parameters from the wasm bundle. Once everything is set up, the backtesting tool will simulate the behavior of the UniLiquidityManager exactly as it would behave in reality. Liquidity positions will be set and moved based on the results of the app wasm bundle. During the test, a UI in the browser will open, showing the price movement measured in ticks. The positions made and added will be shown here as well. At the end of the backtesting period, the client will generate a series of metrics which show the results of the test.

info

@steerprotocol/steer-toolset is in beta, changes may occur.

Steer Protocol Toolset

How to Install

Run one of the following commands to install @steerprotocol/steer-toolset globally.

npm install -g @steerprotocol/steer-toolset

How to Use

You will need to have an app file built to wasm, see our App Guide for making your own or to download the example.

To initiate a backtest run the following command:

steer-toolset app backtest path_to_bundle/app.wasm
Don't have a strategy?

Try out the backtesting tool with a pre-built strategy app! Download the Steer Protocol Keltner Band app here: Keltner Band app. To understand how this strategy works, read more about keltner channels.

If you have successfully installed the CLI then you should see the following output:

Steer Protocol Toolset

info

Hint: You can use the --help anywhere in the CLI for more available commands and parameters.

Simulation Configuration

The CLI will prompt you for various parameters to setup the backtest. These parameters are required. If you don't know what to put in, don't worry, we'll provide some examples below.

Pool

The first necessary parameter is the pool which you want to test the app against. You have two ways to provide this data. You can either provide the pool address directly via the --pool flag or you can search via the CLI. To do so, provide the name or symbol of a token in the pool you want. Once chosen, a subset will be provided to narrow down to the specific pool you want. Once you've set a pool, the CLI will display a summary of that pool.

info

To find pools with high TVL or volume check on Uniswap's Info Page.

caution

Picking pools with low liquidity or volume can lead to innacurate results due to assumptions of how the introduction of the app would affect overall behavior within the pool. Refer to the Assumptions page for more info.

Timeframe

Next, the timeframe will be requested. The timeframe refers to the dates which the backtest will be run against and is set via a date selector.

All historical data which is consumed by the CLI will be cached locally. This allows for subsequent runs to be faster, as the data doesn't need to be pulled from the network.

tip

Target a timeframe that is at least a month after a pool has been created. This will allow the tool to identify accounts which will be used for the simulation. If you have issues with the tool, try changing the timeframe to a later date or report the issue to the development team.

tip

Having issues pulling data? Try using the --sleep flag!

Candle Size

Once the dataset has been retrieved, the candle size which is required by the app should be provided. This is provided in the following formats: 30m,4h,3d.

App Configuration

All app bundles should expose a config function which provides the necessary parameters for a given app. The CLI will read this configuration from the bundle and request and validate inputs from the user.

Final Simulation Configuration

Once the timeframe, candle size, and app params have been entered the simulator can require its final parameters. These include how often the app should run as well as the initial token balances for the wallet managing the app. The final input required is the price of gas. This parameter is used to help in the calculations of total gas costs over the timeframe the simulator will test.

caution

The amount of tokens can have an impact on behaviour of the pool. Large amounts of liquidity can change the price and impact user decisions. Refer to the Assumptions page.

Running a Test

After running the command your browser should open a tab to http://localhost:8012 (example below). Please wait while the backend warms up, this can take a few minutes. This page will update live!

Steer Protocol Toolset

Additionally you can set a gas price that will apply for the entire period and will affect the calculated ETH used.

info

The backtesting is designed to be flexible if the app fails to execute (a log will tell you when it happens). If multiple runs consecutively fail with no success, then stop the test and debug your app.

Results

After the test is finished running, you will see a series of metrics of how your app performed.

General Run Info

  • Backtest Runtime - how long the test ran for in minutes
  • TX Count - how many transactions the toolkit attempted to simulate
  • Failed TXs - how many transactions failed to be simulated
  • Success Rate - how many historical transactions were successfully simulated (over 95% is best)

Initial Balance Info

  • Initial Native Balance token0 - how much token0 principal was provided
  • Initial Native Balance token1 - how much token1 principal was provided
  • Total principal in Native token1 - full value of the principal measured in terms of token1
  • Initial Value in USD - the value of principal measured in USD. This price is accurate to the day the simulation started.

App Info

  • App Runs - How many times the app executed in the time period
  • Gas Spent - how much gas was used in running the app for this duration
  • Eth spent on gas - how much Eth was used for gas, using the user-provided gas price
  • Gas Spent (USD) - the amount of gas spent in terms of USD with eth prices at the last day of the back-tested period

Final Balance Info

  • Final Native Balance token0 - how much token0 principal was held at the end of test
  • Change in token0 Holdings - the percent change of holdings from the start to end of the test
  • Final Native Balance token1 - how much token1 principal was held at the end of test
  • Change in token1 Holdings - the percent change of holdings from the start to end of the test
  • Final Total in Native token1 - full value of ending holdings measure in terms of token1
  • Final Value in USD - the value of holdings measured in USD, this price is accurate to the day

Results

  • Return from backtest - total increase from initial holdings measured in token1 to final holdings
  • Net USD Adjusted for Gas - not perfectly adjusted due to gas price, but takes the net change of all USD amounts. This value might be high while the return % is low due to increase of the assets compared to USD
  • Compounded APY - compounded for backtesting durations with period return rather than epoch lengths to smooth out the return (Most accurate), does not factor gas expenses
  • Linearized APY - the return for the period continued linearly for a year
  • Hold Change in value - USD value of the principal of the tokens from the begining of the period compared to the end