Why you can’t test intraday strategies with daily data

Let’s use this strategy trading XLP ETF as an example. We want to buy XLP at the open if it falls more than -0.1% and if the previous day it lost more than -0.25%. In appearance we could test this strategy with the following simple python code:

import numpy as np
import pandas as pd
import quandl

xlp_daily=quandl.get("YAHOO/XLP",start_date="2005-1-1",end_date="2013-1-1")

"""
    Yesterday must have been a down day of at least 0.25%.
    If XLP opens down more than 0.1% today, go long and exit on the close.
"""
signal =( ( -1 + xlp_daily.Open / xlp_daily.shift(1).Close < -0.001) &
          ( -1 + xlp_daily.shift(1).Close / xlp_daily.shift(2).Close < -0.0025) )

multiplier=1E5

profits=(signal*multiplier*(xlp_daily.Close-xlp_daily.Open))
returns=(signal*(-1+xlp_daily.Close/xlp_daily.Open))

profits.cumsum().plot()

And plotting the profits curve we get the same result as the linked blog post:

What happens if we code this strategy in Quantopian?

import numpy as np


def initialize(context):
    schedule_function(func=sell_at_close,
                      date_rule=date_rules.every_day(),
                      time_rule=time_rules.market_close(minutes=5),
                      half_days=True)
    schedule_function(record_daily_values,
                      date_rules.every_day(),
                      time_rules.market_open(hours=0,minutes=3))  
    

def handle_data(context, data):
    xlp = sid(19659)
    open_today = data.history(xlp,'open',1,'1d')[-1]
    if (xlp not in context.portfolio.positions) and (not np.isnan(open_today)) and (not get_open_orders(xlp)):
        price = data.current(xlp,'price')
        closes = data.history(xlp,'close',3,'1d')
        pct_yesterday = -1 + closes[-2]/closes[-3]
        pct_today = -1 + open_today/closes[-2]
        if (pct_yesterday <= -0.0025) & (pct_today < -0.001):
            order_target(xlp, 5000/price)
            log.info('buying @%s' % price)
            
def sell_at_close(context, data):
    for stock in context.portfolio.positions:
        order_target_percent(stock, 0.0, style=MarketOrder())
        log.info('selling @%s' % data.current(stock,'price'))
       

def record_daily_values(context,data):
    # The record() function allows you to create a custom time-series plot of any variable in your simulation.
    # In this example we will record and plot the market exposure and leverage of our portfolio over time. 
    record(leverage = context.account.leverage)
    record(market_exposure=context.account.net_leverage)

First we notice that even trading a small position of 5000$, the platform warns us that lots of sell orders near the close (5 min before) cannot be filled:

2005-02-18 15:34 handle_data:32 INFO buying @23.42
2005-02-18 21:55 sell_at_close:37 INFO selling @23.47
2005-02-18 22:00 WARN Your order for -213 shares of XLP failed to fill by the end of day and was canceled.
2005-02-22 21:55 sell_at_close:37 INFO selling @23.1
2005-03-10 15:31 handle_data:32 INFO buying @23.33
2005-03-10 21:55 sell_at_close:37 INFO selling @23.4
2005-03-10 22:00 WARN Your order for -214 shares of XLP has been partially filled. 145 shares were successfully sold. 69 shares were not filled by the end of day and were canceled.
2005-03-11 21:55 sell_at_close:37 INFO selling @23.26
2005-03-16 15:33 handle_data:32 INFO buying @23.04
2005-03-16 21:55 sell_at_close:37 INFO selling @23.09
2005-03-16 22:00 WARN Your order for -217 shares of XLP has been partially filled. 75 shares were successfully sold. 142 shares were not filled by the end of day and were canceled.

If we look at the cumulative returns, they look much worse than our simple backtest:

Why such different results?

  • Bad data: Yahoo Open prices have lots of errors
  • Low liquidity: the simple backtest assumes you can trade at open and close prices. Most of the time this is not the case, even with small positions.

 

  • Seine Yumnam

    do you have any example of event-based backtesting or like a sample on how to do event based backtesting?