No. of Recommendations: 14
I compared P123 and GTR1 backtests. In general, there was good agreement, with similar coverage (number of stocks) and similar backtest results. I will add posts to this thread for the following tests:
0. Limitations.
1. Universe Composition.
2. Current Universe Size.
3. Historical Universe Size.
4. Implausible one-day returns (outliers).
5. Replicate indexes: RSP, EWRS, QQQE.
6. Sector, ADR, IPO, Foreign screens.
7. High Net Income Low PE screen.
8. Compare CAGR for several screens.
All results are from 20160104 to 20251128. Only stocks listed on U.S. exchanges were tested.
P123 settings (unless otherwise specified): Price next Close, exclude PIT Method - Prelims, 0% Carry Cost, 0% Slippage, 4 week rebalance.
GTR1 settings (unless otherwise specified): 0 Friction, 19 mkt day hold, rebalance every 19 days.
Some nomenclature:
IoU is the Jaccard Index calculated as Intersection over Union. I use it when comparing 2 lists of tickers. 0% IoU says the 2 lists have no tickers in common. 100% IoU indicates a perfect match.
CorMER is the correlation of monthly excess returns. CorMER above 0.8 indicates good agreement between 2 backtests, and CorMER below 0.6 indicates a significant difference. Monthly excess returns (MER) are calculated from the Portfolio Equity curve, by subtracting the benchmark return from the rolling monthly return. I used yahoo adjusted prices for RSP as the benchmark.
=== link ===
Author: rgearyiii
Subject: Portfolio123 Auditing & Validation
Date: 3/5/2016
"... The first thing I would do is determine exactly what the P123 universe consists of from the non-OTC stock market. ...
count non-OTC stocks over the course of a P123 backtest from universe inception and compare the counts to what the GTR1 backtester predicts ...
Implausible Returns. It appears that a separate backtest of the entire non-OTC universe held in equal weight with a holding period of one market day was also done for me. In this backtest, I noticed two daily returns that were totally implausible: A 1-day forward return of 118.7816% from the close of market day 1/30/2015, and a 1-day forward return of 10.1288% from the close of 10/22/2002. ..."
https://yorickm.com/Message.php?pid=32146036 