Post New

First of all : These are trying times in markets - and I dont want to add to any confusion. This is purely an analytical discussion which I am hoping will add to your thoughts.
If its confusing - stick to your own or follow Jim's writings. Additionally, the scores of Bearish confirmational signals have been going off - I will provide a separate compendium of those - so this is a risk-filled discussion.

WHAT : Bottom Scoring Ensemble.
SINCE WHEN : Out-of-time ie Post Discovery performance since 2016.

BACKGROUND: Inspiration for this effort comes from Jim. I have followed and read all his posts on the Detector since the throes of the financial crisis. I think it was sometime in '09-'10 when he mentioned he was using a statistical model. Well - that piqued my interest! Of course intent and getting to something useful took years - and hence it was only in 2016 when I rolled this out for myself. While there's always learning on the way - the models themselves are UNTOUCHED *which is very important to avoid look-forward bias.

HOW THEY WORK: Its a suite with 3 layers

(1) The Core models: Theres 3 - (a) Days to Bottom (b) Binary Estimate (c) Ensemble Estimate.
The first one is curious : I never expected it to work, and definitely not in the way it did. ( try not to get embroiled in it. just know its a zoner ie it'll converge near ie just before and typically immediately after a bottom. It rarely get the estimate RIGHT on the bottoming day!)

The other 2 are the 2 main components - alongside there are 2 rulesets - Major Bottom check and Trigger CALL

Trigger call follows the same idea which Jim has mentioned time and again - in a cascading ( and current one being vertical) sliding market - CALL is the first day - at least one of the estimates are outside of the Warning Zone. Major Bottom - a bit misleading ( but I guess I was feeling supremely confident when I coded it!) - is simply how fast ( so 2nd degree) it gets out of the zones - this is basically a very strong reversal requirement ( ie it'll not really happen at the very bottom by design)

IMPORTANT: These models really mark a short to medium term tradeable bottom - so outcome window is not as long as Jim's without the other layers.

FULL TRANSPARENCY: Here's the entire OOT signalling history :
https://www.dropbox.com/scl/fi/3emv29szb0a2qgzq8ju...

And this is the visual of it:
https://www.dropbox.com/scl/fi/x5o9gxnp8sr9469k08n...

One of the KEY learnings for me was : The potential usage of the signals are time variant ( statistically a bit non-stationary) - In a Bullish environment you take the Maximal Warning signal ie Prob % threshold(Look at the Continuous estimate and the Greenish-Yellow bars on the bottom ribbon. The index is also painted) - because you can depend on the "Buy the Dip" horde. In a Bear - you absolutely wait for the turn in the estimates. You will of course take it in the chin on the first round in this method ( ala 3/12/2025!)

Also : CONFIRMATION - Confirmatory signals are separate - and this is critical in a bearish environment - as I had mentioned before the 3/12 was non-confirmational.

(2) Bottom Category Classification:
https://www.dropbox.com/scl/fi/k2cxlgs2bwwp0fj7may...

This is actually based on a familiar concept and would be eerily similar to what platykurtic posted ( that feed is included in this). Ethos for this comes from a seminal post by Zee ( IIRC '-07-'08?). So it looks at various metrics around Stocks Highs & Lows percentiles ( not just the H and L) for various constituents

NET MSG : Category 1 Bottom ie Initial. This is NOT GOOD NEWS - you can see these have a habit of cascading ( of course this happening within a Bull you back up the truck!). I am with Jim here - there will be more ie more pain - and circles on the chart and bottom ribbons will start to fill out completely. Weirdly, divergence works on this - ie the very bottom typically the signal count is lower ( Happened many a times)


(3) Pre-requisites or so called Necessary but NOT Sufficient signals. These are mostly my own. So I will provide a surmise - 50% there! Mostly still non-confirm on capitulation.
Shocking to me really - of course the discourse on Friday got it very close but not signal level.

Here's the snapshot of the entire Bottom Scoring table this time round:https://www.dropbox.com/scl/fi/jerzpbbz2v33eqowfk3...

NET MESSAGE : I am slightly hesitant here -from an extreme O/S standpoint everything is screaming - yet I feel there's a large crowd which simply believes in HODL (?) - ie because they are attuned to the indices simply snapping back - COVID being fresh in their minds I guess.

I am going to look for a fulfillment of all pre-requisites or at least 80% of them.


Hope this helps!

Am I the only one who doesn't understand a word of what you wrote? Sorry, but the jargon is going clearly over my head.

Elan

Elan : As I wrote it ... if its confusing - please ignore. Not the best of ideas at this point in the market to burden your mind with something which creates more doubt.

Again - if you just want to look at a visual - Look at the graph with the signals plotted.... The blue circle is the CALL equivalent and the Green triangle is the "Bottom" signal from the spreadsheets. I also provided a version of the continuous estimate ( Its the bottom panel)

Best

Ok I will write my own conclusions ( I just wanted to provide as much detail as possible for folks to come to their own)

(1) Clearly in Bottoming Zone per the models
(2) Stage 1 Category Bottom per the Categorizations. Stage 1 doesn't mean it will not bottom this time - its just that in Bear there happens to be multiple legs with clusters of these - and you can see the charts become full of signals - both on the indices and the bottom ribbons before a FINAL bottom. We need to wait and see how the Confirmatory signals go.
(3) The ONE creating DOUBT: Pre-requisites to a Bottom. I use these to start the bottom scoring process - I can run this every day if I wanted -but not necessary. So it acts like a pre-filter. One would logically expect at this stage for 100% of the pre-requisites to have been met. Not the case - only 50% there. There's an approximator of capitulation - its close - but signaled yet. Its not a massive requirement the way I defined it - yet not been met.

This makes me hesistant

Hope this makes sense!

Regarding your composite models---

Here's a suggestion of something you might want to try to include. I haven't tried.

My goal has been to identify a bottom by looking for days with statistics that seem consistent with the "sell everything indiscriminately" point, maximum capitulation, not just panic. Often this comes from serious stress in the system, margin calls and the like. If you're down in the dumps you sell what you figure you don't need, but go with a flight to safety. Consumer staples might even rise. When things turn bad, you sell whatever the heck you can.

So, an observation that might be useful: In a normal fear response, people will often head to gold and/or the US dollar and/or US government bonds.
If one sees the big bad extremes in market internals being confirmed by falling gold and/or rising bond yields, especially if the move is material, it seems that people are selling whatever they can. That is often the mark of a market bottom, though it might not tell you whether it's a short or medium or long term one.

I note, without predictive intent, that bond (10y yield +13bp) and gold (-1.85%) prices are both down today, on extremely poor market breadth. The WSJ page is showing 1 new high and 1108 new lows so far today on the NYSE.

Jim

Jim : Excellent suggestions! And I would wholeheartedly agree - especially on the development data set.

I am guessing the exceptions would have been 2021-22 Bear and possibly the Stagflationary period - I think ( guessing here without having the data series in front)
towards the end of it - when Volcker started to raise signficantly ie the 1980 and 82 mini recessions. But IIRC - Gold and Silver ( some massive cornering by some TX folks - IIRC I read from an article) rallied like crazy then.

Both of these are out of my data set - since you have THE model - I am sure you know the explanatory drivers and mine come in 2 windows. All that go back to 90s and EVERYTHING going back to 2003.

I will try and test these and see if they improve IN-SAMPLE and HOLDOUT fits. We can always have 2 versions running.

Always appreciate your thoughts and invaluable insights! You were the inspiration for my efforts

Best
AC

But IIRC - Gold and Silver ( some massive cornering by some TX folks - IIRC I read from an article) rallied like crazy then.

Perhaps, but you might want to check what happened on the precise days of the "get me out of here" bottoms, at least if it's a bear market bottom whose last stages involved forced liquidations all over the place.

In those cases you'd expect almost everything to be selling off. Usually the USD and treasuries rally during panics, but if even those sell of, then something serious is broken.

Jim

Thats an interesting thought - the way I am interpreting it - its like a Severity filter.


I will get together the data and do the analysis and see how the overlay works.

Basically when Prob thresholds exceed >=.8 - what's happening with the "Flight to Safety" metrics

Best

4/7/2025 UPDATE

(a) Models (full pic) : https://www.dropbox.com/scl/fi/gz29jglskj7irasyg50...

4/4/2025 -0.3 79.6% FALSE 87.1%
4/7/2025 -1.5 95.2% FALSE 81.5%

That 95.2% print is one of the highest I have seen ( I am not bringing in GFC 2008 - because that's in-sample)


(b) Categorization: A few more to the ribbon today - this is now comparable to 6/16/2022 and early Mar 2020

So - WHOA! level Oversold really.

(c) Pre-requisites: 82% - ie sorta met. But not all.
AND this one should be obvious -its the Capitulation - I track this pretty minutely - so it was almost there with the Open but of course
one a NET BASIS - today was neutral!


NET MESSAGE : Very likely this is the bottom level on this leg (S&P 4800) ...There's of course No Call Trigger or Bottom reversal call - since the estimates are still squarely in danger zone. In this method ( the 2 ruleset tests) - it'll not nail the bottom as I mentioned before.

Best

4/8/2025:Update

4/7/2025 -1.5 95.2% FALSE 81.5%
4/8/2025 -2.5 20.8% TRUE 9.8% CALL

PIC

https://www.dropbox.com/scl/fi/iddgn2bupm5592iukzx...

Hi Jim ... I tried my hand at it ..... Here's the pic .....with some hand drawn arrows and commentary. Everything else is programmatic.

https://www.dropbox.com/scl/fi/j4r7e1ty1ejh4qrj27d...

Data used :
(a) US 10Yr
(b) Gold contract
( Both 1960s+)
(c) USD contract (1990+)

Legend of signals
(1) Rate related : 2 circles ( Red and Green - bidirectional) and the small White triangle inverted
(2) Gold : Top inverted Yellow triangle
(3) $ Related : Bottom ribbon markers

Inferences:

- Its not common to have everything go berserk. I tried to mark those periods with the hand-drawn arrows
- While it has happened at major lows - but really seems to be event/dislocation driven. ie markets can fall further. or really non-event for equities ( 1993-94 Bond crisis)

Would love to hear your thoughts!

Best
AC

Inferences:
- Its not common to have everything go berserk. I tried to mark those periods with the hand-drawn arrows
- While it has happened at major lows - but really seems to be event/dislocation driven. ie markets can fall further. or really non-event for equities ( 1993-94 Bond crisis)
Would love to hear your thoughts!

The image is a bit hard to parse, but it seems to be not bad at all

My suggestion: use this as an enhancement to a breadth based model.

Start by looking mainly for extremes in bad breadth (advance-decline or newhigh-newlow for NYSE or Nasdaq). Then use these things, only at their great extremes, to hopefully discern between the breadth extremes that are bad and the ones that are really bottoms. The reasoning: all kinds of bad days will have lots of stocks going down, but if the usual havens like bonds and gold are moving the wrong way then it's a REAL capitulation liquidation. That's just a speculation, I have no idea if it will work!

If you don't have a history of breadth data, you can get a full history of breadth stuff from Pinnacle for $39. See B1, B2, B3. https://pinnacledata2.com/idx.html
It's only the update service that costs meaningful money, but you can try to keep it up to date with sites like these
https://www.wsj.com/market-data/stocks
https://www.barchart.com/stocks/highs-lows/summary
I think one of those matches the source that Pinnacle uses (Dow Jones news for most things), but I can't remember which

Jim

It's only the update service that costs meaningful money

If you're going the update route, I use Norgate which I think includes a lot more fields for about the same price ($360/yr).
https://norgatedata.com/data-content-tables.php
I subscribe to US Stocks (gold level).

Includes breadth for about 10 indices? Also get historical price data for individual tickers. I don't use most of what is available. Only issue is it doesn't go back as far as Pinnacle, but I append it to the Pinnacle data I purchased a long time ago. I think there's a more expensive package that goes back further.

Question regarding the Pinnacle updater: when does that update Nasdaq breadth? The WSJ page used to update around 5:30PM ET, now it updates closer to 6:00PM. I think Norgate updates later than that, although the Russell data comes in around 4:55PM.

John

Jim .....Thanks for your thoughts!

I have been a long time Pinnacle subscriber ( thanks to you too from MF posts!) - they are the primary source of this data.

Yeah : I had the same idea - ie to overlay these on the Extreme thresholds of Breadths filter I use. The thing I noticed is that - this sort of a capitulation is not necessary
( 2000-01 Bear , Green circle occurs after the bottom) but Quasi Sufficient - while also happening quite a few times in both '08 (Mid Sep - Leh Bros week, Week of 10th-mid Oct, Last week of Nov etc and then Feb and Mar) and '22 bears. The White triangle is an extreme move in Yields - and that's also not very common.

I have an idea - not sure if it'll work out ie just taking a subset from the model filtered dates and run a ruleset? ( CART/C5.0)

John.... Thanks man! :) I had to recently switch over from Pinnacle to Stockcharts after mappping over from Pinnacle to SC due to the update issues. I have spoken to Ed many a times (bless his heart) - but they have a set process and the updater was konking off more often than not. Other than a couple of sources SC typically updates by 8 pm EST. Most of my base data comes from a data provider similar to Norgate ( I spoke to them about an R integration , they currently have a Python API. They said they will email once they have it - they said they might)



Best
AC

If you're going the update route, I use Norgate which I think includes a lot more fields for about the same price ($360/yr).
https://norgatedata.com/data-content-tables.php
I subscribe to US Stocks (gold level).

Yep, Premium Data is the same company.

Hi Jim : Hope this makes sense!

Here's the first look from a Robust CART run ( My dev data is always thru 2016 - so the 2020 and 2022 results are OOT)

CART Rules : https://www.dropbox.com/scl/fi/v2vydeuiy6xpo7xhwei...

The dataset is filtered using an overarching rule of Ensemble Estimate >=60%. My models are a bit shorter outcome window timeframe so this filters it down to about 250 candidate points - and as you can see only 16% (0.16) of them are Major Bottoms - eg in 2008-09 I only flag 3 days - 3/17 , 11/19-20 ( Bottoms on many major indices - Tech/Intl etc) and 3/6-3/9. Similarly only 3 periods in 2000-01 - 3/21/01 ,right after 9/11 , 10/10/02 and 3/12/03 etc.

So the requirement is very high bar


Greener the cells - more accurate the rule % age wise.

Key inferences
(1) It splits on the 10Yr as primary : And its major buying of the Treasuries - in the charts below its the Red dot circle. Its actually not surprising!
(2) Next levels are my OTHER models - I am pleased it ignored the filtering one subsequently
(3) But on the USD $ : Its requiring a Sell-off condition ( Its the Pink bar in the bottom ribbon of the charts). Since the data is filtered - when you look at all days in entirety - the buy conditions pink actually follows a degree of volatility in the $ itself ( Green/Blue ribbon bars)

So - Net : Extreme Breadth ( I did put the H/Ls as their independent columns - they were not picked - since they are already incorporated into the models) with either Treasuries being bought (best) or a decent $ Sell off ( medium)

Here are 3 period pics:
https://www.dropbox.com/scl/fi/ni4c40kuqygxucfas5c...
https://www.dropbox.com/scl/fi/zhg28bmhkkibmj8iekw...
https://www.dropbox.com/scl/fi/xlxbbyn26fb71hke72a...

Note : Covid and 2021-22 are Out-of-time and OOS for the rules.

Also given that rates are a constricted function - its easy to reach the % age volatility thresholds in lower values like in COVID(ZIRP). So the White triangle is an absolute value Bps move. I think the combination works better. Like you take the Max of the 2

I think Gold volatility is almost a given hence it didnt get picked up. Incorporating these into the models is a fairly heavy lift - because it will involve full re-estimation and validation.

Best
AC

Post New

Mechanical Investing FAQ

---

Contact Shrewd'm
Contact the developer of these message boards.