Subject: Re: ML for MI
Hello Boss..... I come from a school of thought - where if you are at Zero - anything positive is an improvement.... even if you are at 45 and failing the course. You have every hope of passing AND with a learner - you might go all the way past 80. Yeah - you wont look so cool as your genius friends who come and seem to conjure up 90+s all the time- but who cares!

QUOTE :
Well I unintentionally stirred up a hornets nest. The two most esteemed old time members of the board mungofitch and zeelotes both seem to dismiss my ML attempt at stock prediction as a complete data mining folly.
: ENDQUOTE

(1) I definitely think you misinterpreted what Jim said. Please re-read. His concern and that's the universal concern with MLs is its ability to remember the whole sequence and just reproduce it forward. In our days in school "Commit to memory , vomit to paper" - till you come up against someone like BKS with questions always from the left most field and over the horizon - ala the MARKET!

NET : Please please ensure you have an OOT ( Out-of-Time) Sample - this is pristine.(at min this should be FY20-23) You will ONLY touch it - after you have your final model. And then perform an OOT validation as a completely separate exercise - if it bombs - well SOL. I have oodles of folders with stuff like that.

(2) Zee : He's from a different school of thought. Man spends hours and hours on developing expert systems with massive amounts of data and fine tuning by hand.
Think of him as your Alpha-GO challenge :)

"The advantages are an automated tool that finds general relationships covering the whole data set at a speed no individual person could hope to match." BINGO!
As per my thought process most modeling involves Search and Optimization - automating it helps you get to a better place - for the larger audience

"The point is I believe an intelligent person with the right tools with a full time multi year effort can always perform better than the ML can. Yuval has been working at P123 alongside the team developing their ML system and he has stated he at least for the present doesn’t see the value in it for him."

True statement - Quote from Deep Learning - Most people cant overcome a computer today at chess. But a human expert with a computer learner is Unbeatable!

The point is - there's unlikely to be a LOT of secret sauce in what you are doing, because I am sure Wall St professionals have possibly already gone thru a few versions of it. But what it hopefully will/should provide the community , just like GTR1 is a consistency of approach and refreshing paradigm on which some of the stuff hereabouts can be measured.