"SPY "NEEDED" all of it's TECH innovations, social network creations, cloud storage/computing , EV development , ETC ...just to stay up with the stodgy Berkshire!"
...
isn't that a benefit of the index? you get the innovation and success by default.

Sure, but you also get the sundry disadvantages of the index as well.
For example, on any given day you have a disproportionate amount of your money invested in whatever is experiencing a valuation bubble that day. The method of index additions and eliminations ensures a material long term drag. And so on.

It seems that the S&P's construction will give you a drag all the time. The rate of growth of EBIT among non-financial S&P 500 firms lagged US GDP growth by -1.2%/year 1962-1989, and by -0.3%/year 1989-2019. That would suggest to some that it's a worse-than-average set of businesses to invest in, and/or is investing in them in a worse-than-average way.

Much evidence suggests a monkey with a dartboard does better than the S&P 500, though using these specific data points as the evidence you might have to include unlisted firms in the mix, as they are included in GDP figures.

As an aside, there is much to be said for private companies. In a typical year, the typical medium sized business with double digit millions in sales might sell for enterprise value of 5-8 times EBITDA. (I don't like EBITDA as a metric, but that's what is commonly used). In recent years the S&P 500 has traded at around 11-16 ratio of enterprise value to EBITDA. It's not rocket science to suspect that the first choice might be the better one for long run returns, if you can find a way to invest with the level of diversification appropriate to your knowledge of the businesses.

Jim