About $375 billion of Berkshire's assets sit in equities similar to the index. It is hard to see how Berkshire outperforms relative to the S&P given these assets as a portion of the roughly $1 Trillion of Berkshire assets.

Given the sense of scale, it's probably worth considering the equity portfolio as "Apple" and "other".

I think it's perhaps a concern for the Apple position, whose current valuation level might be high enough not to support good stock returns in the next few years. There might be some air under it.
So I agree with you there.

For the rest of it, there isn't that much resemblance to the broad market in terms of stretched valuation levels.
Quick glance, the weighted average earnings yield of the non-Apple equities seems to be somewhere around 8.4%, equivalent to a P/E under 12.
The average S&P 500 firm is around 24% more expensive on that primitive metric.

There are a few T&T positions which seem a bit speculative, but their sizes are generally down in the rounding error.

Jim