Any ideas (from anyone) on how to optimize for index correlation?

Oddly enough, using a bit of leverage might help.
If you are writing $100k face value of puts against $100k of cash, you can only do so many.
If you are writing (say) $160k face value of puts against $100k of cash, you can do 60% more positions.

In my view, the main thing about diversification is not to look mainly at historical price correlation.
Rather, think about firms that have different kinds of moving parts.
Interest rate sensitive or not?
A lot of cash or a lot of debt?
Very US focus, focus elsewhere, or very international mix?
Tech fashionability or fuddy duddy?
Big China exposure?
High P/E or low P/E?
Very recession dependent/cyclical?
Dividend payer or not?
Luxury goods or discount goods?
End consumer or business to business?

The reason is that when something bad happens that is NOT transient (the thing you're worried about), it tends to hit a group of firms that are similar in some way.

Of course, the simplest starting point is just to make sure you have at least one position in every sector.

There is no rule that says you have to enter your long positions and your hedges at precisely the same moment.
You're not an investment banker with some box-checking risk committee staring at you.
For example, imagine a given 3 week period with one very high index/happy day that you buy your insurance puts, and one crashy terrible day that you write your single-name puts.
This can make a huge difference to the annual return.

The main thing is that it's a fair bit of work. You want to watch the portfolio.
You'll often find that a position is doing well after a little while: you might make 75% of the maximum possible profit in only 30-50% of the elapsed time.
It's easy to calculate your maximum annualized remaining rate of return on capital at risk.
If it's below a certain number, close the position and replace it with something else. (either immediately, or on the next panicky day).
I used 8%/year rate for a while, though you can pick any number you like.

After a few years I stopped doing the mechanical insurance puts, and it was mainly a long-only portfolio.
I spent more and more time picking the firms I wanted to write repeated puts against, so I had more and more comfort being long my picks.
I did do other hedging, but on a very different cycle: entered only when the market timing entrails seemed to say it was a good time to put them on.
I made very steady returns for a while, but if I look at my returns on the hedging only, I lost a fortune or two.
First rule of hedging: NEVER look at the return of just one side! Always look at the totality.
Because one side is always a huge loser!

Jim