I was under the impression 1" per second was slow. Completely agreed on setup, and after working through this I don't know that I'll ever try to go above 1" per sec again.
tl;dr this is all my fault from using an improper bit.
At David's advice I checked the machine over for loose bolts. I found some loose, particularly on the Z anti backlash nut. I had found a loose screw when unpacking the bot, never reported it as I couldn't see an empty hole. There was an empty hole on the backlash nut. This could explain some Z issues I'd had on an earlier project that a straight bit was recommended for.
I also talked with David about the straight bit I had and calculating ship speeds. First we both agreed I should move back to a proper end mill rather than the Freud router bit I'd grabbed at Rockler to get that unknown out of debugging. At the speeds we are running the bot at, and the speeds the router has available you cannot hit recommended chip loads. So the only thing to do is experiment. I'd say the advice on using the chip load calculator in the docs should be removed as such. It would be nice if we could documenting some starting feeds and speeds for the bot since the chip load calculator won't help you.
After applying some locktite and swapping back to the end mill I hit consistent Z issues where the cut would start in the material rather than on top. David said it sounded like missing Z steps and to slow down. No change. In reviewing everything one last time before calling David again I heard the Z motor grind when I parked the Z axis at 1" above the material in jog mode. I'd told it to go higher than it had room for. So, checked the vcarve files from the fence parts I was using and sure enough they had Z home position set at almost an inch above the table. So there was my consistent lost Z steps. It just so happened the previous bit I was using was much shorter than the end mill so this didn't happen.
So, this corrected reran the project and it's fine. I'm running at 1" per second with the router set at about 5. The material was some rather rough exterior grade ply scrap.
Now that bit. I swapped back to it and immediately hit the same issue that started this post. So maybe upping the router RPMs would do it, but it isn't worth it to try. I'll start a new thread on where to order proper end mills from.
Many thanks to David for working through this with me. I'm going back outside to work on some jigs for my project.