Not single handed, but we won a race in the Solent when the leading boat " lost" the next mark and took half the fleet with it towards the wrong mark. Our Nav man was made of sterner stuff and insisted we did not follow. He was right n' all.Also sometimes you lose sight of the next mark as you may be trimming sails etc. and take your eyes off the mark, which can lose you valuable seconds.
Yep, been there done that when my tiller pilot packed up during a race.If you can't stick your tiller in your bum to steer whilst releasing and pulling in your genoa sheets you've got the wrong boat. Ditto mainsheet.
Winching them in as necessary, one hand in the tiller and one on the sheet winch handle, should also work. You've got self tailers of course?
My efforts so far look more like Norman Wisdom.My boat sounds the same as yours
I race just for fun, an excuse to go out, practice boat handling and learn sail trimming. When things get hectic I tend to back off and let the 'big boys' slug it out.
I don't have a self tacker either, I do get some admiring glances when I compete a tack on my own. From a distance it looks like a well choreographed dance.... apparently.