Oy. On the other hand, it was probably inevitable that your paths would cross eventually, and a mercy that they hadn't until now.

just a random middle-aged white guy ...

Very telling. He and I are the same age. I don't think of and see myself as middle-aged quite yet (maybe I'm deluding myself, however ... ).

On the other hand, he wasn't aging well at all the last time I saw him, and I think it's been a couple of years since then.

I think your response was appropriate ("do NOT engage ..."), even if he now ends up whinging to somebody that you were mean and rude to him.

Assuming anybody will listen at this point.