I agree 100% with you that those that say they hate cats simply haven't had the joy of having one in their lives.
My dad wasn't a cat fan at all. I never had a pet growing up, but when I got married my husband already had a pet cat - so I was super excited but my dad wasn't so much. It's funny because as adamant as he was that he hated cats, Casey has really grown on him. I wouldn't go as far as to say he LOVES him (and actually, my dad gets pretty bad allergies around him!) but he says "he's pretty great for a cat" which is as close to him admitting he loves him as I could expect, hah!
I think cats are wonderful affectionate creatures and I do honestly believe they do their best to try and win over the people that say they dislike cats!