This is a maturity issue, not a gender one, IMO. I remember guys having crushes on me in high school and college; their friends disapproved of me because I was a big girl so they never pursued me in public. Never took the time to get to know me personally, and judged me strictly on their perception of my looks. I thought that was the most immature thing even back then and it still is now.
Unfortunately, that is exactly what people do; they judge and expect you to fall in line with their marching orders. Some sort of societal pecking order that dictates who is/isn't acceptable-who's cool and who isn't. It takes a person who's grounded, mature, and confident to march to their own tune. I never let people make friends for me; if you didn't like them that was your personal problem. I raised my children that way and I haven't changed either.
When I was dating my husband, my friends and family thought I was crazy; he was white and signifcantly older than me (16 yrs my senior). They didn't see who I saw; a man who loved me and became a father to a child that wasn't his own. They just thought he wasn't right for me because of the exterior-they didn't know him the way I did. When we split up, I got the "I told you so's" but they still couldn't tell me why things didn't work out--they just "knew" it wouldn't. That's a whole lot of horse crap to me; us not working out had nothing to do with his age or his race. It had everything to do with why 50% of marriages fail--that's all. Besides, I got a wonderful daughter out of the deal and my son still calls him "dad"-in that respect, I have no regrets.
I hope this answers your question.