No, its not hard for me to have privacy. What your problem sounds like is a matter of personal space, and perhaps not having enough is causing your privacy to be violated.
Unless I'm misunderstanding what you're saying (or are trying to say), you're not talking about people stalking you or being nosy, right? Because if thats the case, then it may or may not be true if its only a feeling you have. If you FEEL that people are watching you, but you have no way of knowing or do not know if thats the case, then its possible you suffer from some sort of paranoia.
That is to say, if you feel this way all the time, it could be a sign of paranoia or even schizophrenia.
I do feel like I'm being watched sometimes, but thats because I know I am being watched and I don't call attention to it or let them know I know they're watching.