Mommy needs a break

I know this not one of my regular post but i have things on mind that i need to get it out.
I have always been the type of mother that never takes time for my self. I dont ever go out because i end up feeling so guilty for leaving my kids. Most of my life i was raised by my grandma and if i wasn't living with her my mom left me with her every weekend so she could take a break from me. I try hard to take some time for my self but i just can't do it. I am not a big drinker and when people invite me out it's usually to a bar or club and im just not into clubs or bars.
I dont even drink wine at home like other people. I have tried to drink a glass of wine at the end of the night, but even that makes me feel guilty.
Am I the only one out there that feels like i shouldn't be out in bars or clubs because I am a mother? How do I let go and learn to take time for my self? I get tired of people talking stuff about me not being able to go anywhere with out the whole family. Uuuggghh help me ill take any advice.