Have you ever been in a situation where you think people are laughing at you or in on some joke?

You answered your own question--walk away. People who do that are immature, annoying and have nothing better do to, and it makes them feel important. So do what you said, walk away...and feel free to say "grow up" as you walk past them.