Is it rude to not answer someone's question when they're intentionally trying to upset you with what they ask?

Sounds like the person asking the question is seeking to deliberately be rude and that you are attempting to be polite by ignoring them rather than sinking to their level.