Is it rude to not answer someone's question when they're intentionally trying to upset you with what they ask?

When you see them approaching, put your hand out and say "STOP! Busy, thanks for understanding" and then go about your business, No one can get to you unless you allow them to. You can also just leave.