Sorry but I have to agree with your brother. Gymnastics is not a sport, but rather it is an athletic competition. The difference is in how a winner is determined. A sport is has a clear way of determining a winner ie (who crosses line first, who jumps farthest, and etc).
Many people like myself and other feel Gymnastics isnt a sport because the scoring is judged. Saying one persons vault is .1 amount better then another is really subjective. Sport is not just about how much athletic ability is required. Bowling is a sport, but no one says it takes any athletism but who wins/loses is clear.
Gymnastics, figure skating, diving are lumped into the same category. I would equate them to something like a Slam Dunk Contest.