Most of the time gymnastics is good for the body, not damaging. It develops core strength, good posture, good balance, good flexibility. Training in gymnastics from a young age, strengthens the muscles to sit and stand correctly and this causes lifelong physical benefits, and means they will be much less likely to be prone to the back pains adults suffer, which often stem for a lifetime of poor posture.
Gymnastics is great for pre school aged children, recent research has found that one of the single most important things children need in early childhood, in order to develop excellent reading skills when they are older is physical activity and the types of movements done in gymnastics in particular, strengthen the neural pathways.
The hanging and swinging also develop grip strength, which make it much easier for them to learn to write, type, ties shoes, play a musical instrument etc, when they start school.
Even if they go on to do other sports when they are older, doing gymnastics in the pre school years, really sets them up for a positive future.
Gymnastics is actually one of the few sports that is really safe for young children to do. many sports have repetitive movements, that use the same side of the body, or a particular muscle set more than others. This can pull the growth out of alignment. Gymnastics uses all different body parts in different ways, which fosters healthy physical growth.
When you hear about gymnastics causing damage to the body, they are generally referring to high level competitive gymnasts who are training many, many hours a week. Kids at age 10 training 30 hours a week to becomes elites etc.