I think it depends on the gym teacher. I haven't gotten my kids out of gym, but it doesn't thrill me that most of the classes involve some variation of whipping balls at each other.
Mine is just in 3rd grade, and they do a lot of cooperative play games, not a lot of whipping balls- their gym teacher is really good though. He’s also very supportive of her as a gymnast- when my daughter bruised the bottom of her foot he let her sit out of class for several days no questions asked, no note from me- she just said she had a meet coming and he said okay.
I don’t know how much has changed since I was in school (my 3rd grader is my oldest) but after elementary when I was in school you did not have to take gym ever again. Which is fine with me. But I do think if done right it’s a good kind of interaction for elementary kids.