While parents with kids in the 10 and under age tend to be a little surprised by the hours my DD goes, once those same kids hit about 5th grade, they often discover how many hours kids sports can take. By 4th and 5th grade, children here are often playing multiple sports or are specializing and starting early travel sports. While they may not do as many hours of practice a week, those sports often have much higher commitments in terms of games and tournaments. And, while some sports are seasonal, sports like soccer have multiple seasons and indoor seasons, etc. To the point those kids are often playing year round.
Gym definitely has a lot of hours at a young age, but it isn't the only sport with significant hours for relatively young kids. Our kids do more conditioning than most, but gym isn't alone in youth sports.
If you are getting a lot of looks or comments, just smile and wait a couple of years and you can say I told you so to many of those parents.