WAG Inconsistent judging scores

Flicfliclay

Active Member
Proud Parent
Aug 12, 2016
948
47
Country
USA
Good day everyone! So, most recently I have noticed a big difference in scores for level 10 meets. We will see a girl get say an 8.8 on vault at one meet and at another meet doing the exact same vault get a 9.4? Sorry, I have been in the gym scene since my daughter was a level 2 and that is just insane and not right. How does this happen? I mean i can see a 10th or so, heck even .2, but .6 for the exact vault?
 

Carabistouille

Coach
Coach
Gymnast
Judge
Jan 22, 2019
111
Country
France
What they teach us in judging course (not in the US) is that what matters is to remain consistent during the competition so that you get the order correct. If you start tough, end tough ; if you start nice, end nice.
Remaining consistent between competitions does not matter as much.

From what you said, it could also be that the gymnast's vault is borderline downgradable (for instance a yurchenko layout that is a bit pikey) ? So that it wasn't downgraded at one competition and it was at the next, hence the big difference ?
 

Flicfliclay

Active Member
Proud Parent
Aug 12, 2016
948
47
Country
USA
What they teach us in judging course (not in the US) is that what matters is to remain consistent during the competition so that you get the order correct. If you start tough, end tough ; if you start nice, end nice.
Remaining consistent between competitions does not matter as much.

From what you said, it could also be that the gymnast's vault is borderline downgradable (for instance a yurchenko layout that is a bit pikey) ? So that it wasn't downgraded at one competition and it was at the next, hence the big difference ?
I do understand that and agree, but i have seen the vaults in the case and they were identical. There were also lots of 9.9's given out in this particular session and that alone is not normal. Just curious as to what others have seen.
 

GymDadWA

Member
Proud Parent
Dec 30, 2017
252
42
Country
USA
Subjective judging has always been tough to account for in any sport. You hope for consistency and unbiased results but at the end of the day the judges are people and prone to see things a certain way.

As recently as 2019 they were testing AI judges at the highest level but no idea of what came of it or if it will ever be used at the lower levels.

 
  • Like
Reactions: springsforlegs

Flicfliclay

Active Member
Proud Parent
Aug 12, 2016
948
47
Country
USA
Subjective judging has always been tough to account for in any sport. You hope for consistency and unbiased results but at the end of the day the judges are people and prone to see things a certain way.

As recently as 2019 they were testing AI judges at the highest level but no idea of what came of it or if it will ever be used at the lower levels.

Oh yeah i remember seeing this. Thought it was fascinating. Would be interesting to see that along with a real person judge..
 

chalkbowl

Active Member
Judge
Fan
Nov 16, 2006
617
Country
USA
The judges job is to rank the athletes, not to get the same exact score for the same exact vault every time. A good vault is going to place in the top, regardless of if it's 8.8 or 9.5 at any given competition. There are too many "up to" scores for scoring to be exactly the same when judged by a human. The level of competition also makes a difference....
 

Flicfliclay

Active Member
Proud Parent
Aug 12, 2016
948
47
Country
USA
The judges job is to rank the athletes, not to get the same exact score for the same exact vault every time. A good vault is going to place in the top, regardless of if it's 8.8 or 9.5 at any given competition. There are too many "up to" scores for scoring to be exactly the same when judged by a human. The level of competition also makes a difference....
Sorry, but i think that is crazy.. There is no reason for a kid to get an 8.8 at one meet and a 9.4 at another with no changes in the vault or whatever event.
 

Flicfliclay

Active Member
Proud Parent
Aug 12, 2016
948
47
Country
USA
I mean...it's just gymnastics. As long as the ranking comes out right it's not really a big deal.
Yes, it is just gymnastics. It still makes no sense to me. Like I said it’s one thing being a few tenths, but we are talking over half a point. Either way it is done and will continue to happen apparently.
 

chalkbowl

Active Member
Judge
Fan
Nov 16, 2006
617
Country
USA
Sorry, but i think that is crazy.. There is no reason for a kid to get an 8.8 at one meet and a 9.4 at another with no changes in the vault or whatever event.


There are FOURTY "up to" deductions for optional level vaults alone. A vault takes about 3 seconds to complete. There is no way that two judges are going to be within .2 at multiple meets within a season.
 

amanda.christine

Coach
Coach
Former Gymnast
Gymnast
Judge
Fan
Oct 14, 2020
23
Alameda, CA
Country
USA
Yes, it is just gymnastics. It still makes no sense to me. Like I said it’s one thing being a few tenths, but we are talking over half a point. Either way it is done and will continue to happen apparently.
Honestly, until/unless they allow judges to look at and re-watch a video recording of the routine, the inconsistencies will continue. Some things to consider:

-Judging takes a lot of practice, can be expensive to get into, and doesn't pay well.
-As mentioned above, the "up to" deductions can make a huge difference in the scores because it's subjective and some people will just always be harsher than others. People will up-to a different amount depending on who taught them and their own personal preference. As long as they're consistently applying this with every gymnast then it doesn't really matter.
-Judges are taught when, in doubt, to give the benefit of the doubt to the gymnast and not take a deduction. This means that at a competition with less experienced judges, you may have less deductions being taken simply because they aren't able to judge as quickly/efficiently as someone who has a lot of experience. There are other reasons why some judges may catch more deductions than other judges, this is just one possible example.

This isn't a judging problem, it's just how the system is set up. The only way to change that is to change the system with more rigid explanations of deductions and being able to re-watch routines. Or, I guess someday AI judges but we're definitely not going to see that implemented any time soon.

If the JO scoring is bothering you that much I would just say, take some time to mentally prepare yourself for NCAA scoring because there is well known bias in scoring there and it's not going away any time soon.

Were the competitions where the same vault got scored significantly different, in the same city/state? I'm from New Mexico and we would always get scored higher at home than when we went to big gymnastics states like California and Texas. Like, a LOT different. The standard of gymnastics and number of competitors is just WAY higher in those states so they have to judge more harshly to really distinguish the order.
 
  • Like
Reactions: chalkbowl

beachgirl

Active Member
Proud Parent
Judge
Nov 7, 2013
1,286
55
Country
USA
It's just math. This is just hypothetical, but if there are 5 up to .3 deductions and 1 judge takes .2 on each and 1 judge takes .1 on each, there is already a .5 difference. Judges scores have to be with a certain range. The range is 10-9.5=.2, 9.475- 9= .5, 8.975-8=1.0, etc. For these deductions the scores would be 9.5 and 9.0. The allowable range for these scores is .5, so the scores fall within range. The next meet you might get 2 judges that both only take .1 for each of the deductions, so each judge would give the score 9.5. The gymnast would receive the same placement at the same meet regardless of the score, but you can't compare the placement and a lower score at a different meet. As a parent though, I totally understand the frustration in the wide range of scores.
 

Flicfliclay

Active Member
Proud Parent
Aug 12, 2016
948
47
Country
USA
Honestly, until/unless they allow judges to look at and re-watch a video recording of the routine, the inconsistencies will continue. Some things to consider:

-Judging takes a lot of practice, can be expensive to get into, and doesn't pay well.
-As mentioned above, the "up to" deductions can make a huge difference in the scores because it's subjective and some people will just always be harsher than others. People will up-to a different amount depending on who taught them and their own personal preference. As long as they're consistently applying this with every gymnast then it doesn't really matter.
-Judges are taught when, in doubt, to give the benefit of the doubt to the gymnast and not take a deduction. This means that at a competition with less experienced judges, you may have less deductions being taken simply because they aren't able to judge as quickly/efficiently as someone who has a lot of experience. There are other reasons why some judges may catch more deductions than other judges, this is just one possible example.

This isn't a judging problem, it's just how the system is set up. The only way to change that is to change the system with more rigid explanations of deductions and being able to re-watch routines. Or, I guess someday AI judges but we're definitely not going to see that implemented any time soon.

If the JO scoring is bothering you that much I would just say, take some time to mentally prepare yourself for NCAA scoring because there is well known bias in scoring there and it's not going away any time soon.

Were the competitions where the same vault got scored significantly different, in the same city/state? I'm from New Mexico and we would always get scored higher at home than when we went to big gymnastics states like California and Texas. Like, a LOT different. The standard of gymnastics and number of competitors is just WAY higher in those states so they have to judge more harshly to really distinguish the order.I
It doesn't bother me THAT much, just was curious on the views! I appreciate our reply and it did give me more perspective!
 
  • Like
Reactions: amanda.christine

Flicfliclay

Active Member
Proud Parent
Aug 12, 2016
948
47
Country
USA
It's just math. This is just hypothetical, but if there are 5 up to .3 deductions and 1 judge takes .2 on each and 1 judge takes .1 on each, there is already a .5 difference. Judges scores have to be with a certain range. The range is 10-9.5=.2, 9.475- 9= .5, 8.975-8=1.0, etc. For these deductions the scores would be 9.5 and 9.0. The allowable range for these scores is .5, so the scores fall within range. The next meet you might get 2 judges that both only take .1 for each of the deductions, so each judge would give the score 9.5. The gymnast would receive the same placement at the same meet regardless of the score, but you can't compare the placement and a lower score at a different meet. As a parent though, I totally understand the frustration in the wide range of scores.
yeah, it is frustrating and i have seen it happen for years... as my daughter is a level 10.. it just doesn't get any easier to see! LOL
 

Pineapple_Lump

Coach
Coach
Proud Relative
Judge
Jan 31, 2008
1,126
I mean...it's just gymnastics. As long as the ranking comes out right it's not really a big deal.
This, it shouldn't happen but if the best vaulter in the competition is one of the first few vaults the judge sees then chances are everyone else's scores will be down. Especially if the judge was maybe a little harsh on that vault in order to leave room to rank potentially better vaults higher. Equally if a group of average/poor vaults are in the first rotation - Good vaults that follow are likely to benefit from higher scores. If the next Maroney/Biles is in that session how do you ensure they are ranked appropriately?

Our competitions are formatted that warm up is rotational and the judges (higher levels) are supposed to watch. So they generally have an idea of what the standard is for a session. This can also be a dis-service to the kid who does an amazing warm up and only an excellent vault in competition because she ecentially raised the bar of what the judges expect with one brilliant warm up.
 
  • Like
Reactions: Puff

Flicfliclay

Active Member
Proud Parent
Aug 12, 2016
948
47
Country
USA
I know some kids are consistent vaulters where every vault looks the same, but there’s also kids like mine...who make each vault look...unique. She’s scored between an 8.9 and a 9.6 this season and every number inbetween ‍♀️ You’d think it would look the same each time but...no
ha ha! That is the truth!