At our gym the youngsters in the rec classes get talent spotted by the competitive coaches or a rec coach might recommend a kid they think looks talented to the competitive coaches for them to have a look at. They will then test them, which usually involves taking the kid aside during their class and testing them for strength, etc. If they pass they will get an invite to either IDP or STEPs. There is also a novice comp class that is like an advanced rec class from where they may get invited. Most STEPs kids start in step 1, but occasionally a kid might start in step 2 or 3, say if they are older and quite talented.
Coming from other gyms, I think they go into the step they were already training. If they are not coping then they could be asked to move down a step, but I don't think that is very common. I don't know if they have to do a formal tryout, I think it is more likely they would just join in a normal training session and be evaluated.
Gymnastics is a small sport here and I don't think we really have queues of people trying to get their kids in. The IDP program is very selective, but if a kid isn't cutting it in IDP they would be allowed to move to STEPs. If a kid wasn't coping with STEPs, they would probably be moved down a step rather than kicked out. I'm talking about the WAG program here, as for the MAG program, I have no idea what they do.