I am in the situation, and it is annoying having to get all my skills back confidently at the start of every season. I would say if you have the option and you want to do summer gym you should go for it! I would give anything to have that option.
If you don't choose to do that though, just try to condition during summer to stay in shape. It seems like at the start of every season I have to take a step back before i can take a step forward and start progressing again which really sucks. It really just depends on how much you like gym and your dedication. If you think you will enjoy training during the summer and it will be beneficial than i would do it. Good luck with your decision!