The timing and angle of the release of the fly away is the same from a giant as it is in a tap swing. The only difference is that you are moving faster. There are many different things that could be happening with your release.
Holding on too long. This can be caused from not rotating your hands quickly enough in your giants. If you hands are behind your rotation, it may feel like you are letting go at the right time but your hands have gotten to the release point too late.
"pulling in" This happens if you close your shoulders or bend your arms or pull on the bar in any way before releasing. This causes you to fly back toward the bar.
You may want to get more comfortable with casting into a flyaway. Keep casting higher and higher until you are doing it from a handstand. This will help you get used to the flyaway that is moving faster. I try to think of it as TAP, THROW, RELEASE.
i.e. Cast to handstand and hollow down, tap (for just a normal flyaway, the tap isn't that important since you are getting plenty of rotation), Throw the bar behind your head by opening your shoulders and let go of the bar.
Once you are comfortable with the fly away from a handstand, the timing out of the giant isn't much different. Just make sure you are getting your hands on top of the bar when you are in the handstand part of your giants.