I have a theory! so take it with a grain of salt. However, i think Gannon is in his late twentys in OoT (in the past) so mid thirties in the dark world (future), and i agree that gannon looks much older in TP, and concidering the way the story works i dont quite understand but im guessing that the twilight realm is the place the saged banned him in....no wait scratch that.
For those of you who are old like me, may remember the fact that when we started ALttP gannon was already sealed away. so in my opinion that is directly after OoT. then in the end of ALttP all we do is defeat the deformed Pig version of gannon that we (and by we i mean us in our late 20's early 30's) have all come to know and HATE! And unlike other games where we see the bad guy either be banished, sealed, or in the case of the non gannon bad guys (besides Demise) out right destroyed. we dont see what happens to gannon at the end of ALttP....enter Twilight Princess!
The sages dont Age because of the.... shall we call it, the void between time! which is where gannon was sealed. as for how he got to the dark world in ALttP i couldnt tell you, like i said im not a story writer or anything. NTM the contradiction that they claim the dark world used to be the Sacred Realm in ALttP. so, i dont know thats just how i see it! Gannon is a clever guy.