What your saying defies logic, as almost all of the Shekieah have died by the time Oot takes place, you can see threir graves in the kakoriko grave yard.
The Interlopers were the twili's descendents, as it was many hundreds of thousands of years before since it take sa long time to devolpe an evolution so they cant survive out of the twilight realm (unless you're Zant who was powered by ganondorf, or Midna with zelda) of course this would be said that evolution in zelda doesnt take a couple hundred years (like the zora to rito bull.)
If the Twili were banished say 1000 years before the game as you said, and they were part sheikiah, then why would the remaining shekiah serve the royal family? They have no connection to the goddessess in any way, their just royal and Hylans who happen to have blabbermouthed daughters named Zelda who tell random kids they just met about their life stories and send them on quests...
Also, the bit about the Gerudo symbol being on Zants armor isnt much, he could have simply done that as Ganondorf does as well have such a symbol on his cape, and was worhsiping his god by adorning himself with that symbol, nothing out of the ordinary... it's like his crusifix.