It's highly unlikely for The Happy Mask salesman to be a member of the magic tribe mentioned in Twilight Princess who became the Twili. The tribe was shown to have used the fused shadows to attempt to claim the whole triforce for their own. However in OOT and MM the triforce is divided between...
I don't think the Yiga were even around back when the magic tribe were sealed. I think the sealing happened prior to OOT and was during the Hylian civil war in which Links mother became injured and gave her baby to the Deku Tree.
Ganondorf was sent to the twilight realm after the magic tribe...