As best I know, and can explain, the Master Sword is granted a power that Hylia herself bestowed upon it. Its ability is basically vanquishing a foe swiftly and more effectively than other sword types (like with how it dealt more damage than, say, the antecedent Goddess Sword), thus how it earns its title: the Blade of Evil's Bane. In my knowledge, no other sword has surpassed the MS's potency. It's an icon to Hylia's people, and plausibly runs on their gauges of hope and faith in its wielder, who happens to be Link.
Also, bear in mind that Ganondorf possesses the Triforce of Power, which would explain why it takes longer to bring him down. The rest of Hyrule's antagonists had some artifact or faculty which made them stronger as well. So while the Master Sword is the downfall to all evil, and the most potent blade, it's shortcoming [theoretically] lies in how powerful the opponent is.
Then there's the case of Ghirahim being able to steal it from Link in the Lightning Round... I don't think that counts, because when Link first fought Ghirahim in those boss battle the Master Sword was still just the Goddess Sword. Evil probably didn't have much of a problem holding it while it was in that state.
If I recall correctly, Ghirahim was also able to hold the True Master Sword in the Fire Sanctuary battle. But in my mind, that's justifiable by the fact that he's a purported antithesis to the Master Sword, because his true form is a sword spirit. Ghirahim isn't much different from Fi in that regard, although we never got to see any special fighting abilities from her. So that makes it very hard to tell...yet the conclusion can still be drawn that he's virtually unaffected by the Master Sword since he's the exact opposite. Sure he took damage, but wasn't utterly defeated; only Demise's intervention near the end terminated him for all we know.
