Am I seriously the only one who things Igos du Ikana is the hardest thing ever?
Alright, granted, this was back when I was age twelve or younger and sucked at Zelda, but I don't remember having any particular trouble on most of the other bosses in the series. Igos du Ikana from Majora's Mask was the absolute bane of my existence. I hated that jerk. He has a move which is impossible to avoid unless you're rolling at full speed as a Goron, which never occurred to me while I still owned the game. No boss in an action game should have a move that's impossible to avoid! Then you have to jump through a bunch of hoops to even have the chance of hurting him, and the split-second requirements to do this were even more aggravating. Maybe I'll do better when I buy Majora's Mask again, but at the time, he was the most annoying thing ever.
I've never played The Adventure of Link, so I can't comment on its bosses, but I hear they're excruciatingly difficult.
The only other Zelda bosses I've really had trouble with are Bongo Bongo from Ocarina of Time (somehow I never thought to use the Lens of Truth, more proof that I sucked back in the day), the final boss of Skyward Sword, and Scaldera on Hero Mode also from Skyward Sword. Most of these bosses were made hard simply because I didn't know exactly what I was doing, not always because they were inherently difficult.