For me it's the Fire Temple. I hate Torch Slugs, and I died while fighting Volvagia like 7 times!!!
I agree completely about Bongo Bongo. I've played through the game 20+ times, and his boss room is the only room in the entire game that I'm afraid to enter because I'm not so sure I'll beat him. My odds of survival are a little bit better than a coinflip; any other boss in the game I'd be embarrassed if I get hit once.