I wouldn't say that Majoras Mask is "By far" the creepiest/darkest Zelda game....
However I think it was mainly the death scenes, to be more specific the Mikau death in Great Bay. There was just something about this scene, ypu would enter the bay which seemed like a nice place to live however you aould be greeted by mysterious music and the area was largely deserted. You would come across Mikau, laying helplessly floating in the water. I guess it was must the position that he was laying in and after returning him to the shore that is where his death happens. Ou also saw the effect that this had on the rest of Zoras as he was a key part of the band and the race, the fact that once you left Termina and you knew that the rest of the Zoras would be mourning his death was quite sad.