I'd very much recommend doing the side quests, as that fills out the entire story of the game. Majora's Mask is basically 50% story and 50% sidequests and they are some of the best sidequests in the series.
It is doable without a walkthrough, but it might take you some time to figure it all out...