Spinoffs sound interesting enough when done well. I do not think a racing game would really be much of a spinoff of the main story as it would just be a themed game with familiar characters. Normally for games like these they tend to forgo any real story or just keep it very simple. Something along the lines of a greater evil has picked up these characters from across time and space, etc. Link's crossbow training mostly ignored story for gameplay, I do not see why it could not be done again with a racing game.
A riding game would work well enough over anything motorized, many characters have appeared riding a different animal with varying strengths and weaknesses. I can see Link on Epona being the average speed, control, and durability. Then Maybe Midna on Wolf Link with less speed but much better control, King Bulblin on a bulbo with the least control but could ram straight through all obstacles. Ganondorf on his Gerudo Stallion with greater control and less speed. Tingle could hang from a balloon and have the lowest speed but ignore all obstacles. Zelda or Impa on the white horse could have the greatest speed but least power. There could be Koume and Kotake on a flying broom, Phantom Ganon, or Anjean. Other characters could be added as something to be unlocked like Malon and Ingo. Even Moosh, Dimitri and, Ricky could be unlocked.
Each character could have some abilities of their own that would be helpful as well, like Zelda firing light arrows, Link and Ganondorf have weapons to stun or attack other riders, Koume and Kotake have fire and ice magic to burn down obstacles or create ice patches in the pathway. Tingle can drop bombs etc.