Never mind, I did find the square gem and you do need it to get access to the next room. I just forgot. :P
And there's only two swords. Oshus' sword and the Phantom Sword. I do wish the Phantom sword had a black hilt instead of a blue one. It would've looked cooler and it wouldn't have...