It's considered scary and eerie because it's isolated, and the music especially helps in giving the sense that the temple is rather secluded and abandoned, completely forgotten by civilization and now only strange, foreign creatures inhabit the area.