Most Greek temples faced east, but it's not true that all of them did. A famous example of one that doesn't is the well-preserved temple of Apollo at Bassae in Arcadia, which lies on a north-south axis. At Bassea, it seems to have been built that way for practical reasons (limited space). There is, however, a door facing east. I was at a dig in Asea, Arcadia in 1997, and the temple there did not face east, and there was no obvious reason for this.

>Why did they do this--was it prescribed by ritual custom?

That's the best explanation I guess - a custom, most likely started since it would face the sunrise when the door was in the east. Traditionally, churches are often laid out in the same way, with an apsis where the altar is placed in the east end. It's hard to tell when it started, but I've seen an unpublished thesis on Mycenean sanctuaries, and it seems that the direction is more or less random there. Iron Age sanctuaries are a bit
tricky, since most of the remains are just postholes, so it's not until the Archaich period that we can see a pattern.

> I also heard that there is something odd about the Parthenon in this respect--like the front faces East but you have to approach the west end of the building and walk around the thing before you can get into the bowels of the temple?

The east side of the acropolis of Athens is very steep, so the entrance was built on the west side. Thus, the first thing you see when you enter through the Propylaea is the west side of the Parthenon, and you would then have to go to the east side to find the entrance.