Answered by Carmen from United Kingdom | May. 30, 2013 22:32
I remember there are two direct buses between the two destinations, leaving the grottoes at 10:30 and 15:30. So you can consult whether they are in operation or not.
The second option for you is to take bus no. 81 to Luoyang Railway Station, near which you can take a direct tourism bus to the Temple. The whole journey takes aorund 2.5h.