how to go to victoria peak by public transport?

Well, it's not necessary for you to take a taxi.
Most tourists choose to take cable car to arrive at the the peak.
You can take metro to go to Central Station first. Then walk to the Peak Tram Lower Terminus and take the cable car directly.

Hi, walk to Garden Road Peak Tram Lower Station to take the peak tram to the top. It costs 40HKD for a round trip and 28HKD for a single trip.

By bus, you should walk to Central (Exchange Square) bus stop to take bus no. 15 and get off at Mount Kellett Road, Peak Road. Then walk to the destination. The bus fare is 9.8HKD and it takes 45min.
By taxi, it costs 60HKD or a little more.