Can I buy bullet/high speed train ticket to Hangzhou at Shanghai Longyang Station?

I’m afraid that you cannot buy the train tickets there. You can get one from Shanghai South Railway Station or Hongqiao Railway Station.

Is that mean I need to board the high speed train at South Railway Station? Is it the same with Shanghai Railway Station?

Yes, you can take a train at South Railway Station or Hongqiao Railway Station. South Railway Station is different from the Shanghai Railway Station. South Railway Station is located at no.9001, Humin Road, Xuhui District, while Shanghai Railway Station is located at no.303, Moling Road, Jing’an District.
If you are near Shanghai Railway Station, you can also take a train from there. It’s only three train one day, which depart at 08:05, 09:38 and 17:54.