can i mail back my passport to malaysia local travel agency so they can reapply in malaysia while i still in china?

I don't think this is a good idea. You should keep your passport with you. Sometimes, you may need your passport. For example, what are you going to provide if the police asks you to show your passport? It's too risky to send your passport back to Malaysia. What if it is lost? By the way, I heard that the applicants should stay in the country when he/she applies.

You may visit the local entry and exit management department and see if you can extend your current one. You should go there at least 7 days in davance before it expires.