The temperature would be mostly around 15-21 celsius, you can wear T-shirts and jacket, that will be enough. I suggest you keep noticing the weather forecast of Taiwan days before you come, since the weather would change and you'll get more specific information from the weather forecast. Welcome to Taiwan!

Thanks to Mr. Henry's reply~By the way, may i know is there any ground tour there which can bring us around Taiwan as i am going freelance without joining any travel agency from Malaysia~

Sure there are ground tour agencies, also you can go and tour on yourself, I recommend the "backpackers" website to you, if is very useful for backpackers, just google it.