I intend to travel to Taiwan in the month of June, is it advisable? I have an elderly in pushchair.

To be honest, June will be the typhoon season in this area. Thus there will be many heavy rainy days, which may influence the sightseeing a lot. So it's suggested to change the travel plan.
Generally speaking, October to May next year is regarded as a good period. You can go there during these months. It's not necessary for you to worry about the transportation. Almost every railway station have speacial facilities and passageways.
Have a good journey!