Lantau Island and Hong Kong Island in a day?

We are staying in Kowloon for a 36 hour layover. Is it possible to go to Ngong Ping 360, Big Buddha, Po Lin Monastery on Lantau Island then over to HK Island to do the Peak Tram to see the Victoria Peak.
We are not particularly interested in a tour just doing this ourselves so if you have recommendations for transportation also to get from place to place.
Thank you.