Visit Hong Kong island during 4 hour transfer

We are scheduled to arrive at terminal 1 Tuesday at 4:45 pm and depart again later that day at 9 pm also from terminal 1. Our luggage is checked through to our final destination. having never been to HK before and wishing to get a first feel of the place we wondered if it was possible (and advisable) to leave the airport, take the HK Airport Express to its final stop on HK Island, walk around for an hour or so and return to check in and passe security clearance and all with only our carry-on luggage. Or do the arrival and departure procedures usually take many hours?