Where is the terminal for the ferry from Xiamen to Gulangyu Island?

Will be docking at this city on Diamond Princess on October 20, 2018 (Saturday). The ship will be at the XM port from 10 a.m. to 5 p.m. Do we have enough time to tour the Island and make it back to the ship on our own? Is the ferry terminal within walking distant from the cruise ship terminal? We can purchase tickets there in person after we docked? Thanks for your input in advance.