Which is closer, the ferry from Jiangmen to Hong Kong or the ferry to Macao?

Our family will be staying for three days in Hong Kong after visiting China.
While we are in Hong Kong visiting sites and relatives for the first time, we'd
also like to visit Macao where my great great grandfather had property at one time.
Is it better to take a boat to Macao after we are already in Hong Kong or is it better
to take the ferry to Macao and go to Hong Kong after Macao?