We will be travelling to Jiangmen as family of three kids and my spouse.

You can take direct ferry at China Ferry Terminal. There are two schedules every day: 08:30 and 15:30. The duration is about 150 mins. It costs 240 HKD per adult and 173 HKD per child (1 to 4 years old).

Or you can take direct long-distance bus at Prince Edward Station. They depart at 08:05, 09:05, 10:05, 11:05, 16:05, 18:05, 19:05 and 20:35. The duration is about 3.5 hours. Adult ticket is 240 HKD per person and 100 HKD per child.

Thank you Kevin for your prompt response. If we take direct ferry at China Ferry Terminal, how it will be the baggage procedure? Do I have to collect my baggage before take the ferry or there is a procedure at the airport that transfer my baggage to the ferry?
Is it better to buy ferry tickets in advance?

No, you need to collect the luggage before taking the ferry. As I know, only Skypier provides automatic luggage transfer.
Generally speaking, it's not necessary for you to buy the tickets in advance. Just buy them when you arriving.

From the airport, you can take airport express to get to Kowloon Station. Then take bus no.973 at Western Harbour Crossing Toll Plaza, get off at China HK City, Kowloon Park Drive. Then walk west around 310 yards to find the terminal.