Visa Query (HK - Macau - HK - China)

I am travelling to China in June on a single entry one. My flight arrives in Hong Kong, so I will spend 4 nights there first. I was considering going to Macau for a day/night - then back to Hong Kong, before going to China. I was worried that they might accidently stamp my passport on entry into Macau - and therefore not allow me to enter China. Is this a valid concern - or do i have nothing to worry about?
Thanks :)