Visa on arrival for baby possible?

Felix, you can easily visit both Shenzhen and Zhuhai by obtaining relevant visas i.e. 5 days Shenzhen visa on arrival and 3 days Zhuhai visa on arrival. It is important to emphasize that you cannot travel directly from Shenzhen to Zhuhai and vice versa when in possession of these visas. You have to exit Shenzhen to Hong Kong or Macau and then travel to Zhuhai. The same thing is valid for a travel from Zhuhai to Shenzhen. Also, all of you will have to obtain a visa regardless of age. The approval record for Swedish citizens is excellent.

I would suggest you one far better way to visit Zhuhai, Shenzhen and even the rest of Guangdong province - visa free, based on 144 hour visa free transit. You just have to adapt slightly your travel plan and create the following itinerary:

Hong Kong - Guangdong province - Macau

or

Macau - Guangdong province - Hong Kong

You can exit from Hong Kong toward mainland China by ferry to Shekou port (in Shenzhen) or to Nansha port (south from Guangzhou). From Macau, you can exit to Shekou port.

To exit China, you will have to take a ferry to Hong Kong or Macau or a train to Hong Kong, but your exit region has to be different from the region you have used to enter China. Also, you will have to book your exit ticket in advance, even before you exit Hong Kong or Macau because it will be a proof of your transit intention. This ticket has to be printed and you have to have it at hand, ready to be presented to relevant authorities. In the end, you will have to take exactly that ferry or train you have bought your ticket for.