Norwegian and/or Chinese passport for our baby born in China

Me (Norwegian) and my wife (Chinese) have two month old born baby here in China. We are going to move to Norway later next year. If we apply for the Norwegian first we can not apply for the Chinese. Opposite we can but then can we keep both even we move to Norway?
Is there any good reason for apply the Chinese passport? Someone with experiance . We plan to travel to China for holiday one time a year.
If we have both passports for our son. Can we only use the Norwegian passport (with Chinese visa) when we travel to China for holiday?