Cancellation for residency permit

Hi I will be leaving China on the 20th of Nov (2017) and will be leaving early. My employer says they need to cancel my residency permit and take my passport off me to do that. I am nervous to do so given that it took them over a month to give me my permit in the first place. Do I have to hand over my passport in the first place, as I want to avoid this at all costs? And what would happen if I simply left before cancelling my residence permit?