Should I submit my expired passport with my present new one?

I have a permanent resident visa for Japan. When I renewed my US passport, I was told by the Immigration Office in Japan that I no longer needed a visa or any stamp on my passport and that my residence card was enough. So, my new passport has no visa stamp on it.