I am traveling from US to Tokyo. I have a layover from Shanghai to Tokyo.

Yes, you can visit the city. Upon arrival, you just need to show your passport and onward air ticket when passing through the immigration. And you need to tell the officer you wanna leave the airport, so the officer wil stamp a temporary stay permit onto your passport.