I am currently spending 5 weeks in Xi'an and have found it remarkably easy to get around without knowing Chinese (I too detest guides and tour packages). Buying tickets for attractions has required no speaking at all on my part, and most tourist attractions have signs written in English. However, I would advise buying a map in Chinese and labeling the sights you are trying to see, as well as where you are staying. That way you can simply show it to a taxi driver.