Recently I heard that many cities/provinces in China will not even send your invite letter to Japan or Korea anymore. One guy was specifically told he MUST receive the invite letter in his home country. Just a few months ago, you could receive your invite letter and process your Z visa in Japan, Korea, and Mongolia. No longer the case, apparently.
And don't even think about Taiwan. As far as the Chinese government is concerned, Taiwan is still 'in country' and you are not out of China.
However, these are the current rules and laws and they are subject to change at any time. Also, every city and province has different visa requirements and processes. Bigger cities (Beijing, Shanghai, Shenzhen, Guangzhou) have much tougher requirements and restrictions. Smaller cities seem to be more lenient.