This generalization issue in RL in specific was detailed by OpenAI in 2018

https://arxiv.org/pdf/1804.03720