Even when I vacationed in Bayern (Bavaria), a lot of people switched to English when they realized that my German wasn't that good. Which was frustrating, because I wanted to practice speaking with native speakers.
Although, there were also people who only spoke German.
If you want to learn you just pretend that you don’t speak English and keep on speaking German. They will switch back fast.