> ... imagine my surprise to learn that Systemd was causing my long standing frustration with changing my dns settings. and further surprise to learn that server admins have this same issue and many switch away from using systemd-resolved.
That's introductory course to systemd's shenanigans. People are going to tell you that you're not doing it properly, that there's of course this setting (unless that other setting takes precedence etc.), yada, yada, yada.
If I really have to suffer systemd the first thing I do is manually edit /etc/resolv.conf and then chattr +i /etc/resolv.conf.
And of course remove/purge systemd-resolved.
Not only is it "always the DNS" but then things turn from bad to worse when "it's the DNS, but with systemd".
Removing systemd-resolved is the first step. The second one is moving to an OS or a Linux distro that doesn't have systemd at all.
resolved is fascinating.
Did you know that it will convert the answer to a relative query [0] that has generated an NXDOMAIN into a REFUSED? It doesn't do this for fully-qualified queries, and it doesn't do this for relative queries that return something other than NXDOMAIN.
Why do they do this... even if ALL of the resolvers configured in resolved return NXDOMAIN for the query? «Because we believe that the standards say that resolvers can return REFUSED for any reason at all. This is any reason. Now get lost.». Why don't they do this for fully-qualified queries? «chirping cricket noises»
[0] I think this might also be known as a "zero dots" query. Assume that your DHCP-provided search domain is home.arpa. You can do 'ping pc' and every resolver I remember using will convert that hostname into 'pc.home.arpa' and do a lookup with that name, rather than the one you entered.