If you're not developing an iOS/macOS app, you can skip Xcode completely and just use the `swift` CLI, which is perfectly cromulent. (It works great on Linux and Windows.)
Even if you're developing for macOS you can skip xcode. I've had a great time developing a menubar app for macOS and not once did I need to open xcode.
I would avoid it for Linux and Windows. Even if they are "technically supported", Apple's focus is clearly macOS and iOS. Being a second- (or even third-) class citizen often introduces lots of issues in practice ("oh, nobody teenaged that functionality on Windows"...)
There'a great indie app called Notepad.exe [1] for developing iOS and macOS apps using macOS. You can also write and test Swift apps for Linux easily [2]. It also supports Python and JavaScript.
If you hate Xcode, this is definitely worth a look.
[1]: https://notepadexe.com
[2]: https://notepadexe.com/news/#notepad-14-linux-support