Connecting to localhost still involves the network stack and a fair bit of overhead.

SQLite is embedded in your program's address space. You call its functions directly like any other function. Depending on your language, there is probably some FFI overhead but it's a lot less than than an external localhost connection