Of course, if you decide to account for Windows / macOS conventions, it’ll be a bit trickier, but pulling in a library for that is a bit overkill, yeah.
Adding a library has nothing to do with that. You can implement it yourself in a few lines of code, especially if you only need one path, like configs. You do need to research for caveats (e.g. check $XDG_CONFIG_HOME on Linux and not just put it into ~/.config) but it’s not rocket science.
And if you choose to just trust library authors, you are putting their ideas there instead. There’s that Rust crate that uses XDG on macOS, for example.