Which is a legitimate concern since they are a gaping hole in security and isolation. Visiting website should be treated like phone calls from the bank. If you get called/mailed you don't follow the information there but call back / visit the site yourself e.g. from bookmarks or copy url from pw manager.
I am now wondering if Safari's integration with the system-wide password manager is similar to having a 1Password browser extension installed in a chromium browser
There have been exploits for them in the past, it's a legitimate concern.
Deciding between the two setups is a tradeoff between one security issue and another.