I've seen other devs open source their games and simply release it in a non-functional state with statements clarifying what's not included, why, and what it would take to get things up and running. Just about anyone looking to build a modern game from this sort of source release is going to know what they are getting into with or without these assets and may still get value out of the release even if it doesn't provide all the necessary bits to create an executable.
It's possible, but the patches would be tied to a very specific version of the asset. Unity has made this easier recently by making asset store assets part of the package manager, but my stuff predates all of that.