It's much easier in 2014 than it was in 2004. POSIX has evolved, and POSIX conformance has substantially improved. Most systems are, in practice, nearly 100% conformant to POSIX-2001.
Excepting Windows, I rarely run into difficult portability problems except when I deliberately use non-POSIX functionality or newer POSIX functionality.
I target Linux, OS X, OpenBSD, NetBSD, FreeBSD, Solaris, and AIX. The biggest laggard was OpenBSD, particularly wrt to threading, signal handling, and real-time extensions. But in the past couple of years that's been substantially addressed.
One of my biggest headaches now is OS X. They appear to have stopped trying to track POSIX, so while everybody else is busily implementing POSIX-2004, POSIX-2008, and tentative POSIX features, OS X is nearly at a stand-still. OS X hasn't fixed any significant conformance issues, adopted real-time extensions, nor adopted any POSIX-2008 features for several years, now.
I used to do cross platform across Aix, HP-UX, Solaris, GNU/Linux, FreeBSD and Windows NT/2000 back in the .COM days.