The endianness of file formats and handwriting is irrelevant when it comes to deciding whether your code should support running on big-endian CPUs.
The only question that matters: Do your customers / users want to run it on big-endian hardware? And for 99% of programmers, the answer is no, because their customers have never knowingly been in the same room as a big-endian CPU.
Saying that (hand)writing is irrelevant is a bit of a strawman implying I said writing hexadecimal numbers big-endian on paper matters for coding.
The second sentence, weather your customers know if they have been in the same room with a big-endian system (CPU alone doesn't matter) is irrelevant when the point is to write correct code. Many of then aren't interested in this or other details and that is ok as they are not responsible for the implementation.
Changing the endianness either direction did have show bugs to me several times, that could be fixed, and it was worth it for that alone.
The only question that matters: Do your customers / users want to run it on big-endian hardware? And for 99% of programmers, the answer is no, because their customers have never knowingly been in the same room as a big-endian CPU.