A more traditional approach to the calendar might have January 1st on the same day of the week every year, with a variable period of holidays at the end of the year that aren't part of the normal week system.
If we did things that way, the issue of "where should we start week 1?" couldn't even arise.
There are and have been calenders with intercalendar days, how does that make anything easier? Specifically, not every week would be a full week, which in itself is an exception because now a Monday could be followed by a Sunday.
If we did things that way, the issue of "where should we start week 1?" couldn't even arise.