If you asked a lawyer, they would probably say the agreement/exemption should be signed in writing (with more legalese) and not just an email agreement. That said:
There is a lot of subjectivity and interpretation when it comes to law and lawsuits.
If you went to trial, you would be showing the jury your employment agreement saying all IP belongs to the company, right before showing the jury a print out of the email saying the exact opposite. I think most jurors would side with the employee if the employer agreed to exclude IP assignment for a specific side project in writing.
That doesn’t mean your company can’t still sue you (anyone can sue anyone for nearly anything). But doing so costs everyone money (and reputation), so it’s rarely in the best interest of the company to sue employees - especially if you have a paper trail indicating your employer is aware of what you’re doing outside of work and has given you written permission.
All of the above assumes that the IP you’re creating is not something that would be competitive or damaging to your employer’s business. And that you’re acting in good faith and not misleading or somehow screwing over your employer in the process.