I would say do every actual sidequest but don’t bother clearing the map of all question marks. Hunting for Witcher school gear is also just mostly cosmetic and optional, but they’re the coolest armors and swords.
Also, if you’re not playing on the lowest difficulty, read the infos in your journal regarding the creatures and prepare accordingly.
Ah, but your entire brand image probably isn’t agressively lying.