Cleaning up a site can be a chore, but it needs to be done for proper SEO. Trusting WordPress to do that automatically is a mistake. Believing that a Plugin like Yoast can correct for basic errors is also a mistake. Bottom line is to get it right at the base level so SEO stuff takes care of itself.
- Image Name, ALT, etc.
- Page or Post
- Yoast Related XML stuff on Page or Post
- Database: wp_posts, GUID (For RSS feeds, Created when Page or Post is initially created, never seems to get updated if anything else on a page is changed (Permalink, etc.))