Not only will a link checker expose pages that are 404′ing, it will also expose any pages that are flat out 500′ing or returning any other kind of unwanted response.
Link checkers are very handy for checking for errors on your website. If your site has a lot of 404, 500, 503 and whatnot pages then it will definitely affect not only your visitors experience of your site, it will also affect your ranking in search engines as an unstable or broken website is certainly an undesirable one.
I tend to run it a link checker regularly on sites that I work on, and always a before a release as part of the pre-release regression process. If I’m doing any redirect work then I will use a link checker regularly to validate whether I have introduced any errors once the work is complete.
Xenu link checker is free
Xenu Link Checker is an awesome free link checker, it’s easy to use and the data it produces is easy to filter and work with.
Before you crawl a site check Xenu’s preferences under the options drop down, in here you can specify the depth to which you want Xenu to crawl your site. I normally go for a depth of 4-6 on large websites, otherwise the crawl can take hours and hours to complete. I also never bother checking external links, since for the most part they are out of my control and will most likely add a lot of extra links to the crawl. Also, when crawling a site be aware that it will of course cause extra load on said site, I tend to only use Xenu in Dev or QA environment as opposed to a live site.
Once the crawl has completed Xenu will ask if you want it to compile a report for you, I don’t normally bother with as the standard set of results that it leaves you with are normally all you will need. You can sort the results by status so 404′s, 500′s and whatnot will be organised together so you can see any faults in your site. All you need to do now is check and fix and of the errors, simple!