My Favorite Website Problem Checker

Xenu Link Sleuth to the rescue.

Xenu Link Sleuth is my favorite website problem checker. It is the best broken link checker I have found. I use it like a website problem checker. I can use it to answer questions like, "do I have any links that go to websites that are no longer online?"

Screenshot from Xenu's Link Sleuth Website
Xenu Link Sleuth from 

Pink lemonade sitting next to keyboard representing "link juice"
Link Juice of a different kind...(Photo by J. Kelly Brito on Unsplash)

Do you know where you are going to? 

Is your "link juice" flowing to places to which you had no idea you were linking? Let me explain. Website owners are frequently linking to places outside of their direct control. 

The easiest way I found to avoid issues is to use Xenu Link Sleuth periodically, to run a report on the website. 

It is multi-threaded, fast, and free. However, it only runs on Windows. You'll need to look for another tool if you need an online tool to extract links from a web page.

Xenu Link Sleuth screenshot

I used it a few hours to go to get a list of all links from my website. I had recently purchased a commercial theme for this website and after it was installed I had graphics and menus that I knew I had not uploaded showing up on some of my pages. So, I turned to link sleuth to make quick work of it.

The commercial theme was small and could not have uploaded images to my server. So, it was time to do some detective work.

Stereotypical spy, a silhouette of man with hat and raincoat.
Who are you linking to? Photo by Craig Whitehead on Unsplash


I had no menu so my site was using theirs.

I discovered that there were many options linked to their hosts because I had not yet defined those options. For example, I had no "documentation" choice on my drop-down menu, but after installing their theme I had a new menu complete with irrelevant links, presumably as a demonstration? 

I used Xenu Link Sleuth on my website.

Xenu Link Sleuth scanned my website producing a list of all URLs, including css, js, jpg, png, gif, etc. I was not pleased. I had to delete these links, but I had to find them. If one clicked on these links, one would find themselves looking a website theme store. Certainly, that was not the user experience I had intended to deliver.

By pasting the output into a spreadsheet, one can do some pretty elaborate filters, like show me all .jpg links being loaded from a particular domain.

Other uses for Xenu are to find orphan pages in a website. These are pages that don't aren't link to other pages on your site.

It is also a handy tool to get all links from a website. You can use it on websites with which you have no connection as a competitive intelligence tool.

Denial of Service Attack?

Scouts honor sign meaning don't misuse these tools.
Tools can be misused...(Photo by Matt Seymour on Unsplash)
A word of warning. Since its multi-threaded, Xenu can cause a great spike in a server volume. Theoretically, Xenu traffic might look like a denial-of-service attack. Multiple copies of Xenu running multithreaded sessions on a server would be server-intensive and could deny other users access, which is the goal of a Distributed Denial-of-Service attack. So, repeat after me, "Scouts Honor, I am not a bad guy. I will only use Xenu for good."

Follow Robots.txt (or don't.)

Link sleuth, by default, will honor the robots.txt file, a file that webmasters configure to direct spiders to index only certain areas of the server. However, Link Sleuth can be configured to ignore robots.txt.

Put Xenu Link Sleuth in your toolbox.

I think you will find that Xenu Link Sleuth is a great addition to your website tools. I know this reads like an affiliate link post, but Xenu is not for sale, it is free. 

Don't Give Away Even Imaginary 'Link Juice'.

I am just a satisfied user. Xenu fits a need. Until I get this newly created website under control, I'll be running Link Sleuth to keep tabs on what add-ins add-ins are doing. The proverbial "link juice" is important. One can use Xenu to see where the link juice is flowing...

Day 4, Stay Safe, Dan