Improving everyone’s favorite feature – address not found
March 10, 2009
One area of Firefox that could use some improvement are the warnings we give when pages are not found. These could be network errors, firewall issues, URL typos, etc. Curtis Bartley has been looking at this issue and documenting progress here, and Jesse Ruderman started a bug with some insightful comments here.
Ideally, a good error page does two things:
- Tells you what’s wrong in a way that’s both understandable and diagnosable
- Helps you decide what to do next
If I type “www.example.cmm” into Firefox now, here’s what I get:
Internet Explorer, Chrome, and Safari all have slight variations on this page. IE’s isn’t very useful; it gives easy text but no tools. Safari’s design is very minimal, but it does offer search. Chrome is cutesy, but also intelligently offers suggestions based on what was typed – the most useful of all three. All three browsers offer additional information via a link, thus removing the bulk of explanation Firefox currently shows.
So what should Firefox do?
For blatant, common URL typos, I think we should redirect straight to the correct URL. Google.com currently goes to http://www.google.com, why shouldn’t google..com or ww.google.com? URLs themselves are an example of forcing users to behave like machines – rather than the preferable reverse – so why not take a step in the right direction by making them more forgiving? (bug freakin’ 175634)
For the addresses that are not blatant typos for existing pages, there’s a few approaches we could take. User-experience-wise, I would love to be able to do better detection of what the problem is and direct the user towards the likely solution. Rather than presenting the user with questions as we do now, we could give them an answer. We should aim to provide a consistent UI, but perhaps we could change the text sightly based on what the user inputted and try to find the most helpful suggestion. When there’s a very likely solution, that should be the most obvious next step available.
Here’s four scenarios that would cause a 404 today with text customized to what what the user imputed:
While catching all of these scenarios may not be feasible for now, we could be concentrating harder on giving users tools at a 404, not just a warning. I’d love to hear you feedback on this – especially what would help a tech-saavy person such as yourself when you hit a 404.
Just for fun… spoilers!
Jennifer Morrow's Blog 
March 10, 2009 at 9:09 am
Guessing what a typo meant is probably a bad idea–that’s actually something that’s under a webmaster’s control, if they want to do that. Most webmasters do redirect “google.com” to “www.google.com”. Also, I think you might considerably increase DNS load worldwide by simply doing an immediate second (or third, or fourth) query every time somebody mistyped something. But that might not be so bad.
Also, ICANN apparently plans to start selling TLDs relatively soon, and it’s theoretically possible that somebody could buy “.con” or “.cmm” (although that would be a really stupid investment at $100,000).
I think that having a different error page depending on what the user did and what the actual situation is is a good idea, though.
Note that the browser actually can’t tell, technically, whether you were stopped at the firewall on your side or a firewall on the web-server side (without doing a traceroute, which I doubt you’d want to do before showing an error page).
I like the idea of “view an archived version”, although I suppose you’d then have to start defining an “archive provider” and either use Google’s cache or the Wayback Machine, either of which could (a) fail to have a URL or (b) [unlikely] stop existing. Also, you’d have to check to see if there was an archived version before displaying the error page (or AJAX it to check while displaying the error), and make sure that the archive provider was okay with the increased load.
-Max
March 10, 2009 at 9:12 am
Oh, actually, after looking at what Chrome does, I think that that sort of typo suggestion could totally work. If they implemented in some performant way, it must be possible.
-Max
March 10, 2009 at 9:59 am
Hey, Boriss. Max made all of my points already so, I’ll just agree with him. Correcting TLDs automatically seems like a bad idea, as they are going to be for sale soon, and, really, there’s a lot of random ones out there. Suggesting alternatives for the user to pick seems great, though.
March 10, 2009 at 10:25 am
The “Try Again” button does work in the latter two cases (always assuming you were able to fix your settings) as well in the case of the dodgy wifi, which is something I’m personally experiencing 😦
March 10, 2009 at 10:29 am
Nooo!
Don’t take away reload!
I use that often. Really.
I find myself in situations where the page didn’t load for whatever reason and reloading actually works. Sometimes I’ve got too many tabs open on a restart of the browser and my puny connection can’t handle it and several pages don’t load. Sometimes I go from home to work and didn’t change my proxy yet (Firefox on OS X *still* doesn’t support the native proxy settings). Sometimes wifi is flaky or I got disconnected or my home connection gets flaky. In all these circumstances I have use for the reload button.
Yes, I can always use the toolbar reload but, 1) my cursor is more likely to be lower so it’s convenient and 2) most ‘normal’ internet users must have the options they find viable presented to them in the 404 info page.
March 10, 2009 at 10:32 am
At least some way to tell the difference between a DNS timeout, DNS NACK, a connection refused, a 404, and connection timeout, etc would be good. Also use of some of the conectivity awareness apis would be good for things like “Wifi disconnected” or “No network adapters with IP bound”, etc.
March 10, 2009 at 11:08 am
That’s no 404 what the screenshots are showing. A 404 is a response from a webserver indicating that a page isn’t available. The webserver itself can be configured to deliver useful information on what went wrong and what alternatives exist. That’s nothing a browser should interfere with. What you’re writing about is not being able to contact a webserver at the given hostname:port combo. There understandable *and* technically correct information from the browser is very welcome.
March 10, 2009 at 11:28 am
Did you test the cheesy distance algorithm on non-ascii urls yet?
I don’t think that intercepting 404 pages from web sites is cool.
… and given that we’re string frozen, I don’t see this going into 3.5.
March 10, 2009 at 11:45 am
I like it 😉
March 10, 2009 at 11:59 am
Safari 4 has 2 nice features
(1) a button for Apple’s built-in network diagnostics / repair tool. Windows has a similar feature (Repair Connection) which also be targeted.
(2) If the connection is temporarily down, and then the connection comes back up while I still have the “page load error” page open, it does a reload. Very nice for flakey wireless connections.
March 10, 2009 at 12:32 pm
The screenshots comparing Firefox to IE, Chrome and Safari aren’t all of the same situation. Firefox and Safari are dealing with a failed DNS lookup, whereas the other 2 are dealing with a 404.
I don’t like the idea of overriding a website’s 404 pages.
March 10, 2009 at 12:35 pm
Automatic typo redirect is going to make many webmasters rather angry with you. It also seems to me to be exploitable for phishing (what, your bank removed a page – what if it decides to redirect to the same URL on a phisher-owned typo domain?)
Site search – well, that’s a totally different matter and a useful addition. Should be open to using different search providers (Google, Yahoo!, etc) though.
March 10, 2009 at 12:41 pm
I find it a great idea to tell users what really happened, but most of the time, “usual users” don’t read any messages
.
March 10, 2009 at 1:26 pm
1: Do you suggest to override the server’s 404 page or do you only talk about when the server does not provide any body in a 404 message?
2: Of your four suggestions, the last two does not make much sense for a 404. It might makes sense for other networking errors, but when you get a 404 error back from the server, you know that there is a connection.
3: How would you make the suggestion? Search in the Places database? You cannot query some search engine or other online service because of privacy issues.
4: I think there are many other network errors which are much more interesting to discuss, for example the error you have a screenshot of in the beginning of your post.
March 10, 2009 at 1:37 pm
Well, as far as I can think of possible situations, all the comments Jesse made in the bug are good, and what Chrome did when I tried it briefly seemed cool too (but they do seem to depend on it constantly talking to Google’s servers).
Firefox should be really really careful about actually deciding answers though.
Having used other URL-correcting things (either from a DNS provider, or as an addon), it’s really bad when they think they know what you want but get it wrong. Instead of the site you wanted, you get some other site and the browser is then busy (and often rather unresponsive) loading up a whole page of content, plugins and whatever else while you’re frantically clicking buttons to stop it doing what you didn’t want, and then rather than correcting starting from what you typed, you’re correcting from whatever URL you have been redirected to.
Getting the errors messages is also bad – taking a random one of your examples, giving a link to their firewall settings is likely to cause more harm than good if Firefox isn’t 100% sure that those settings are the problem. Most people won’t understand the settings when they see them, and if they’re actually not the problem, they can only make things worse.
The biggest thing would be allowing me to easily edit whatever I typed – generally I can instantly see the problem, but Firefox is busy loading things and redirecting and whatever, so it actually takes a few seconds to make it stop and let me edit what I typed.
Fixing bug 175634 would be cool. Offering to search for a site when the server can’t be found/reached would also be good, as that’s usually what I do anyway, but that depends on it being a problem at their end and not my end, and Firefox doesn’t know that. So fixing bug 196076 would also be good. Being a tech-savvy user, actually just telling me the error would be fine. But doing most of the other things suggested pretty much depend on that too. At the moment Firefox has no clue (and so gives me no clue) whether it’s just the one site I tried that doesn’t exist, or if the connection to the internet is broken (well, connection my internet provider’s DNS).
I’m not at all sure about the Google cache/ archive.org links. They are pretty patchy to work with – I’m not sure displaying outdated and often quite broken (due to missing scripts/styles) sites is something to unleash on the world in general.
Lastly, you’ve mentioned 404 – that’s only for when a site is found but the page isn’t. Most sites do their own thing with that, and I’d hate to have Firefox come up with its own idea of what I want to see in that case rather than giving me what the site provided. I think (and hope) we’re mostly talking about times when the site is unreachable…
March 10, 2009 at 2:08 pm
I think I have to agree with Max and Michael — be very careful before you start assuming you know what the user wants. Even if you get it right 99.99% of the time, that .01% when you get it wildly wrong will be enough to make Firefox suck, and you’ll never get it right all the time.
(Easy example: When I type google.co, I swear aloud because I meant to type google.com. But I’m sure there’s someone out there who types google.co, but meant google.co.uk. If you ever try to correct that typo, one of us is going to lose.)
I would definitely support a “Did you mean?” link, but I’m not at all convinced that auto-redirection is ever a good idea. For a good time, go read the Slashdot threads that have happened when ISPs have tried to redirect DNS errors; then imagine getting one of those pointed at Firefox. The issues involved are obviously slightly different, but there are some in common.
http://tech.slashdot.org/article.pl?sid=08/09/30/1236229&from=rss
http://tech.slashdot.org/article.pl?sid=08/07/19/158208&from=rss
http://www.networkworld.com/news/2007/110907-verizon-redirects.html
If you very carefully never ever redirect a typo to any kind of search page, you might be able to avoid the shitstorm; but Max has also pointed out several solid technical reasons why you ought not be doing that.
It’s definitely not a good idea to be doing more queries before you give me my error page; I want it as quickly as possible, so I can type in what I really wanted and be on my way. If you start making archive.org and google.com queries before giving me an error, you’re not helping.
March 10, 2009 at 2:31 pm
Suggesting likely corrections sounds great, but just like the other commenters, my first thought after reading the article was: please do not make the browser go automatically to the site it’s guessing I wanted to type!
March 10, 2009 at 2:34 pm
Oh, and I do use that “Try again” button pretty often. It takes time for the computer to reconnect after e.g. returning from hibernation, or sometimes I forget to switch the net connection to one that works for the current location.
March 10, 2009 at 3:34 pm
Not to be a pedant, but the word’s input not inputted 🙂
I like most of what you outline above, but like others I’m not fond of the idea of the browser presuming it knows what the user meant to request and automatically redirecting. I like the idea of suggesting these destinations to the user and leaving it up to them, however.
March 10, 2009 at 4:36 pm
@Jesper –
1. No way, I don’t think we should ever override a server’s 404 pages – this is only when there isn’t one. 4chan alone would kill us.
2. Ok, I’m using the idea of address not found very broadly here, not just for 404s – I should fix that and not mention 404s in this post in fact.
3. That’s why I said this isn’t actually possible right now – I’m thinking about what we’d *like* user experience-wise rather than what is practical to implement. I believe it’s important to start here before we come down to what can be done and how – otherwise we end up adding what’s technical feasible and can forget what we wanted for users.
March 10, 2009 at 4:55 pm
I’d love to see sites in my places that are close matches to the 404. (favorite pages in the same domain, domains that nearly match, etc)
I’d love to be able to report the broken page to the webmaster automatically with a single click, if there was an elegant way to whois the domain and get the technical contact email.
March 10, 2009 at 5:22 pm
> For blatant, common URL typos, I think we should
> redirect straight to the correct URL.
That sounds like a security nightmare: all it takes is knocking a DNS server off the network (say with a DOS) and crafting something that makes us guess wrong for an attacker to force a load of a malicious site.
March 10, 2009 at 5:43 pm
For my usage scenarios, I think leveraging the cache could be very helpful. The vast majority of 404s I get are mis-typings of frequently-visited pages (because since they are frequently-visited, I’m typing too fast). So rather than trying to resolve http://www.firefox.con by assuming I meant a different TLD and trying that, another response to the 404 could be to rummage through my cache with a fuzzy text match and see if I might have meant something I often visit.
The awesome bar in Firefox 3 already does this to a certain extent; this suggestion is to essentially take smarts similar to those found in the awesome bar and apply them to the default (non-server-supplied) 404.
March 11, 2009 at 8:59 am
What kind of server doesn’t provide a 404 page? Surely not the bbc.co.uk webserver, I just checked.
1. Make 100% sure you do not drop the actual error code, you can’t debug problems if you do.
2. Try ‘The website cannot be found’ vs. ‘the webpage cannot be found’ for dns errors vs. 404 errors
3. You may dream about hiding the existence of hosts from the user, but on the other side every browser is emphasizing the host part of the URL as that is the only part that cannot be spoofed, especially when SSL is used. Pointless.
4. Transient dns problems are a fact of life, that’s why there is a reload button. They happen on dodgy wifi connections and on spiffy T1 connections.
I don’t quite understand people. When using the telephone, people are quite happily recognizing weird tone combinations for ‘no response’, ‘the number doesn’t exist’ and ‘the area doesn’t exist’. Why is it that when computers are involved, they suddenly become so retarded they can’t understand a simple message? Perhaps the error page should play sound clips of the phone codes in the background.
October 1, 2009 at 9:57 am
The thing what I can’t understand is how can you offer suggestions based on the place DB ?. Because as Jesper said generally you aren’t allow to access to place DB.