The Mozilla user experience team often designs features that represent sites to users in a variety of ways. For example, Firefox tabs display favicons and page titles, while Panorama displays favicons, titles, and page thumbnails. So, I thought it would be useful to investigate the effectiveness of various ways of representing sites to users.
One interesting piece of research on page representation was published by Shaun Kaasten, Saul Greenberg, and Christopher Edwards at the University of Calgary in their paper How People Recognize Previously Seen Web Pages from Titles, URLs and Thumbnails (download it here). This team conducted a series of studies, most of which involved increasing one variable which represented a site the user had previously visited (such as thumbnail size) until the user recognized it, at which point the user would buzz in to stop the expansion and identify the site.
Here’s some key takeaways from what the Canadians learned:
– The graph above plots the thumbnail sizes at which test participants could recognize a domain (black lines) and a specific page within a domain (blue lines). The dotted lines show all responses, and the solid lines show only correct responses. You can see that by the time a thumbnail was 962 pixels, 60% of test subjects had identified it. 80% of test subjects identified sites by 1442 pixels, and by 3042 pixels everyone had identified the site.
– Users’ guesses about what site a thumbnail was representing were correct about 90% of the time. Not bad, considering on most sites they had no readable text to go by until the thumbnail was over 962 pixels. This shows how effective thumbnails are at identifying sites to users.
– Color and layout in were the most important factors for identifying a site when the thumbnail was 642 pixels and smaller. From 642 to 962 pixels, color, layout, images, and text were equally important. Above 1002 pixels, text was most important. This is presumably because at that size, sites were not yet identified because they were visually similar to other sites and text was the only effective differentiator.
– Looking at only truncated URLs and page titles, test subjects could correctly identify sites 90% of the time. The researchers experimented with URL and title representation by showing users right, middle, and left truncated strings and recording when they buzzed in to identify the site correctly.
– The graph above shows the running sum of correct answers in identifying sites based on only page title (top graph) and URL (bottom graph). You can see that right truncation proved the most effective for domain-level site identification. For titles and URLS that were truncated on the right, sites were correctly identified 15% of the time with 5-6 characters revealed, 30% of the time with 8 characters, 60% of the time with 13-15 characters, and 80% of the time with 25-31 characters. Left truncation was the most effective for identifying a specific site within a domain. So, if you want users to identify a site based on a string, at least 15ish characters are needed for even a majority. If you want users to identify a subdomain, clip right left side of the URL. To idenfiy the domain itself, clip the right.
June 15, 2011
Whenever you open a new tab in Firefox, your goal is to navigate somewhere. To aid your navigation, on this new tab Firefox currently offers you… nothing. Just a blank page. 100% white, and 100% not useful.
Firefox has been displaying this blank page when users open a new tab for as long as there’s been a new tab. And, partially, it’s deliberate. After all, a blank page is guaranteed not to distract you from your current task. It’s just clean and white, like a canvas, offering no suggestions for the next move and no distractions from it. Alex Faaborg explains very well in his recent blog post the concerns we have with distracting users and the ways that data overload on a new tab page can be harmful.
This isn’t the case when you open a new tab in other browsers. Opera was the first to offer a “Speed Dial” with giant thumbnails linking users to their most frequented sites. Safari’s giant wall-o-televisions offers much the same. Chrome has played around with different designs, first trying a speed dial like Opera’s and later integrating other content, such as apps. Internet Explorer, the most unusual of the designs, offers you some options: reopen closed tabs and sessions, start private browsing, or use an “Accelerator,” which usually means do “something with Bing.”
So, which approach is best for our users? Would presenting large thumbnail targets to direct people to sites they frequently visit save them time? Could we present information to make it easier for users to navigate to their next destination? Can we do so without being distracting and leading users away from the task they had in mind?
We realized that we couldn’t answer these questions without finding out more about our users. So, a few people at Mozilla are heading up studies to find out how people use tabs and how different designs of new tab page effect how they browse and user the web.
Here’s what’s going down:
Intern Lilian Weng is currently working on a quantitative study within Test Pilot to capture data on what users do after they open a new tab. This should answer questions surrounding user intention when opening a new tab, and possibly how long users take to perform actions after opening a new tab.
Interns Diyang Tang and Lilian Weng are preparing to do an A/B test using Test Pilot to determine how user behavior differs when presented with a new tab page vs. none. They are attempting to answer questions such as:
– Does a new tab page discourage breadth in visited sites?
– How do users navigate to a website after they open a new tab in each scenario? (location bar, search bar, top sites, bookmarks, history, etc.)
– Are there users who are more mouse-based and some who are more keyboard-based? How does a new tab page affect them?
Diane Loviglio and myself are preparing more qualitative “cafe” tests to gain insight into how people use tabs currently. We’d like to know why and when users open new tabs in a more contextual perspective than Test Pilot data provides. Our goal is to find a wide enough range of users that the most common new tab behaviors can be grouped and discussed in a more tractable framework.
Once the research from tests 1-3 is available, variations on new tab pages will be implemented and tried out with real users. There are multiple testing methods that could be useful here, such as a multivariate testing or even journaling to gain insight into how new tab pages effect behavior of a user over time.
Not quite a research project, but intern Abhinav Sharma is designing and implementing an experimental new tab page which uses contextual information about a user’s current browsing session to offer suggestions. His page makes intelligent recommendations about where you’re likely to go next based on where you’ve been. The project’s still in alpha, but you can see the code he’s done already for a basic speed dial implementation on his github.
You’ll notice that a lot of this work is being done by our awesome new Summer 2011 interns! It’s only early June and they’re already rocking hard.
I’ll post what we learn from these studies as results come in. I predict we’ll gain some insight into user behavior that will inform not only Firefox’s new tab design, but many other features besides!