July 21, 2011
I’d like to highlight the awesome research project that intern Lilian Weng is leading around Firefox’s new tab page.
While our goal is to make users more efficient at their browsing tasks, what makes them more efficient is a question we keep returning to. Most other browsers display links on new tab pages based on frecency. Frecency is a portmanteau which combines frequency and recency. At Mozilla, we use it to refer to sites that users have been to often, recently, or both. It’s how we calculate what should be the first, second, third, etc site that appears when you type a letter into Firefox’s URL bar.
Using frecency to list links on a new tab page seems an obvious design direction, but we want to truly investigate whether another solution would be best for users. So, Lilian is spinning up a brave new study. Once her test is ready, users of Test Pilot, our platform for collecting structured feedback on Firefox, will be asked if they’d like to participate in a new study. If they say yes, they will be randomly assigned one of six new designs on their new, blank Firefox tabs. One of these six designs will be our control group: a blank white tab, just as Firefox users see currently. The other five will look almost identical to each other. They will display a simple 8×8 grid of favicons set on a button which is colored to highlight them based on a color-matching algorithm designed by Margaret Leibovic:
The only variable that will be changing among the five designs is which sites are displayed in this grid. Here’s the five variations we’re testing:
- Frecency. A combination of a user’s most frequently and most recently visited sites.
- Most recently bookmarked sites. By displaying prominently what a user has recently starred, we effectively turn the new tab page into a read it later list.
- Most recently closed sites. This could lead users to treat new tab page as an undo feature, or close tabs in order to temporarily store them in the new tab page as a short-term read it later list.
- Sites based on content similarity. Intern Abhinav Sharma is trying out his project, called Predictive Newtabs, which displays sites based on where the user has opened a new tab from. For instance, if the user has been browsing a news site, a new tab would offer other news sites the user has been to.
- Sites based on groups of sites frequently visited together. In another part of Abhinav’s Predictive Newtabs experiments, he has designed an algorithm to predict sites to show based on sites users visit in groups. For instance, if every time you get to work you first check the weather and then check stock prices, this new tab would offer you a stock page on a new tab after you checked the weather. If you want to try this experiment out yourself, you can download the Jetpack here.
The above study is still in preparation, and once it goes live I predict that we’ll learn tons of valuable information about how new tab suggestions can positively impact users. Lilian will be collecting data on many aspects of users’ responses to these designs, such as how they effect the breadth of sites users visit, how likely they are to click on each item in the grid, and how long they spend deciding where to navigate. I can’t wait to start pouring over the data that comes back: it’s very new research in an area that has a profound impact on how we use the web.
June 15, 2011
Whenever you open a new tab in Firefox, your goal is to navigate somewhere. To aid your navigation, on this new tab Firefox currently offers you… nothing. Just a blank page. 100% white, and 100% not useful.
Firefox has been displaying this blank page when users open a new tab for as long as there’s been a new tab. And, partially, it’s deliberate. After all, a blank page is guaranteed not to distract you from your current task. It’s just clean and white, like a canvas, offering no suggestions for the next move and no distractions from it. Alex Faaborg explains very well in his recent blog post the concerns we have with distracting users and the ways that data overload on a new tab page can be harmful.
This isn’t the case when you open a new tab in other browsers. Opera was the first to offer a “Speed Dial” with giant thumbnails linking users to their most frequented sites. Safari’s giant wall-o-televisions offers much the same. Chrome has played around with different designs, first trying a speed dial like Opera’s and later integrating other content, such as apps. Internet Explorer, the most unusual of the designs, offers you some options: reopen closed tabs and sessions, start private browsing, or use an “Accelerator,” which usually means do “something with Bing.”
So, which approach is best for our users? Would presenting large thumbnail targets to direct people to sites they frequently visit save them time? Could we present information to make it easier for users to navigate to their next destination? Can we do so without being distracting and leading users away from the task they had in mind?
We realized that we couldn’t answer these questions without finding out more about our users. So, a few people at Mozilla are heading up studies to find out how people use tabs and how different designs of new tab page effect how they browse and user the web.
Here’s what’s going down:
Intern Lilian Weng is currently working on a quantitative study within Test Pilot to capture data on what users do after they open a new tab. This should answer questions surrounding user intention when opening a new tab, and possibly how long users take to perform actions after opening a new tab.
Interns Diyang Tang and Lilian Weng are preparing to do an A/B test using Test Pilot to determine how user behavior differs when presented with a new tab page vs. none. They are attempting to answer questions such as:
– Does a new tab page discourage breadth in visited sites?
– How do users navigate to a website after they open a new tab in each scenario? (location bar, search bar, top sites, bookmarks, history, etc.)
– Are there users who are more mouse-based and some who are more keyboard-based? How does a new tab page affect them?
Diane Loviglio and myself are preparing more qualitative “cafe” tests to gain insight into how people use tabs currently. We’d like to know why and when users open new tabs in a more contextual perspective than Test Pilot data provides. Our goal is to find a wide enough range of users that the most common new tab behaviors can be grouped and discussed in a more tractable framework.
Once the research from tests 1-3 is available, variations on new tab pages will be implemented and tried out with real users. There are multiple testing methods that could be useful here, such as a multivariate testing or even journaling to gain insight into how new tab pages effect behavior of a user over time.
Not quite a research project, but intern Abhinav Sharma is designing and implementing an experimental new tab page which uses contextual information about a user’s current browsing session to offer suggestions. His page makes intelligent recommendations about where you’re likely to go next based on where you’ve been. The project’s still in alpha, but you can see the code he’s done already for a basic speed dial implementation on his github.
You’ll notice that a lot of this work is being done by our awesome new Summer 2011 interns! It’s only early June and they’re already rocking hard.
I’ll post what we learn from these studies as results come in. I predict we’ll gain some insight into user behavior that will inform not only Firefox’s new tab design, but many other features besides!