October 19, 2011
Mozilla’s manifesto describes the internet as an integral part of modern life and a key component in communication. However, communication on the web has far to go before it’s as rich as face-to-face communication. Real-time video communication on the web should be easy, rich, and readily available to developers in a way that proprietary formats can’t be.
That’s why a new project is spinning up at Mozilla called WebRTC (Real-Time Communication). WebRTC will allow developers to use the web platform to include video and audio conferencing as part of their websites and applications, both mobile and on the desktop. In its first phase, WebRTC will make webcam feeds a primary object in the browser, allowing sites to create rich interactions such as video calling and conferencing. In later phases, WebRTC will allow interactions like co-browsing, in which users can share their screen with a friend.
Privacy and Security
Privacy and security are major concern in enabling open video communication on the web. A face and voice are two of the most identifiable kinds of shareable data, and keeping users in absolute control of who has access to them is vital. As the IETF states in its WebRTC draft document, the ability for users to control access to their webcam, be able to cancel communication at any time, and not be eavesdropped upon are essential.
Even a trusted site could be compromised, both during a call or after. And, since the sites themselves would control and display the UI of the call itself, Firefox needs to give the user both constant indication that they are in a call and the ability to disconnect at any time.
However, guarding against threats only goes so far towards keeping users in control of their webcam communication. Clear messaging, useful tools, and sensible defaults need to be in place for video conferencing to safely take root in the browser.
The first phase of enabling WebRTC will allow the most basic use case: giving a site access to a user’s webcam and microphone. The browser already serves as a mediator for other user data, such as location and access to cookies. Firefox usually asks for permissions using a door hanger notification. Door hangers stem from the URL bar to show the site is asking for a permission, and it extends past the content area to show that Firefox is the mediator of the permission request. Using a door hanger notification for WebRTC is both consistent within Firefox and correctly conveys visually that the site has requested access, and Firefox is asking the user for that permission.
Usually, these door hangers simply ask the user for a permission, and in a click the user can give it. However, webcam access requires a secondary stage: showing a preview of the webcam feed. This approach has three benefits:
- It gives users the ability to make sure their webcam and microphone work correctly
- If users had casually or accidentally accepted the webcam permission, nothing makes people more aware of what they’re about to transmit like showing them their own grubby mug
- It gives users the ability to fix their hair/put on a shirt/remove incriminating items from background before beginning call
In some ways, it’s unfortunate to ask users to pass through two dialogs to give webcam feed rather than one. After all, in most cases the site itself will be providing all necessary UI, and perhaps even a video preview before a call is initiated. So, this could all be redundant in many cases. However, we cannot predict what purpose a site may be requesting webcam feed for, nor what UI will be in place for the user on that page. Even with all our efforts against security threats, any request for webcam access must be treated as potentially malicious.
Once a user has given a site access to their webcam and is likely engaging in face-to-face communication, that interaction should be given a heightened level of priority within the browser. For a user to lose that tab or forget they are broadcasting could range from mildly embarrassing to, well, use your imagination. If a user is actively sharing their webcam feed, they should be able to jump to the tab where data’s being shared or simply cut their webcam feed from anywhere within Firefox. This will require at the very least a toolbar-level Firefox control that appears once a user’s actively sharing.
Designing and implementing a new API is always a complex process. If you’re interested in reading more or contributing to this project, here are some resources:
- Mozilla WebRTC feature page
- Mozilla notes on first WebRTC security discussion
- The IETF’s draft document on WebRTC Use-cases and Requirements
- Robert O’Callahan MediaStream Processing API Proposal
- Mozilla’s RTC API Proposal on GitHub and on Web Activities, a service discovery mechanism and light-weight RPC system between web apps and browsers
- Eric Rescorla’s paper on WebRTC Security Considerations, and his corresponding presentation slides (PDF)
- Cullen Jennings’s PDF slides on WebRTC API Design Questions
- W3C WebRTC meeting notes, including a PDF of Mozilla’s implementation status
June 15, 2011
Whenever you open a new tab in Firefox, your goal is to navigate somewhere. To aid your navigation, on this new tab Firefox currently offers you… nothing. Just a blank page. 100% white, and 100% not useful.
Firefox has been displaying this blank page when users open a new tab for as long as there’s been a new tab. And, partially, it’s deliberate. After all, a blank page is guaranteed not to distract you from your current task. It’s just clean and white, like a canvas, offering no suggestions for the next move and no distractions from it. Alex Faaborg explains very well in his recent blog post the concerns we have with distracting users and the ways that data overload on a new tab page can be harmful.
This isn’t the case when you open a new tab in other browsers. Opera was the first to offer a “Speed Dial” with giant thumbnails linking users to their most frequented sites. Safari’s giant wall-o-televisions offers much the same. Chrome has played around with different designs, first trying a speed dial like Opera’s and later integrating other content, such as apps. Internet Explorer, the most unusual of the designs, offers you some options: reopen closed tabs and sessions, start private browsing, or use an “Accelerator,” which usually means do “something with Bing.”
So, which approach is best for our users? Would presenting large thumbnail targets to direct people to sites they frequently visit save them time? Could we present information to make it easier for users to navigate to their next destination? Can we do so without being distracting and leading users away from the task they had in mind?
We realized that we couldn’t answer these questions without finding out more about our users. So, a few people at Mozilla are heading up studies to find out how people use tabs and how different designs of new tab page effect how they browse and user the web.
Here’s what’s going down:
Intern Lilian Weng is currently working on a quantitative study within Test Pilot to capture data on what users do after they open a new tab. This should answer questions surrounding user intention when opening a new tab, and possibly how long users take to perform actions after opening a new tab.
Interns Diyang Tang and Lilian Weng are preparing to do an A/B test using Test Pilot to determine how user behavior differs when presented with a new tab page vs. none. They are attempting to answer questions such as:
– Does a new tab page discourage breadth in visited sites?
– How do users navigate to a website after they open a new tab in each scenario? (location bar, search bar, top sites, bookmarks, history, etc.)
– Are there users who are more mouse-based and some who are more keyboard-based? How does a new tab page affect them?
Diane Loviglio and myself are preparing more qualitative “cafe” tests to gain insight into how people use tabs currently. We’d like to know why and when users open new tabs in a more contextual perspective than Test Pilot data provides. Our goal is to find a wide enough range of users that the most common new tab behaviors can be grouped and discussed in a more tractable framework.
Once the research from tests 1-3 is available, variations on new tab pages will be implemented and tried out with real users. There are multiple testing methods that could be useful here, such as a multivariate testing or even journaling to gain insight into how new tab pages effect behavior of a user over time.
Not quite a research project, but intern Abhinav Sharma is designing and implementing an experimental new tab page which uses contextual information about a user’s current browsing session to offer suggestions. His page makes intelligent recommendations about where you’re likely to go next based on where you’ve been. The project’s still in alpha, but you can see the code he’s done already for a basic speed dial implementation on his github.
You’ll notice that a lot of this work is being done by our awesome new Summer 2011 interns! It’s only early June and they’re already rocking hard.
I’ll post what we learn from these studies as results come in. I predict we’ll gain some insight into user behavior that will inform not only Firefox’s new tab design, but many other features besides!
January 26, 2011
As we approach the release of Firefox 4, the last few polish and stylistic changes are happening in the add-ons manager. Some are simply graphic cleanup, while others are the result of beta testing the new manager for the past several months.
I wanted to highlight one change in particular that you’ll be seeing in the Firefox nightlies soon. The date an add-on was last updated, rather than being displayed in list view, will now only appear in the detailed view of an add-on. This also means that installed add-ons can no longer be sorted by last updated date.
For some users, this change is substantive and will feel disruptive. So, I wanted to give the rationale behind this design decision.
1. Providing a simplified overview
The intended purpose of the add-on manager’s list view is to give a brief overview of the users’ add-ons and to provide only the minimal, most used information and functionality. This minimal information is the name of an add-on, its icon, and a short description. The minimal functionality is the ability to disable and remove an add-on. Even the author name we’ve removed to provide the simplest, most visually scannable design. By removing the last updated date, we not only visually clean an add-on’s list entry, but also eliminate the need for a sorting bar at the top of list view. This gives back both whitespace and a cleaner appearance at the top of the list.
2. Updated date does not provide important functionality for most users
For most users, the last updated date does not give information meaningful enough to justify its placement in list view. It allows users to see which add-ons have been updated automatically most recently, but does not give any details about the updates nor provide tools to interpret the information.
Some advanced users use the last updated date as a diagnostic tool to identify which add-on updates may be causing a recent problem in Firefox. However, the date makes a very poor diagnostic tool. One reason is that the date does not give any information about the size nor scope of the update, and thus can only be used for diagnosis by disabling one add-on at a time to isolate a problem. In many cases, a problem in Firefox caused by an add-on are instantly identifiable as being caused by a particular add-on. Even in the rare case where a problem suddenly appears in Firefox, the chances of it being from an add-on update are not large. A problem could be caused by any number of online events, which is why Firefox provides tools such as the Error Console and about:crashes to help diagnose them. And, even if we were to give fuller information about updates in the add-ons manager and make it into a better diagnostic tool, why should this tool be so far removed from other diagnostic tools? How could a new user figure out that, to access diagnostic tools related to add-ons, they should go to the add-ons manager rather than a more comprehensive diagnostic tool? It would be wildly inefficient to apply this elsewhere in Firefox by placing diagnostic tools only on the interface elements they relate to.
What we should do is add diagnostic tools about add-ons to comprehensive tools such as about:support. Then, we could provide expert users the information they want in a better format while keeping one-off diagnosis away from list view in the add-ons manager.
3. The goal of removing updating entirely for most users
The intended purpose of automatic updates is to remove updating from the list of items the user has to care about and remember. By exposing the updated date in list view, Firefox insinuates both that the updated date is very important that this is a process the user should manage.
Actually, the actual reason sorting and the last updated date were initially proposed in the add-ons manager design was to give users the ability to sort their add-ons by performance, not updated date. Sorting by performance would allow users to find out how their add-ons effect Firefox on metrics such as startup time and memory. However, the ability to rank an add-on’s performance is going to be a part of FIrefox after the 4.0 release, making the remaining sorting categories (alphabetic and updated date) much less useful.
By the way, Firefox 4 beta 10 is out, so please try it out and tell us what you think!
December 5, 2010
One of the best reasons to use Firefox is with thousands of add-ons available to customize it, you can turn it into exactly the browser you want. To make it easier for you to find and use add-ons, members of the Firefox team and community have been working to redesign the add-ons manager for Firefox 4.0. The new add-ons manager will be easier to use, sleeker, and faster than ever before.
Here are a few highlights of the new design:
Add-ons update automatically
No more warnings when your add-ons are out of date; Firefox will now update them automatically. This should happen without you even noticing, keeping add-ons safe and fast while eliminating the hassle of updating.
Make changes to add-ons without restarting
Restarting your browser is a pain. Developers can now build their add-ons for Firefox 4 such that no restart is required; add-ons built using the Add-on SDK get this for free. Restartless add-ons can be installed, modified, and removed without a single restart needed! More and more restartless add-ons are being created and made available on addons.mozilla.org every day.
Add-on manager in a tab, not a window
Instead of managing your add-ons in a small, separate window, the add-ons manager now loads in a tab. This means it won’t be so small and easily lost among other windows, and you can interact with it identically to other tabs, including resizing and moving.
New Get Add-ons pane
Justin Scott has been leading a project to create a new section in the add-ons manager we call the Get Add-ons pane. In the old add-ons manager, we displayed five featured add-ons that could be installed. This was done to show you some examples of add-ons – much like buying a picture frame with a stock photo already inside. Justin’s done a thorough revamp of Get Add-ons, building a page which introduces you to the concept of add-ons, highlights particular add-ons that are editorially selected, and helps you explore and discover other add-ons that match your interests. Justin’s currently working on a new feature for this pane which makes personal recommendations of add-ons you might enjoy based on which add-ons you already have installed.
Quickly find any add-on
If you want to make a change to an add-on but don’t know which category it’s in, you can now simply search for it in the new global search box. The add-ons manager can quickly locate an installed add-on or find you some matching add-ons that are available to install.
If you’re using Firefox’s nightly builds, you can already see many of the above changes in action. Blair McBride has recently put a lot of work into the new theme change, so now we’re working on the final few bugs and polish. If you’ve already used the new add-on manager, please share your thoughts by commenting or leaving a message on Firefox 4.0’s feedback page!
August 29, 2010
I’ve posted quite a few mockups on this blog of Firefox’s redesigned add-ons manager’s features and interactions. What I haven’t shown are screenshots of how the manager will actually look in Firefox 4.0. The following are designs, based on Stephen Horlander’s work on the new Firefox theme, of the add-ons manager in OSX and Windows 7. The icons are still placeholders, but the rest of the design is pretty near finalized.
In the image below, you’ll see three windows for each operating system. The first row shows list view, where a short summary of each add-on and its description are shown. Buttons allow the user to disable an add-on or launch its preferences. Clicking “More” takes the user to detail view.
The second row shows detail view, where the user sees more information about a single add-on. The full description is displayed as well as a contribution box if the add-on’s author chooses.
Finally, appearance view shows installed themes and backgrounds (previously personas). Since these add-ons are primarily visual, the interface gives a large preview of each item and does not display a description.
Dave Townsend and Blair McBride have been working hard on implementing these visual changes as well as the slew of under-the-hood improvements that are making the add-ons manager faster and more stable. To see how it’s coming along, try running a nightly build in your operating system of choice. Hope you like it!
It’s World Cup month! Let’s start it out right. With another post about removing the status bar.
I know I implied in my last post that you’d be free of this topic forever, but something was bothering me. A piece of the puzzle was missing. I talked it over with my skilled user experience cohorts last week. Whiteboards were involved. I think the kinks were worked out.
The problem with putting add-on icons in the bookmark bar by default is that Firefox’s interface could become easily overcrowded if add-ons add more than just a 16 by 16 pixel icon. If an add-on creates a long horizontal widget, for instance, the whole bookmark bar could taken up after its installation.
Also, many add-ons have come to rely on bottom-anchored functionality – partially because of the location of the status bar. Firebug, for instance, uses a status bar button because its interface is anchored to the status bar.
My first post on this topic noted that users should be able to easily move their add-on icons and widgets wherever they want just by dragging it – to the bookmarks bar, the UI panel, the title bar, wherever. This is still going to be a huge benefit to users who wish to configure their browser. Where add-on icons should be encouraged to install by default is the open question. My proposal is this:
When a user has no add-on installed, there is no status bar.
As soon as the user installs one add-on that wants to use the status bar, a small bar appears in the bottom right of the Firefox window. It’s only long enough to accommodate the add-on icons and widgets the user currently has installed.
If the user hovers over the status bar, a small arrow appears on its left. If the user clicks this arrow, the status bar shrinks into a small button in the bottom of the window. This button gives a faint glow as the status bar animates into it. If clicked, this button will bring the status bar into visibility again. The button is only visible if the user mouses near it, minimizing visual clutter.
The benefit of this design is that only the smallest possible status bar is shown, and if the user prefers it can be entirely dismissed. The panels for rich interaction would still be added to the API, as well as a way for add-ons to identify themselves as acting on the current page content (perhaps via a subtle “glow” effect).
I think this design addresses the multiple goals for add-ons in the ui: minimal disruption to current add-on functionality, minimal visual clutter, and trivial configuration if the user wishes to modify the default behavior.
Recently, I wrote about Firefox’s status bar and how we’d love to remove it, relocate its functionality, and dedicate that area back to page content. Though many add-ons currently place icons in the status bar, these icons could be displayed elsewhere in Firefox’s user interface. By treating add-ons as UI elements that users can move within Firefox’s UI, the bottom of the browser window could be dedicated to content while the functionality of add-ons preserved.
I proposed that by default, add-on icons display in the bookmark bar of the Firefox window.
But, if the user wishes, he can move the add-on icons to other areas. Firefox can subtly change the appearance of an add-on icon to visually match where it is placed.
Several commenters on my previous post noted that they preferred add-ons in the bottom right of the window, both because of particular add-on functionality and familiarity. Indeed, it should certainly be an option to reenable the status bar, and one way to do this should be to drag an add-on icon to the bottom of the window in customize mode.
The cases that would be especially suited to enabling the status bar are add-ons that require long, horizontal space in the main UI. Especially newer add-ons built by Jetpacks are increasingly using this horizontal space in creative ways. If users want this kind of functionality, giving up a significant portion of the bookmark bar should not be the only way to achieve it.
While this horizontal layout that some add-on developers choose is useful for many kinds of information display, it takes up a lot of space in Firefox’s interface and may not always be necessary. Providing API support for rich content panels that launch off of add-on icons could allow add-ons to provide rich interactive content, accessible with a click, that takes up little UI. This is a win for developers because it would enable them to easily add rich content to their add-ons. It’s also a win for users, because accessing rich add-on content won’t always mean sacrificing interface space. Some possible uses for such panels include: