User Interface Design section.
Touch interfaces are increasing in popularity and ubiquity since the introduction of the iPhone and other mobile devices. Touch screens deliver a larger interactive surface and direct manipulation of the user interface to users. The direct manipulation leads to an intuitive interaction experience, but the touch has a particular Achille's Heel that the mouse doesn't suffer from - the user's finger is blocking their view of the screen.
There are several methods for addressing this difficulty, some of which are employed currently on touch-enabled phones. The iPhone, for example, will popup hints when you hold your finger over a letter on the keyboard or on a link on a webpage. However, this interaction is used sparingly. In the presentation of the new Palm WebOS the application menu bar "Wave" is drawn just above the user's finger so the user can see what they are selecting.
Context menus are a particular case in which the touch interface obscures the data being selected. Context menus also suffer from a typical problem in the transition from large, high-resolution desktop displays to a mobile device - they are too large and rely on vertical scrolling. Hierarchical context menus particularly aggravate this situation by requiring more vertical scrolling.
A better context menu could be based on a radial popup menu. The available options could popup in a circle around the user's finger. This allows all of the options to be visible at once, and the user can move their finger horizontally and vertically to make a selection. The interface can register a selection based on the direction of the user's finger, whether or not the finger is actually hovering over a menu item. This allows the user to move their finger outside of the radius of the context menu and make a selection without obscuring the text describing the function.
Hierarchical context menus can be implemented similarly with successive levels of detail opening out of the radial menu when the user moves their finger over that section of the menu. The speed of opening the hierarchy must be closely controlled and adjusted based on user tests to keep users from making unintentional selections.
Canceling the selection can be facilitated by the user moving their finger back to the center position that originally initiated the context menu.
When you're building your interface, you cannot predict how users will behave. You put common links on the side, support breadcrumb navigation, and build a flexible menu system. But the users still have to jump through hoops to move from task to task. Perhaps they regularly jump to the home page or main menu and then trek back down through the system to find the tool they need next.
Users are forming Desire Lines in your UI. They are telling you what order they typically perform tasks in, but you cannot see them. Unfortunately, these lines are not as obvious as the dead grass and dirt that characterize them on university campuses. As a UI designer you must gather the data and observe the trends. Then you can pave the desire lines with concrete and help your users get where they really want to go.
http://en.wikipedia.org/wiki/Desire_lines
A desire line is a path developed by erosion caused by animal or human footfall. The path usually represents the shortest or most easily navigated route between an origin and destination. The width and amount of erosion of the line represents the amount of demand. Desire lines were used in early transportation planning, prior to the advent of computerized models.
They are manifested on the surface of the earth in certain cases, e.g., as dirt pathways created by people walking through a field, when the original movement by individuals helps clear a path, thereby encouraging more travel. Explorers may tread a path through foliage or grass, leaving a trail "of least resistance" for followers.
The lines may be seen along an unpaved road shoulder or some other unpaved natural surface. The paths take on an organically grown appearance by being unbiased toward existing constructed routes. These are almost always the most direct and the shortest routes between two points, and may later be surfaced.
Desire lines can usually be found as shortcuts in places where constructed pathways take a circuitous route.
Many streets in old cities began as desire lines which evolved over the decades or centuries into the modern streets of today.
If users could leave desire lines in our applications, then these would instruct later users, as well as designers, in how users find their paths across our applications. Don Norman calls desire lines social signifiers. They are signifiers that are created by the behavior of social groups. Our pristine application interfaces aren't trodden by the feet of countless travelers, but would they be better interfaces if they were?
Perhaps interface designers can look for ways to instrument their user interfaces to report this data and improve constantly. Certainly this is an accepted technique for observing users' paths through a website and it should be adopted and supported by user interface frameworks across all platforms.
This commercial plays at the end of some of the TED Talks. What I noticed was the way the video performs transitions between shots. I like the clean colored text and the outline which then becomes a portal to a different scene. This is a nice way to do cuts and something that could be used in a Flash/Flex website or application.
I have always been interested in the idea of affordances in user interfaces since I first heard the term. So naturally I was very interested to read an article by Donald Norman - the original researcher to bring the term from psychology to design - about the importance of signifiers instead of affordances.
The term affordances was coined by psychologist J. J. Gibson to explain the possible actions that a specific agent can take on a specific object. Norman brought the idea of affordances, which are realities that exist regardless of the agent's knowledge of them, to design in the guise of "perceivable affordances." For Norman's design work, an unperceived affordance is essentially nothing.
In his article, Norman espouses the idea that designers should forget about affordances and focus on the signs that affordances exist; these he calls signifiers. His reasoning is straightforward - it's not the fact that this user interface widget can be dragged that is important to design, but the signifier that indicates to the user that the item can be dragged.
I didn't pick this example at random. I believe drag and drop is one of the most problematic of design elements. There is no standard signifier for the capability to be dragged. There are signifiers for the affordance of being clicked such as raised shading and the hand cursor, but there is no universal indication of "draggability." The scrollbar is one of the few signifiers that is universally recognized as draggable.
Take, for instance, the ability to reorder the icons on the iPhone. This capability is induced by holding your finger on an icon for about 2-3 seconds. Then all the icons start dancing and jiggling around on the screen. Apple thus created a relatively novel signifier for the affordance of dragging. This is nice, but there is no signifier that indicates holding your finger down on an icon will induce this behavior.
Dragging on a computer with a mouse might be easier to provide a signifier for: simply change the cursor into a shape that could indicate draggability. However, there is not a generally agreed upon shape for that behavior (the move cursor with four arrows might be the best), and many widgets that can be dragged can also be clicked for a different response. This means that the widget will already display the hand cursor. You could use a custom cursor which displays both icons, but this starts to be confusing and unclear.
The iPhone supports several other interactions which have no signifiers. For example, in the Mail application you can delete an e-mail from the summary view by swiping your finger left or right across the title. I didn't find this for weeks. Although the affordance existed, I didn't perceive it because it had no signifier. I think Norman is pointing out something significant here.
Don Norman is a professor at Northwestern University and cofounder of the Nielsen Norman Group, with Jakob Nielson whose Alert Box website is a great resource for web usability. I want to summarize Norman's recent opinion piece Signifiers, not Affordances in ACM interactions publication and offer some commentary (he has posted the entire Signifiers, not Affordances essay on his own website).
After my post about Don Norman's Signifiers, not Affordances article, I had an interesting discussion about the relationship between signifiers, affordances, and features. The importance of signifiers was again reiterated by several people as the key to driving user satisfaction and user capability within an application. Features can only be accessed if the user has an understanding that the feature exists, and signifiers are the agents you as a designer can use to communicate with your users.
Don Norman also discusses the importance of social signifiers in communities, whether physical or virtual. Social signifiers may be intentional or unintentional, but either way they communicate to users. One social signifier that I realized I use is the popularity of a given application. If I hear from many people that they love application XYZ I am willing to give that application more time and attention than application PDQ. In particular, if I have difficulty with application XYZ I am willing to dig further, try harder, and search for solutions. However, if I have heard negative reviews (or no reviews) of application PDQ I will more readily abandon it if I have a bad experience with it.
Early user abandonment is of key importance to application designers. Kathy Sierra explains that users initially feel incompetent with a new application and only enjoy it once they pass the "suck threshold." She explains that creating passionate users is about getting them past this threshold and into the "I kick ass" zone.
To define our terms we might start with a feature. A feature is some capability that an application supports. An e-mail client might have the capability to read, write, and delete e-mail, for example: this would be considered 3 features. If the feature doesn't exist, no amount of work on the part of the user will cause the application to locate and perform the function.
An affordance might be seen as similar to a feature, but my understanding from the design and HCI community is that an affordance refers to a specific method of interaction with a particular user interface widget. For example, the Delete button has the affordance of being clickable. If the button is labeled "Delete" but is not clickable, then again the feature cannot be performed by the user.
The appearance of the button along with the text "Delete" are the signifiers to the user that this is a clickable widget which should perform the action of deleting an e-mail. So, the affordance of the button combined with its semantic signifier ("Delete") and the signifier of being clickable, result in the user being able to invoke the feature.
It is worth noting, although fairly obvious, that a given feature may be invoked through many different methods which each have their own signifiers.
As user interfaces progress to processing touch and video as input the design, HCI, and development communities must work together to develop new signifiers to communicate to users the potential behaviors of our interfaces. Most of these signifiers will be visual cues of one sort or another, although haptic feedback devices might become available and useful in the future as well. History suggests that aural feedback can be an annoyance to users, but perhaps if it is subtle enough it could be used to provide texture to the world of the user interface.
The most readily available platform for experimenting with touch devices remains the iPhone. There is a wide user base for this device at present and a good development environment as well.
Resources on usability, user interface, interaction, design, design patterns, data visualization and more:
Inventor of sparklines.
http://www.edwardtufte.com/tufte/
Books/Essays:
Information Visualization - Design for Interaction (2nd Edition)
http://www.amazon.com/exec/obidos/tg/detail/-/0132065509/
http://www.infovis-wiki.net/index.php?title=Spence%2C_R.:_Information_Visualization_-_Design_for_Interaction_(2nd_Edition)%2C_Pearson_Education%2C_2006
Master's thesis "The Form of Facts and Figures" by Christian Behrens in the Interface Design program at Potsdam University of Applied Sciences.
55 design patterns that describe the function aspects of graphic components for the display, behavior and user interaction of complex infographics.
http://niceone.org/infodesign/
http://safari.oreilly.com/0596100167
Uses Processing
http://safari.oreilly.com/9780596514556
Work in progress...
Custom cursors offer tactile feedback to users of an application about what operations are possible on the underlying component as well as the state of the application. The busy cursor encourages the user to wait for an operation to complete while the hand cursor informs them that this user interface component will respond to their mouse clicks. However, there are unique problems with the custom cursor implementation in the Flash player.
The Flash player utilizes a rendering pipeline that is affectionately referred to as the elastic racetrack. The racetrack represents an infinite loop of processing in which one lap around the racetrack represents a single frame of animation. The Flash player is deliberately single threaded and therefore if an operation takes up a significant portion of time the racetrack can lengthen, the framerate will decrease, and users will notice the application becoming unresponsive.
Most rendering pipelines have an issue such as this, however the Flash player's single threaded nature and the general difficulty of deferring work from one frame to the next causes this to be particularly noticeable. A typical desktop application may also enter a long period of single threaded processing, however this behavior does not negatively impact the mouse cursor. The reason is that most desktop applications modify the system cursor which is not rendered by the application's render loop, but is handled by the operating system in a more robust fashion. Flash player, however, renders its custom cursors using its own render loop - so a long running operation will cause the mouse cursor to stick, jump, and disappear.
This is obviously a very jarring and frustrating experience to most users. Due to the typical behavior of desktop applications, most users will interpret the jumping or nonexistent cursor as an application crash rather than simply a long running operation. This is particularly complicated in the Flash player as the busy cursor is also rendered by the Flash render loop and also suffers from these limitations.
There are two ways to address this problem. The first is to pay careful attention to the processing time of any code in your application. This is a common suggestion, but one that is not trivial for most developers. This is complicated further by the issue that even if an operation is determined to be taking too long there are limited options available to the developer. The developer can first try to optimize the operation as much as possible so that it does not take so much time. This is the most direct approach and often leads to good results. The next approach is to try to spread the operation over multiple render loops. This is often nontrivial and not obvious. This can be further aggravated by the delayed and queued rendering behavior of most Flex components. If a Flex component is not provided with its data during its first frame of existence it may be rendered during that frame in a partially complete manner, which can often cause significant visual jarring in the application when it resolves to its correct state.
This visual jarring can be reduced by causing the component to wait to render itself until all of its data is available, but this is not a common practice and it is not immediately obvious how to achieve this using the Flex framework.
The performance lag can also come from other sources such as an extended service call (which should not itself slow down the framerate of the Flash player), parsing large amounts of XML or other data, or rendering complicated graphics. The Flash player parses XML data when it is returned from a service call into XML objects and with significant data sizes this can cause a lurch in the Flash player. The Flash player also typically renders the entire visual representation during one frame and complicated visuals - which may perform adequately after their initial rendering - can cause a pause.
The second way to address this problem is by altering the use of custom cursors in the Flash player. Since the movement of the cursor is governed by the framerate of the Flash player this can be beneficial even when there are no significant performance barriers. Users will be accustomed to their cursor moving at near the refresh rate of their monitors - usually above 60 frames per second, or 60 Hz. Flash movies and Flex applications typically run at framerates of 24 Hz, although developers can override this setting. If you are developing an application where smooth animation and cursors is a necessity, it would be wise to raise the framerate somewhat, however, problems have been reported with server calls when the framerate is set too high.
Thankfully, there is a compromise possible between abandoning custom cursors altogether and using them in the typical manner. This compromise is to decorate the system cursor rather than replacing it completely. Decorating the system cursor involves attaching a Shape or Sprite to the current mouse position and avoiding operations which set the Flash cursor directly. This decoration may at times lag behind the system cursor, but it will at least allow the user to move their mouse with confidence across your application.
Decorating the system cursor can also be a significant help when the user is performing precise manipulations such as in image editing or layout. Since the system cursor will still render at full speed the user can be confident in their motions, even though the Flash player may lag in updating the user interface.
Observing users is, as always, one of the best ways to identify usability problems with your application, and should be relied upon to focus your efforts on important problems. However, attention to detail will always be appreciated by users even when they cannot specifically identify the problem they are experiencing. Unhappy users are more likely to simply quietly disappear than to register their complaints, and you may lose them quickly if they feel your application is unresponsive. Most users will equate an unresponsive application with a broken application. If your application is involved in selling anything to users this can lead them to question whether your application is safe enough to enter their financial information into and result in abandonment.