Methods of gaining user research or a user’s perspective for subsequent design purposes :
n.b this was generated in 2002 and needs updating, but may be useful.
The following are techniques assimilated during a Teaching Company Scheme (now known as a Knowledge Transfer Partnership) performed with PDD Ltd a London based Product Design consultancy. A range of techniques were gathered in order to perform user research applicable to design within an SME with commercial constraints. The form and nature of the data differs for each method and influences the way it is interpreted and subsequently used to create artefacts or systems. In practice the boundaries between each technique may be loosely defined. Methods for research have been categorised by Sanders into what people ‘Say’, ‘Do’ and ‘Make’. (Sanders 1999) She argued that it is insufficient to listen to what people say about their lives and that watching what they do will reveal a fuller picture and allow triangulation of information. What people say and what they do are often different.
Say – defines (usually more traditional) tools based on verbally probing people about what they think and how they feel. Typical methods are focus groups, interviews, and questionnaires. They have usually been used in some form by market or cultural consumer researchers.
Do – covers observational techniques. This method of understanding people is more strongly based on anthropological principles and is often referred to as ethnography. Studying what people do allows one to see behaviours that the person may not be aware of or be able to articulate.
Make – describes projective and participatory creative techniques. The new tools according to Sanders are focused on what people create from tools used in expressing their thoughts, feelings and dreams. (Sanders 1999) These methods have been described as the dream tools or synthesis tools.
These categories provide a useful starting point as they are simple to understand, however it is also useful to note within the ‘say ‘ category that some techniques are used by the participant alone and some include interaction with the researcher.
The following covers some general principles for performing research that usually apply for an application to understanding users for design purposes. (From Fetterman)
Gaining an outside perspective: User research methods are usually employed to gain insight into the user’s world for a design situation. The researcher is therefore aiming to put aside any pre-conceived ideas and to attempt to leave their own opinions and ideas behind as far as is possible. This means in turn that they must avoid prompting or leading participants when interacting with them during studies, especially during early stages. However, the researcher will have to guide the study to keep it relevant, they need a pre-defined structure for their activities. For example, during interviews, they need guidelines for enquiry but must learn to avoid asking closed questions in interviews and allow the participant to elucidate on points freely. For self -documented approaches, the material should not be created in such a way that it suggests answers. At the later design stages it will also be important to avoid adding personal viewpoints to the interpretation of the results. In practice it is often impossible to observe or interact with a situation without influencing the outcome but this is the aim.
Triangulation; It is important to know when actions, opinions or ideas on the part of participants are significant. If a point has arisen on more than one occasion during investigations, this usually indicates that it is worthy of further exploration. If certain data is backed up by different people or observed in different ways it can be described as ‘triangulated’. This then becomes evidence and provides a more concrete understanding of the situation. Using a variety of techniques (like Sanders Say Do Make proposal defined below) provides different types of insight. If the material gathered by different means supports the other and a qualitative appreciation of the situation has been gained, then triangulation has occurred. When no new information is being obtained, it is time to move on and call the research complete.
Natural Settings: Attempting to gain insight into a design situation usually requires the researcher to enter the natural context for the user’s activities. This might be their home, their workplace their means of travel and so on. Many of the clues to be obtained will be visual or auditory rather than just verbal. This is largely why observations in a ‘fly on the wall’ fashion have become popular for gaining insight.
Iterative approach: When information has been used to provide an insight for designers and suggests particular attributes, it is important to check back the interpretation with the user. Designing is an iterative process, where sometimes it is necessary to return to earlier stages with new assummptions. User research should provide greater confidence in solutions but does not replace testing and checking with users.
Holistic: The purpose of the research is to gain a holistic and rounded understanding of a situation, including feelings and emotional reactions to the situation rather than simply factual or functional data. Anthropology, emphasised the gestalt i.e. that societies are more than a sum of their parts.
Similarly information about people should be understood, not only within the wider context that they exist within, but also as part of a broader basis for experience.
Research Techniques Creating an Issues map:
A typical issues map for a PDD project, entitled: What delights Youth?
It helps, before any investigation to attempt to clarify, however broadly, some of the main issues to be considered. AT this stage many heads are better than one and, where possible a brainstorming of relevant subjects can create a starting point for thinking. In practice this would only be used loosely as a guide and would probably undergo some iteration as the investigation continued.
Participant Documentation Participant-documentation describes a variety of methods of extracting information from users at times convenient for them. Materials are provided for them to record particular events, images or thoughts in situ as they happen. Particular examples are:
Diaries facilitate recording experiences over a period of time. In most cases the diaries will be issued to people to fill in outside the presence of an investigator (Jordan 2000). The ‘diary’ can include instructions, time frames for recording, daily prompts and extra areas for ideas or impromptu comments. The length, organisation and design of the diary is very important., since it should not be boring or difficult to fill out. Diaries are often sent to respondents before they come to participate in interactive events. Self-documentation through diaries makes people aware of the patterns they use in everyday life. This new insight brings their experiences to the forefront of their minds prior to the workshop or interview, allowing a deeper exchange of information.
Photographic Images. ‘Inanimate objects including photographs can have a role in bringing the stories into people’s minds, and in that way become personally significant to people (Czikszentmihalyi and Rochberg-Halton, 1981; Londos, 1997, Koskijoki, 1997). Diaries can be illustrated with photographic images or any other graphical means adding an extra dimension and greater understanding. Images can also be used to express experiences without a time-based focus. A picture paints a thousand words and is a far more interesting medium for designers, who usually have a visual approach to their work. For instance a picture of a workspace can give a deeper insight into the ways people organise their lives and their stuff (Sanders 2001). They show you how they view their lives and often reveal what is most important to them. Often a participant is offered a disposable camera to record events or situations and make their activities clearer to others. Typical instructions might be to use the camera to ‘document an experience of interest such as: places I like to go; who I communicate with. Show us the picture of what you will wear today or what you will carry. Or take a picture of something ugly, something beautiful, something interesting, something boring.
Illustrations: A variety of illustrative methods can be used to add further meaning to time based recordings or to research which is categorised by other means. Drawings of facial expressions, have been used so that particular encounters can be annotated with an image representing their reaction.
Metaphorical images or photographs which represent feelings or other characteristics are also useful.
Video and audio recordings are also used by the participants to gather their thoughts before researcher intervention. These can be in-context, perhaps with instructions: “introduce us to your bedroom and the objects in it” or there may be a walk in booth, with a subject for discussion. Audio recordings are clearly similar. Wearable recording technologies are another possibility, however, for ethical reasons it would usually be important for the user to have full control of the device and not have continuous recording. .
Graphical representations. Occasionally ‘graphs’ of positive versus negative feelings have been drawn over time to illustrate the ups and downs of an experience. The image shows data gathered at PDD Ltd. of experiences over time with a washing machine.
Mixed media approaches In practice any combination of media including text, photos, illustrations and moving imagery can be utilised for the purposes of communication. In combination frequently the results are more meaningful, and the participant may be left to decide how best to get their ideas across. The figure shows a representation of feelings about health related experiences, using a range of materials. The message is the most important factor but how it is communicated can affect its interpretation , thus the material and suggested method should be carefully guided.
Questionnaires or other documentation: This covers written material offered to the respondent to explore questions before interviews or other interactive procedures take place. This can be useful as it doesn’t take up meeting time, and is suitable for items that don’t require much discussion. The technique can be boring for the participant however, and mistakes can be made unless the instructions are very clear, so an imaginative approach to the presentation and nature of the material is needed. The results also require time to review and assimilate, the researcher does not get the same instant feel for the issues as the might do in interviews or ethnographic studies. It is also difficult to avoid asking leading questions sometimes, or for them to be wrongly interpreted.
Cultural Probes Gaver and Dunne introduce a method that tries to elicit what people think, feel and do through a variety of tools called ‘Cultural Probes’ (1999).
This is a descriptive title for an inspiring collection of items intended for use by participants, allowing the researcher to have insight into the intended culture. Just as a scientific ‘probe’ would come back with data, the devices are intended to be self explanatory vessels for information operated by the user. They invite users to a more playful and creative way of giving both information and inspiration to designer-researchers. The probes include packages composed of maps, postcards, a disposable camera, and a diary that invite users to actively participate in the design process. Gaver states ‘The maps, cards, and photographs seemed to capture specific facets of the cultures. The probes provoked them to think about the roles they play and the pleasures they experience’.Frequently they require a mix of the types of media defined above in creative formats making their use more enjoyable and straightforward. This is important since self documentation of this kind can be boring and confusing.
Interviews: Add stuff from seminar at Surrey. Interviews consist of a conversational exchange between people (typically researcher and respondent). Through interviewing, we learn what people want us to know about how they think and feel.
For social research different kinds of interviews serve quite different purposes. Three important versions are the ethnographic interview, the depth interview, and the long interview. The ethnographic interview is an intensive method used involving immersion of the researcher in the culture under study. The depth interview is similar to techniques used by clinical psychologists that yield a comprehensive psychological profile of an individual. The long interview is a more economical form of ethnographic interview developed by researchers working within their own cultures. There are three broad categories of interview – unstructured, semi-structured and structured. Because of the prompting, semi-structured interviewing techniques can ensure that each respondent covers a central set of issues. (Jordan 2000)
Design group PDD Ltd. established that the semi-structured interview using a visually spaced template provided a powerful basis for gathering information within the time scales of a commercial operation.
“the conversation is permitted to wander onto associated topics whilst keeping a focus on the main issues under investigation.” (3rd LMC meeting of Teaching Company Scheme)
The following illustrates a rough interview structure used at PDD Ltd. Interview structure; Use an experience model as a template for mapping the issues; What motivates the person to undertake the particular activity?; What exposure have people had to the task/activity/product before?; What do they remember?; What language do they use, jargon, stories?; What is their mental model?; What have they heard about the activity/product?; What friends say; What experts say; What competitors say; What the ads say
Interviews are very versatile in that they can be used throughout the design process. There is ess chance of misinterpretation of the questions. However, participants are valuable mainly for learning what people want you to know and biases are difficult to rule out. (Robson 1993). The method is also time consuming.
Semi- structured interviews were found to be an ideal method for creating urban designs (R.Erol, R.Cooper, et al.2000) Informed designing permitted the inclusion of crime-reduction measures during early design stages which were found to be effective. For the Adshel information unit, providing interactive information about towns and cities was designed to meet the needs of the end user by including them in the process.
Contextual Inquiry: A recent slant on interviewing has included ‘Contextual enquiry with artefacts’. This is where participants will introduce the interviewer to items in their own home on request to investigate issues such as: Lifestyle (hobbies friends, interests, trends.) and Values (things that they appreciate like family, religion, politics)
Some examples are: the kind of devices or products the user owns and why; their favourite objects and interiors at home.
Stories and Narrative. ‘According to the new approach in social sciences people form stories about themselves in order to know themselves and in order to tell others who they are’ (Mc Adams 1997). We can use the stories people tell to describe situations of interest or to understand more about them, they can be recorded by the user before the researcher arrives, be part of an interview or participatory workshop. Stories represent the way people condense and remember experiences, how they choose important themes to communicate in certain situations, and how experiences are relayed from person to person. Stories for design research rely on narrative and framing and can be told in any expressive medium, including prose, photographs, illustrations, diagrams and the spoken word. How we tell stories is mediated by our culture. When you ask for someone’s story, you get it all, bundled together in a tight package. Inside that package are the reasons why he or she does things, and these reasons hold the keys for designing. This is possible because stories and storytelling are largely about two things: culture and structure. (Brooks 2001) In scenario writing and storytelling the storyteller is present as the subject and thus it is an appropriate expression of experience. In its course, narrative describes human beings and systems, as well as their relationship and the background, that relationship creates. (Forlizzi, axis nov 2000)
Focus groups This is a group of people gathered together to discuss a particular issue or design. A focus group usually consists of a discussion leader and a number of participants. The agenda is usually loosely structured to allow participants to take the lead in determining the direction in which they wish the discussion to go. This should ensure that the points raised will be those that are of most concern to the participants. (Jordan 2000). Issues may arise that the investigator may not have anticipated and they can be more efficient than interviews as they avoid overlap and repetition (Morgan 1997) However, there is a danger that one or two of the members of the group may prove particularly dominant. This can be overcome by careful management of group dynamics by the leader.
The advantage of self documenting initially is that it can help participants uncover issues that they may not have been aware of before they started, allowing them to ‘know themselves’ giving greater material to discuss later with researchers. It also gives a longer period to record information. It requires less of the researchers time and therefore is a cost-effective way to gain information and commercially viable. The problem with self documenting that it requires increased demand on the participants in terms of time, effort and understanding and there is no guarantee they will achieve the requirements properly. Taking a ‘cultural probe’ or a creative, appropriately designed approach to the technique is therefore very important in raising interest and providing understanding of the tasks in hand. Straighforward interviews without such self documenting are less revealing and people come to the issues cold without proper review.
Ethnography and Observation The researcher (and subsequently the designer) is interested in understanding candid human behaviour within a particular identified context. Observing in a ‘fly on the wall’ fashion allows the researcher to see behaviours that the person may not be able to express in other ways. Approaches vary depending on the situation, observations can occur in the users natural environment, at work or in a laboratory setting. Laboratory situations have usually been used to perform relatively scientific, structured and perhaps quantitative ‘test’ style observations while the users environment is preferred for revealing an added dimension of qualitative information. More recently, however, experiments with qualitative results have also been established in specially organised environment. The arrangement is probably best not described as a laboratory, perhaps ‘studio’ better indicates the more creative and open approach to learning about the participant. GVO state: “At least 80% of all human communication is non-verbal. Getting at that other 80%, where much of what people truly think and feel is buried, can be tricky. The adaptation of various qualitative research techniques to design, such as ethnography and contextual inquiry, are developing to support this area”. (GVO storytelling)
Traditional definitions are:
Systematic observation – Behaviour, not intentions – quantitative data, its focus on overt behaviour describes what happens, but not why it happens. It does not deal with the intentions that motivated the behaviour. No information on the experience, and on the cultural context.
Participant observation – Blending in, no one knowing they are being researched, this method discloses things through the researchers experience, ethnographically based.
Cognitive Analysis – Measures such abilities as memory, attention, workload, decision making etc. Actual observed behaviour, as well as perceived.
Ethnographic methods rely on watching people in their own environment. A key feature is that the observer is present not only physically, but also enters into their social and symbolic world through learning their social conventions and habits, their use of language and non-verbal communication. (Robson 1993). The ethnographer uses all senses in this task and also participates in community and activity rituals. (TSO 1999). The philosophy behind including ethnography as a method for design is based on a desire to include people in design decisions and therefore to understand the most about their lives, problems, and circumstances. Ethnography broadly defined, is a methodology used to represent day to day living.
Video adds another potentially time consuming aspect to the ethnographic process, since it must be downloaded, stored and edited. However it is also the best way for third parties to appreciate the full experience of participants in their homes or work situations (besides actually being there) and records information for posterity that might have been overlooked at the time. For this reason modern day ethnographers for design are rarely seen without a video camera. Stephen Donaldson is the Consumer Insight director for Unilever Bestfoods UK, he describes the benefits of videoing people in their homes, to discuss key issues, for the companies marketing department. (Consumer Insight seminar 2002) He describes two main advantages of using video based ethnography: firstly the direct contact with consumers-making sure they are understanding the target market in real life situations, secondly the broadcastable and interesting nature of using video to communicate the information around the organisation. The objective is to experience the consumer directly rather than through reports or others opinions. Another benefit they found of this visual approach was being able to see the broader context, rather than just the answers to specific questions.
Melissa Frost is a partner at the Fourth Room, a group of senior creative strategists whose mission is to help companies to grow through connecting with customers. In her talk entitled Practical Application of Modern Anthropology (Consumer Insight Seminar 2002), some reasons for employing new methods are given. The unpredictability of people; the extra choice available, making more decisions necessary and the fact that we are bombarded with different marketing messages, meaning that it is more difficult to get through. The policy there is to offer a pieced together set of methods to suit the problem: known as “Bricolage”.
Remote data collection, new technologies
New technology permits increased opportunity to observe users in a wide variety of environments, through video, audio, network applications, tracking capabilities and through other computers based communications. This can be done in real time or asynchronously. Tracking technologies are becoming increasingly pervasive, and can be watching our digital based moves as we interact with the computer world (Blomberg) This raises ethical issues because it is not always within a users control and information could be being collected without consent. Usability experts have assessed web site use by analysing server log files This involves looking for patterns in log files, indicators or online behaviour. (Kantner 2001) The idea was that this quantitative information might provide patterns that can be subject to qualitative exploration. New programmes allow real time observation of activity on computers with live audio ; video and recording of interactive behaviour.
Look for ideas about seeing what other people see onscreen
Field observation uncovers information more directly than self documentation and reveals true pictures of every day life. Visual and auditory clues are often present to the trained eye which might otherwise be missed. For design these methods are extremely beneficial in interpreting the results of their work or indicating potential design directions. There some potential problems, including the time consuming nature of field studies, which have to be economised for design purposes. The observer may also be unaware of affecting the results of the study through their presence and there are also ethical issues to be considered.
Scenarios “scenario-building” can be described as the development of a series of alternative fictional portrayals – stories – involving specific characters, events, products and environments, which allow us to explore product ideas or issues in the context of a realistic future” (Suri and Marsh 2000, 152). The benefits are hailed as: scenario building encourages a holistic understanding, it facilitates interdisciplinary interaction and it enhances communication of the findings. (Rothstein 1999). Though important in some environments, highly detailed reports filled with statistical data are of questionable value in a hectic design business environment (Crossley 2000). Scenario building is more effective by translating data and information into a narrative, often accompanied by visual material. A narrative, of course, provides an audience with something they are accustomed to remembering – namely, stories about characters, activities and events Elizabeth Sanders and James Couch, authors of an article titled “To Understand the User,” describe how scenario-building is one of the most effective tools designers and researchers can employ to provide holistic design solutions. They note that a well-researched and well-crafted scenario is “. . . the form of design representation with the most potential to tie together users, events and designed objects” (Couch, Sanders, Welker 1997, 26). Alternative recording technology to make collaborative stories.
Deeply understanding the “life context” of users is seen as a necessary initial step in the development of meaningful and exciting scenarios. In this regard, field research is identified as the foundation upon which to build scenarios and design experience. In a sense, the proponents of scenario-building and experience design argue that visions of the future are best built upon knowledge of the past and present. (Rothstein 1999)
Participatory Design Participatory design techniques involve the user in a creative process and are used to increase their presence as a stakeholder in decisions made about a design. They may be present with a range of interdisciplinary specialists and will be perhaps asked to discuss proposed design features, or even may be involved in a more hands on fashion: sketching or using material kits provided to arrive at ideas. Designers are therefore directly exposed to the users view in a forum where features and aesthetics are being decided.
Ina research project called ‘Kidstory’ the aim was to develop technologies that support children’s collaborative story creation in school environments (Neale and Stanton 1999). They concluded that they needed more than a single input device for maximizing their creative collaborative story construction. They devleoped a platform for multiple input devices allowing two or more children to inteact with the computers creating stories together.
Research for a project called Today’s stories focussed on everyday activities They developed a wearable technology called Kid Cam for for educaitonal use . they believe it will facilitate and support their social and emotional development. (Panayi et al. 1999)
The disadvantage of such a technique is that it may be intimidating or demanding for participants. They may be unable to come up with realistic solutions in such a situation. The design event must be made as appropriate as possible and materials used which will encourage activities that are within the remit of the participants. Some of these are outlined below:
Collaging is a technique that allows people to articulate their feelings and thoughts about how a proposed design should be. It can also be used to explore existing experiences with interfaces and is useful also for self documentation described earlier. The user creates a collage using a range of imagery and tools to evoke particular emotions.
Sanders give a few rules of thumb for what to include in a collage set:~ a balance of positive and negative images and words,~ a balance of abstract and concrete images and words,~ both natural as well as man-made things, and~ people of all types: male and female, young and old, racial balance, etc.
Reduce the set to a manageable level, certainly no more than 100 words and 100 pictures, and preferably 150 of the two combined. In our example, a picture of a clogged highway and the words “traffic jam”, both address the same thing, so it may be better to discard one or the other. (sanders 2001)
Velcro modelling is a way of making the needs of the user tangible and three-dimensional at an early stage. It involves providing people with a kit of many generic components that can be stuck together to construct their ideal design. Designers use a series of mock-ups to express and evaluate their ideas, this is the same principle for participants. (Sonic rim website) These tangible ‘things’ would trigger the users imagination and create associated meaning. (Binder) Velcro-modeling is best used at a stage where people have ideas in mind and they quickly scan through hundreds of components to find those that best fit their ideas. (sanders 2001)
Role Play The world of theatre has provided an inspiration in design circles as a method for understanding users and gaining empathy for them. Ehn and Sjorgen (2000) created theatrical approaches to design and they stressed the productive role of stages and props to create a common language of engagement between designers and users. They state ‘ We are convinced that the world of participatory design can find inspiration from the world of drama’. In a design exercise, Ehn used drama as a way of empathising with the role of a refrigeration technician by creating four different actors as a guide. The purpose was to create the ideal smart tools and toolbox for the job. The designers pretended to be Allan the technician and used drama as a way of learning more about his values, likes and dislikes. Besides the use of simple card board props of possible products the team experimented with the use of dream tools as props in the design process. The props were a crystal globe where you could see whatever you wanted, a magic wand where you could do whatever you felt like, an a magic box where you could store whatever you needed.
Cognitive Mapping This is a method of graphically representing peoples thought processes, understanding of or emotional reactions to an experience. Designers have created toolkits to facilitate this process; they might for example contain a poster board and symbolic shapes, which can be used to map out connections, clusters or hierarchies of concepts. The shapes may be used literally or they may be used metaphorically. It is advisable to include very simple shapes such that many different interpretations are possible for each shape. (Sanders 2001)