Dropbox Paper

Research methods differences

This table presents a generalised summary of most typical variations of each method from HCI perspective. All methods can be / are adapted based on these characteristics

Method	Behavioural or attitudinal	Researcher & participant roles	Qual or quant	Lab	Formative or summative	Sample size	Range (suggested) heuristics	References
Diary study	Either	Self-report	Either	Contextual	Formative	M to L	5-1200 (30) HCI: 10-20 Pros Natural environment Responses more ‘in the moment’ Sampling over longer duration More time for detailed, well-considered responses Cons Reliant on accuracy/detail of self-report More opportunity for inconsistent responses Recruitment can be difficult Can have high drop-off rates When to use Diary Studies Following up to a prior piece of research such as interviews to gather more detailed and contextual information.	Hektner, Schmidt, & Csikszentmihalyi (2007)
Interviews	Attitudinal	Self-report	Qualitative	Either	Formative	S to M	12-20 (20), 30+ in quant HCI: 12 Pros Rich data source Can be organic — interviewer can adapt with follow-up questions Good for determining attitude & emotion Cons Analysis can take a long time, especially for qualitative data Can be leading if handled incorrectly What users say and do may not be the same thing… When to use 1:1 Interviews Verifying user requirements or product goals, and to gather detailed information on a specific topic.	Guest, Bunce, & Johnson (2006) Green & Thorogood (2009)
Card sort	Behavioural	Observation	Quantitative	Lab	Formative	Small	15-30 (15) Pros Simple, quick & inexpensive Don’t need huge numbers, and 10–12 typically gives good results Well established Provides insights into how users organise information Informs information architecture / layout Cons Analysis can be time-consuming Results can lack consistency Card sort is not task-driven, can lack context May not be deep enough When to use Card Sorting and Tree Testing Developing an information architecture, commonly used in web design for menu labelling and hierarchy.	Tullis & Wood (2004)
Field study	Either	Observation	Either	Contextual	Formative	Medium	Grounded theory 20-30, Ethnography 30-50 Pros More realistic than controlled study Can reveal unanticipated, context-specific use cases and issues	Creswell (1998) Morse (1994); Bernard (2000)