Measurement and Evaluation

Learning how to learn from users with a high level look at design research including user interviews, contextual inquiry, and the Think-Aloud Protocol. Note that the lesson was changed after the rehearsal recording. See links.

Lesson Table of Contents

Video
Written materials
Links
Exercise
Reading
Next
Works cited

Video

Resources mentioned in the video can be found in links. Videos are hosted by Vimeo. You can load the video as an embed within this page or may view the video on Vimeo in a separate window / tab. If you enable on-site video, your preference will be remembered using a cookie.

Enable Video Embeds View on Vimeo

Resources mentioned in the video can be found in links. Videos are hosted by Vimeo. You may disable Video Embeds or view the video on Vimeo.

Links

Links which may be helpful to complete this lesson:

Written materials

In addition to the video, you may also download slides from this lesson.

Exercise

For this assignment please record yourself (instructions for Mac, Linux, Windows) doing a think-aloud walkthrough of the introductory sequence at https://app.pyafscgap.org/. Please follow the prompts at the top of the screen and do not click skip intro. After you record yourself, listen back to how it went. What is this activity good at soliciting? What might this activity be poor at uncovering? Write two sentences in reflection.

Note that the Zulip community is not available to this MOOC. Please consider sharing your exercise via social media such as Bluesky with the tag #OpenDataVizSciCourse.

Reading

I would love to share a talk from Jane Fulton Suri. Sorry the screen isn't visible but it's an excellent follow up to today and also one of the only recordings available of it. The hospital example involved a camera basically fixed to a patient's head which revealed that their the day is spent looking up at the ceiling with people looking down at them.

Next lecture

Ready to continue? Go to the next lesson.

Works cited

This is the works cited from the lecture. Note that additional sources may be used in exercises and other supporting documentation.

"The Deep Dive: One Company's Secret Weapon for Innovation," ABC News, 1999.
"The Art of the Insight: Learnings from 30 Years of Curiosity and Empathy," IDEO U, 2019. Available: https://www.ideou.com/blogs/inspiration/the-art-of-the-insight-learnings-from-30-years-of-curiosity-and-empathy
C. Nickerson, "Symbolic Interactionism Theory & Examples," Simply Pscyhology, 2025. Available: https://www.simplypsychology.org/symbolic-interaction-theory.html
J. Tong, "Diagrammatic thinking and audience reading of COVID-19 data visualisations: A UK case study," Convergence, 2024. doi: 10.1177/13548565241309886.
B. Hanington and B. Martin, "Universal Methods of Design," Rockport Publishers, 2012.
"User Interview Example Questions," Yale University. Available: https://usability.yale.edu/understanding-your-user/user-interviews/user-interview-example-questions
J. Gallegos, "50 Powerful User Interview Questions You Should Consider Asking," Respondent, 2022. Available: https://blog.respondent.io/best-user-interview-question
B. Shneiderman and C. Plaisant, "Strategies for Evaluating Information Visualization Tools: Multi-dimensional In-depth Long-term Case Studies," BELIV '06, 2006. doi: 10.1145/1168149.1168158
C. Lewis and J. Rieman, "Task-Centered User Interface Design: A Practical Introduction," HCI Bibliography / IBM Research, 1993. Available: https://www.hcibib.org/tcuid/
C. Lewis, "Clayton Lewis" Available: https://sites.google.com/view/clayton-lewis.
"What is Deign Thinking? An Overview," AJ&Smart, 2020. Available: https://www.youtube.com/watch?v=gHGN6hs2gZY.