Lean UX Research Tips from GitHub

The thesis: research doesn't have to be a big scary thing that blocks ships. Lightweight research, done right, runs alongside engineering and makes everything better.

• The case study. GitHub's Suggested Changes, which lets a reviewer propose an exact code edit inside a comment, shipped as the janky markdown option—the least favorite of four designs, because it needed no new tech. A two-hour InVision prototype tested with five people confirmed it would work anyway, and correctly predicted the top two feature requests that 3,000 survey responses echoed months later. Project start to shipping for GitHub staff: six weeks, two rounds of research included.
• Plans before people. Every plan starts with what you want to learn or validate; everything you don't know for sure is an assumption, even the ones you're confident about. The work-in-progress pull requests project is the cautionary tale. Interviews revealed the entire premise (draft pull requests as notification noise) was wrong, while testers simultaneously adored the shiny prototype.
• Order matters. Interviews before usability tests, vague questions before specific ones. Once someone sees your design, they can't unsee it, and everything after is colored by it.
• The bad question, dissected. "Do you often suggest single-line changes?" fails five ways: yes/no, hypothetical, assuming behavior, leaking what you want to hear, and answering nothing about relative importance. The fix is a ladder of open questions grounded in what people have actually done.
• Session craft. Always bring a note-taker (and your stakeholders: watching someone use their feature converts engineers to research believers). Ask testers to say what they'd click and what they expect before clicking. When they ask how something works, ask them how they think it works. Debrief immediately.