ITEC 825 · Usability Testing · Spring 2026

SIMPLIFY

A Canvas plug-in for course accessibility, tested with the people who would actually use it.

Jared Gochuico  ·  Ayhorng Heng  ·  Melissa Lee  ·  Sophie Meryash

The Tool

What is SIMPLIFY?

A Canvas LTI plug-in that scans course content for accessibility and usability issues, flags problems, and offers guidance, including one-click fixes.

Screenshot of the SIMPLIFY plug-in interface inside Canvas, showing a course scan with 47 out of 100 score, 33 flagged issues, and a list of items including Missing Alt Text, Objectives, Communication, and Layout Tables, each with severity levels and auto-fix availability

Why we are testing

Purpose & Goals

How we are testing

Mixed-Methods Approach

Phase 01 · Heuristic

Three independent reviews against Nielsen's heuristics. Two by team members on the NN/g worksheet, one by Claude Code as a third lens.

Phase 02 · Moderated

Nine participants across three user groups. Canvas sandbox. Think-aloud. SEQ after each task, SUS at the end. One pilot dry run, plus a calibration session before independent sessions to align observation.

Figure 01 · From the Test Plan

Mapping Tasks to Research Questions

Task Description Related Research Question(s)
1Initiate a course scanCan users successfully initiate a course scan without assistance?
2Interpret scan resultsDo users understand scan results and required actions?
3Navigate issue typesCan users navigate issue types and system organization?
4Locate and apply a fixCan users locate and apply recommended fixes?
5Use an AI-generated fixDo users trust AI suggestions? Is the language clear and actionable?
6Publish changesCan users navigate key controls (scan, fix, publish)?
7Undo changesDo users experience confusion or need for recovery actions?

Figure 04 · From the Test Plan

Who Does What

Participant

Faculty / ID

  • Shares screen
  • Thinks aloud during tasks
  • Completes 7 scenarios
  • Rates via SEQ & SUS
Facilitator

Guides the session

  • Follows the standardized script
  • Introduces task scenarios
  • Encourages think-aloud
  • Does not assist with navigation
Observer

Silent documentation

  • Captures success or failure
  • Notes navigation patterns
  • Documents confusion points
  • Saves questions for debrief

Note. Observer role is used only during the dry run; subsequent sessions have no observer.

Section 02

Data Gathering
Instruments

What we collect, and how we collect it.

Overview

Four Instruments

Instrument 01 of 04

Single Ease Question

Asked after every task. "Overall, how difficult or easy was this task?"

1
Very Difficult
2
Difficult
3
Neutral
4
Easy
5
Very Easy

Modified from the validated 7-point version (Sauro & Dumas, 2009) to a 5-point scale to reduce participant cognitive load. Results will not be benchmarked against the published 7-point norms.

Instrument 02 of 04

System Usability Scale

Used unmodified at the end of every session. Ten questions, scored 1 (strongly disagree) to 5 (strongly agree).

"I think that I would like to use this system frequently."

12345

"I found the system unnecessarily complex."

12345

"I thought the system was easy to use."

12345

…and seven more (Brooke, 1996). Kept as-is for established benchmarking.

Instrument 03 of 04

Observation Template

A structured form the moderator fills in real time, one per task. Here it is for Task 1.

Task Task 1: Initiate a Course Scan
Scenario "You have just installed SIMPLIFY in your Canvas course. You want to check for accessibility issues for the course ITEC 800. Please do that."
Outcome SEQ (1–5) Observations / Notes
✓ Success
☐ Fail 		4 Hesitated ~8s before clicking Scan Course
"oh there it is" when she found the button
Scrolled past the dashboard once

Instrument 04 of 04

Think-Aloud Transcript

Sessions are recorded over Zoom with auto-transcription. The transcript becomes a second pass at every session; the moderator's notes are the first.

From the field · what worked

"I wasn't just using an app. I was learning concepts. I was learning design choices. This is why I would use it." — Faculty participant · Debrief

From the field · what concerned them

"I'm increasingly worried about the use of AI. Some instructors who are pressed for time might not carefully read the AI-generated alt text. Is that really what you wanted your students to know?" — STEM Faculty participant · Task 5 (AI-generated fix)

Section 03

Documents &
Communications

What participants receive, sign, and read, from recruitment through the session itself.

Pre-session

Eligibility Questionnaire

Screens for participants who match our user groups: experience with Canvas and course design.

See Appendix A

Pre-session

Consent & NDA

Recording Consent

Confirms participants understand the session is recorded, that data will be anonymized, and that they can withdraw at any time.

NDA

Protects the prototype, since SIMPLIFY is still in development. Participants agree not to share screenshots or details outside the session.

Signed before the session begins.

During the session

Task Scenarios

Each task is framed as a realistic faculty workflow inside Canvas, not a UI checklist.

Task Scenario
1Initiate a course scan
2Interpret scan results
3Navigate issue types
4Locate and apply a fix
5Use an AI-generated fix
6Publish changes
7Undo changes

Section 05

The Walkthrough

A practice run of one task, end to end, so you can see how the materials come together in a real session.

Demo · Setup

Before the session

Demo · Live

Walking through Task 1

Initiate a course scan. We will moderate this in real time.

References

01 / 27
←→ to navigate