MHCI Capstone Project

Goodwill Lens: AI-Powered Price Support at the Warehouse Rack

Designing a human-in-the-loop ML classification tool and trust framework for Goodwill of Southwestern Pennsylvania.

📸Capture at the rackPoint a $300 phone at the garment — no walks, no kiosk
🧠On-device ensembleMobileNetV2 + ResNet50 consensus via Core ML, zero cloud
🗂️Goodwill taxonomyPredictions mapped to 16 operational clothing categories
✅Override & tagProcessor confirms or corrects, QR label prints

TL;DR

  • The Problem: Goodwill clothing processors experience high cognitive loads, manually evaluating and routing ~10M items annually. With a ~50% annual processor turnover rate, tacit brand and pricing knowledge constantly drains from the organization, causing inconsistent pricing and revenue leakage.
  • The Solution: Developed Goodwill Lens, a zero-cloud-dependency iOS app using Core ML and Vision to classify garments, suggest pricing and routing at the rack, and print tags — governed by a "backup, not boss" design principle to drive adoption.
  • The Outcome: Modeled a ~$5M annual mission opportunity (from a $0.50 average selling-price lift), delivered an on-device iOS classifier, and pivoted design recommendations to integrate with Goodwill's newly launched ReNu softlines platform.

Context & Challenges

Goodwill operates on tight operational margins where processing speed directly dictates retail store inventory and funding for community programs.

  • My Role: UX Engineer & Researcher (designed the iOS interface, implemented the Core ML/Vision ensemble prototype, and co-led the Wizard-of-Oz trust study).
  • The Team: 5 CMU MHCI graduate students (Brittany Jain, Will Pan, Casey Potrebic, Yusen Zhang, Holly Zhu) collaborating with Goodwill SWPA leadership.
  • Scale & Variance: Analyzed sales data across 32 stores, uncovering a 37% average selling-price variance in electronics and 34% variance in footwear, proving that category pricing discipline could drive major revenue gains.
  • Revenue Leakage: Diagnosed roughly $935K in sales from "No Tag" items, representing un-attributed revenue that bypasses category tracking.

User Research & Observations

How we spent weeks on the warehouse floor conducting contextual inquiries with 12 processors across 4 stores.

Through job-shadowing and think-aloud sessions, we mapped the back-room processing workflow. Processors are fast manual laborers, but their flow is interrupted every time they have to walk over to a computer kiosk to type, search brand values, or decide whether to route an item to e-commerce.

Key Insights:

  • Cognitive Bottleneck: The bottleneck is cognitive, not physical. Processors lose time when they must guess garment size, identify obscure brands, or debate pricing.
  • Tacit Workarounds: Processors already create personal decision aids—backwards hangers for up-pricing, batching high-value items, and coworker consultations. We aimed to formalize these aids rather than overwrite them.
  • Shopper Pricing Tolerance: Conducted 20 in-store customer interviews across 3 locations, confirming that shoppers value discovery and routine more than accidentally underpriced items, de-risking our pricing consistency goals.
  • Affinity Diagram: Consolidated research into a standard affinity hierarchy (Green → Pink → Blue) containing 6 top-level themes, 20 sub-themes, and 40+ clusters.

How Might We...

How might we guide week-one hires to make consistent pricing decisions without relying on months of accumulated brand knowledge?
How might we design explainable AI recommendations that processors treat as "backup, not boss" rather than a replacement for judgment?
How might we integrate AI recommendations directly into the physical workspace to reduce walks and keystrokes?

AI Adoption & Trust Behavior

A key turning point in our project occurred when we looked beyond technical feasibility to face user trust directly.

We conducted a Wizard-of-Oz recommendation study with 9 processors across 3 stores, capturing 80 item decisions with simulated AI suggestions.

  • The Trust Mismatch: We measured a 51.3% agreement rate and a 48.8% override rate with the AI. Processors consistently overrode prices downward, revealing a gap between absolute market value and store-level sell-through realities.
  • The "Great Walkout": During our research, three processors declined to participate after learning the study involved AI. This became a critical design signal: it pivoted our project focus away from generic accuracy and toward adoption transparency, change management, and override control.
  • Backup, Not Boss: 8 of 9 processors expressed that they wanted the AI to act as a supportive safety net (a "backup") to catch missed brands, rather than an automated decision maker (a "boss").

The Prototype: Goodwill Lens iOS App

Built to operate on-device in under-connected warehouse backrooms.

To address the cognitive bottlenecks, I built Goodwill Lens, a SwiftUI iOS prototype. The app lets a processor capture an image of an item at the rack, triggers an on-device classification consensus, maps the output to a standard Goodwill taxonomy, and previews a printable QR tag.

  • Ensemble Model Strategy: Built a "DeepFashion" classifier ensemble wrapping ResNet50 and MobileNetV2. The app analyzes ImageNet outputs and normalizes them into our 16-entry operational clothing taxonomy (tops, bottoms, activewear, etc.).
  • Zero-Cloud Dependency: Designed for reliability. The prototype runs all inference on-device using Core ML. It avoids third-party package lock-ins and requires no network connectivity, meaning it can run on a $300 phone in any brick warehouse.
  • The Recommendation-with-Override Engine: Designed the parser so that recommendations serve as starting points that processors can quickly override with single-tap selections.

Pivot & Handoff Strategy

Responding to real-world technological roadmaps.

Near the end of our spring research phase, Goodwill SWPA launched its own softlines platform (ReNu) via mobile devices. Building a parallel, competing app would have created operational friction.

We pivoted our deliverables from a standalone SDE prototype to a human-experience layer template designed to integrate directly on top of ReNu. This template provides guidelines for local price calibration, explainable recommendations, trust building, and override mechanisms.

Validation & Executive Storytelling

Presenting findings to Goodwill executives to secure buy-in.

  • Spring Presentation cold-open: We began our presentation to Goodwill executives with a live, hands-on pricing experiment—passing real Goodwill garments around the room to highlight the massive cognitive variance in human-only pricing before revealing our data.
  • "Three Things We Now Know" Video: Produced a 2:30 documentary-style video using anonymous customer and processor audio to articulate customer pricing tolerance, adoption barriers, and ReNu readiness.
  • Regional Scaling: Our research deliverables were so well-received by Goodwill SWPA leadership that they distributed our mid-semester readouts to other regional Goodwill chapters across the country.