What Happened and Why? Trace-Guided Micro-Episodes with Elicited User Explanations for ProductĀ Iteration
A workshop position paper about how interaction traces and short, in-context user explanations can work together to support product iteration in creative AI tools.
Abstract
Teams shipping AI workflows in design tools can measure usage yet often struggle to explain why features fail. In creative work, standard metrics are ambiguous: a long session could imply productive exploration or frustrating struggle with stochastic outputs. We argue for trace-guided micro-episodes, a unit of analysis binding interaction logsāwhat users didāto their intent. Rather than relying on disruptive surveys, we propose a āutility-for-rationaleā paradigm: systems offer optional, context-aware controls at likely friction points, capturing user explanations as a byproduct of real-time error recovery. This approach converts ambiguous telemetry into causal evidence without breaking flow. We posit this methodology serves a dual purpose: equipping teams with diagnostic clarity to iterate on vague failure modes (e.g., controllability vs. quality) while generating the grounded alignment data required to train future agents.
Overview
Creative systems produce detailed logs, but those logs are often hard to interpret on their own. A long interaction sequence might reflect productive exploration, repeated verification, or a user struggling to recover from an unsatisfying result. This paper starts from a simple observation: traces are good at showing what happened, but they are often poor at revealing why.
To make those traces more informative, the paper proposes trace-guided micro-episodes: short windows of interaction paired with the local interface context and a lightweight explanation from the user at a moment of friction. Rather than treating explanation as a separate survey task, the proposal is to gather it through useful recovery-oriented interactions inside the tool itself.
Why it matters
This position paper argues that telemetry alone is not enough for understanding creative AI workflows, because the same trace can reflect productive exploration, careful verification, or real friction. It proposes trace-guided micro-episodes as a way to look more locally at moments where users are trying to recover, clarify intent, or repair an output, and pairs those traces with lightweight in-context explanations from the user. The value of that framing is not that it solves the problem outright, but that it gives teams a more concrete way to interpret ambiguous behavior and reason about what kind of support or product change is actually needed.
Cite this paper
- Venue Herding CATs: Making Sense of Creative Activity Traces (CHI 2026 Workshop)
- Publication Date February 25, 2026
- PDF what-happened-and-why.pdf
Use the BibTeX below if you want to cite the paper or add it to a reference manager.
BibTeX
@article{taohappened,
title={What Happened and Why? Trace-Guided Micro-Episodes with Elicited User Explanations for Product Iteration},
author={TAO, SIRUI and MCCARTHY, WILLIAM P and DOW, STEVEN P}
}