Part 2 of 2
Part 1 of this article closed with a question. What does the interrogation of AI output actually look like? And can it be built into a workflow?
Human-in-the-loop has become a compliance term. A checkbox. Someone is nominally present at the handoff points, reviewing the brief, approving the criteria, and signing off the design. That sounds just about right.
But presence isn’t an interrogation. And a practitioner who hasn’t developed the discernment to tell salience from significance will confirm a fluent brief just as readily as an AI will generate one.
So, the workflow question and the capability question are the same.
Let’s take Training Needs Analysis (TNA) as the case in point. Discernment at the TNA stage would be three simple questions intentionally resolved before the brief is frozen:
- What will the learner do differently that they are not doing now? In which specific situation is it required?
- What is actually stopping them? Is that something a learning intervention can change?
- What does success look like, and can the deWhat does success look like, and can the design create the conditions for it? sign create the conditions for it?
These aren’t any new questions. Every experienced practitioner knows them. The problem is that fluent AI output — a coherent brief, crisply worded objectives, a plausible structure — creates the feeling that they’ve already been answered. The interrogation doesn’t happen because nothing signals that it’s needed. And in a workflow optimized for speed, nobody builds that signal in.
The same deliberation applies at every stage of the design process, not just TNA. Design, development, evaluation. The questions change. The discipline doesn’t.
There’s a difference between a review gate and an interrogation point. Most AI-assisted workflows have the first. Almost none have the second.
A review gate asks: does this look right? An interrogation point asks: have we answered the questions that would tell us if this were right? The first is a read. The second is judgment. They can look identical from the outside. For instance, a practitioner sitting with a brief, a set of objectives, or a design structure. The difference is in what they’re actually doing.
Here’s what discernment looks like at each stage of an ADDIE workflow.
Does the gap sound plausible?
What will the learner do differently, in which specific situation, that they are not doing now?Five stages. Five discernment points. None of them are reviews. Each one requires a practitioner to make a judgment the agent cannot make, because it requires knowing the difference between output that is fluent and output that is grounded.
That’s what an active human in the loop looks like. Not sign-off. Not oversight. A specific judgment, at a specific moment, determines whether the efficiency AI delivers becomes meaningful or just fast.
Can discernment be built into an AI-enabled workflow without slowing it down?
The assumption is that rigor and speed trade off against each other. They don’t. But rigor is not something a prompt can generate. It has to be built into the system.
What does systems design deliver that prompt engineering cannot?
Prompt engineering optimizes what the AI produces. Systems design determines whether what gets produced is the right thing to act on. Conflating the two is how AI-enabled L&D workflows produce faster versions of the same insufficient designs.
Salient content delivered with confidence. Significant outcomes are left unaddressed.
The architecture that changes this is not complicated. AI brings speed and coherence. The practitioner brings discernment. The workflow makes discernment a structural requirement, not a personal trait.
That is better business value. For the L&D team. For the client organization. For a function that has spent too long measuring activity instead of impact. And it is available now, to any L&D function willing to design it. Do you want to explore how you can ensure significance rather than salience within your learning programs? We are happy to have a chat and identify how you can design discernment into your workflows.
References
Alter, A. L., & Oppenheimer, D. M. (2009). Uniting the tribes of fluency to form a metacognitive nation. Personality and Social Psychology Review.
Schwarz, N. (2004). Metacognitive experiences in consumer judgment and decision making.
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency.
What does the interrogation of AI output actually look like? And can it be built into a workflow? Human-in-the-loop has become a compliance term. A checkbox. Someone is nominally present at the handoff points, reviewing the brief, approving the criteria, and signing off the design. That sounds just about right.
