The Four Lies We Tell Ourselves About AI Interviews (And What Actually Works)

By Jilleun Eglin, Executive Director, Product Last updated: May 2026

Key takeaways:

• Discuss’s Interview Agent is an AI moderation platform built for qualitative research at scale, with the human researcher directing every meaningful decision.
• AI and human moderation work best together: AI for volume, speed, and global reach; human moderators for the conversations that require judgment in the room.
• The quality of your brief determines the quality of your results. A vague Activity Goal produces a vague study.
• Question design for AI requires more precision than for human moderators. One question, one idea. No multi-part setups.
• Probing instructions are where most teams leave insights on the table. Tell the AI what a useful answer looks like, not just what to ask.

AI-moderated interviews are having a moment. Everyone’s talking about them. Some people are overselling them. Others are writing them off entirely.

And in the middle of all that noise? A lot of researchers are telling themselves stories that just aren’t true.

Let’s clear the air. Here are the four biggest misconceptions about AI interviews and what you should actually be doing instead.

Do researchers have to choose between AI and human moderators?

Lie #1: “I Have to Choose Between AI or Human Moderators”

You don’t. And thinking this way is costing you time, money, and better insights.

The best researchers use both.

One of our agency customers had 40 product concepts to test. Forty. If they’d run human-led IDIs on all of them, they’d still be scheduling interviews when their stakeholders needed answers yesterday. Instead, they used AI to quickly narrow the field to the top five concepts, then brought in human moderators to go deep on the finalists.

AI for volume. Humans for nuance. Done in a fraction of the time.

You can also reverse it: run a handful of exploratory human sessions first to figure out what questions actually matter, then validate those findings at scale with AI across hundreds of respondents. Mastercard uses exactly this approach, blending human-led research and AI to close the gap between what consumers say and what they actually do.

The other thing worth calling out: AI doesn’t care about time zones or PTO. Jo Lindenberg, Director of Global Consumer Insights at HelloFresh, put it plainly after her team moved to always-on AI-moderated research with Discuss: “We’ve moved beyond even the 24-hour research cycle to 24 seconds.” That kind of reach requires a researcher who knows what to ask. The AI handles the volume.

Three things AI can do that you’re probably not using it for:

• Democratize research across your org. When brand and product teams are moving fast and need research faster than an internal insights team can deliver, the right platform and guidance lets non-researchers run their own AI-led studies. Your insights team stays focused on strategic, high-impact work.
• Run research while you sleep. AI interviews run 24/7 across the globe. Set them up right, and respondents in Brazil, Japan, and Germany are talking to you while your team is offline.
• Go global without hiring a translator. You don’t need moderators who speak Portuguese, Japanese, and German. The AI handles it.

Stop limiting yourself to “either/or.” Start thinking “yes, and.”

How do I brief an AI moderator?

Lie #2: “The AI Will Just Figure Out What I’m Looking For”

Would you hire a moderator, hand them a one-sentence brief, and say “just wing it”?

No. Because that’s a disaster.

So why phone it in with an AI moderator?

The Activity Goal is where you brief your AI. Phone it in, and you’ll get lame results.

What a good brief includes:

• The research type and audience. Skip “consumers.” Try “budget-conscious millennials who’ve tried and abandoned at least two finance apps in the past year.” Specificity matters.
• Your actual objectives. What are you trying to learn? What decision hinges on this? A vague objective produces a vague study.
• What you’ll do with the insights. Are you shaping a positioning strategy? Prioritizing features? Refining messaging? When the AI knows what decisions are on the line, it steers the conversation toward what’s useful rather than what’s merely interesting.
• Background context. Give the AI the same pre-read you’d give a human moderator. What does it need to know about your brand, product, or industry to ask smart follow-ups?

Garbage in, garbage out. A lazy brief gets you lazy insights.

Is writing questions for AI different from writing them for human moderators?

Lie #3: “Writing Questions for AI Is the Same as Writing Them for Humans”

Almost. But not quite.

A skilled human moderator can recover from a poorly worded question. They can read the room, adjust on the fly, and clarify what you meant. AI will ask exactly what you wrote and probe based on exactly what you told it to do. Your questions need to be airtight.

Three rules:

1. One question. One idea.

Don’t ask: “What are your thoughts on the pricing and the usability of the app?”

You just asked two completely different questions and mashed them into one sentence. Split them:

• “What do you think about the app’s pricing?”
• “How would you describe the app’s usability?”

2. Keep your setups short and clear.

A little context is useful. A paragraph of preamble is a disaster.

Good: “Think back to your last shopping trip. What stood out to you about the store entrance?”

Bad: “Tell me what you think about the app, and whether you’d keep using it if the features improve, which they might in the next version, depending on feedback.”

Nobody knows what you just asked.

3. Ask truly open-ended questions.

Start with why, what, how, or tell me about. These invite stories, not yes/no answers.

• Exploratory: “Tell me about the last time you tried to stick to a budget.”
• Comparative: “How is this different from other tools you’ve used?”

AI doesn’t let you get away with lazy question writing. That’s a feature, not a flaw. It makes you a better researcher.

How does probing work in AI-moderated interviews?

Lie #4: “If I Just Ask the Question, the AI Will Know How to Probe”

This is where most teams leave insights on the table. The AI can probe. But it doesn’t know how you want it to probe unless you tell it.

Probing instructions are where you turn a surface-level answer into something you can actually use.

Tell the AI what you’re hoping to learn.

Don’t just ask the question. Explain the why behind it.

Example:

Question: “If you were telling a friend why you use your budgeting app, what would you say?”

Probing instruction: “Capture how participants naturally talk about value. Listen for both emotional and practical language.”

Now the AI isn’t just collecting an answer. It’s listening for something specific.

Give it concrete follow-up directions.

Be explicit:

• “If the answer is vague, ask for a concrete example.”
• “Probe on how they chose that option and whether they tried others first.”
• “Ask them to walk through it step by step.”
• “Dig into how it made them feel.”

Example:

Question: “What tools do you use to manage your budget?”

Probing instruction: “Get the app names and their role in the daily routine. Probe on why they picked that one and if they’ve tried alternatives.”

Match your probing depth to the question.

Discuss’s Interview Agent lets you set how many follow-up exchanges the AI will have per question. Most teams don’t use it. It’s one of the most consequential settings in the platform.

How to think about it:

• 1 to 3 exchanges for quick, straightforward questions
• 4 to 6 for deeper exploration
• 6 or more for complex, multi-layered probing

If you write detailed probing instructions and set the depth to 2, the AI won’t have room to do what you asked. Give it the space it needs.

The real truth about AI interviews

The teams doing this well are using AI to go further than they could before, faster, across more markets, with more respondents, while keeping their research instincts at the center of every study.

Brief them like you’d brief a human. Write tight questions. Tell them how to probe. That’s the job. The AI handles the rest.

Ready to put these into practice? See how Discuss’s Interview Agent works or reach out for a demo.

Frequently asked questions about AI-moderated qualitative research

What is AI moderation in qualitative research?

AI moderation uses an automated interview agent to conduct qualitative research conversations with respondents, following a researcher-designed guide, asking follow-up probes, and adapting based on responses. The researcher designs the study; the AI runs it at scale, around the clock, across languages. Discuss’s Interview Agent is an AI moderation tool built specifically for qualitative research.

How is AI moderation different from human moderation?

A human moderator reads the room, improvises, and recovers from a poorly worded question. An AI moderator runs exactly the study you designed, with no variance between respondents, at any hour, in any market. AI requires more precise upfront design. The researcher’s judgment shifts from the interview itself to the brief, the question structure, and the probing instructions.

Do I have to choose between AI and human moderators?

No. The most effective research programs use both. AI handles volume, speed, and global reach; human moderators go deep on the topics and moments that require judgment in the room. Many teams use AI to screen and prioritize, then bring human moderators in for the highest-stakes conversations.

How do I write good probing instructions for an AI interview?

Explain the intent behind each question, not just the question itself. Tell the AI what a useful answer looks like, what a vague answer should trigger, and how many exchanges it should pursue per question. Specific follow-up directions (“probe on which alternatives they tried and why they switched”) outperform generic instructions (“ask for more detail”).

What is probing depth and how should I set it?

Probing depth controls how many follow-up exchanges the AI will conduct per question. A depth of 1 to 3 suits straightforward questions; 4 to 6 works for moderate exploration; 6 or more is for complex, layered topics. Setting the depth too low for a question with detailed probing instructions means the AI won’t have room to follow through. Match the depth to what you’re actually trying to learn.