Machines as Method: The Use of Artificial Intelligence in Advertising Research

Editors

• Jameson Hayes, University of South Carolina
• Edward Malthouse, Northwestern University

Description

Advertising researchers are increasingly using artificial intelligence as a research tool—large language models now serve as survey respondents, moderators, analysts, and predictors. This shift is outpacing academic attention; while practitioners have deployed synthetic respondent platforms and AI-moderated research at scale, concerns about bias, validity, and accuracy remain largely unexamined. This special issue seeks rigorous work evaluating whether and when AI-based methods produce trustworthy advertising research.

The special issue addresses several pressing concerns: synthetic respondents may simulate what people say about ads rather than what ads actually do to them, given that much advertising effect operates through low-attention and implicit processes. The field lacks clear standards for validating these tools, and the gap between commercial deployment and published research is widening. Both quantitative and qualitative approaches, benchmarking studies, and independent evaluations of commercial tools are welcomed.

Potential topics

• Does It Predict? The Validity of Synthetic Ad Testing - When can LLM-based synthetic respondents stand in for human participants in advertising research, and when do they fail?
• How well do synthetic panels reproduce human results on core advertising outcomes (e.g., attention, recall, attitude toward the ad, purchase intent), and under what boundary conditions?
• Can synthetic respondents replicate established advertising effects (e.g., mere exposure, source credibility, fear appeals), and what does failure to replicate reveal?
• What is the appropriate validation criterion: human panel agreement, or in-market outcomes such as sales and brand lift?
• How much variance compression, homogenization, or demographic distortion do synthetic samples introduce, and how should uncertainty be quantified and reported?
• Given that models are trained on the advertising literature itself, when does an apparent confirmation of theory reflect evidence rather than regurgitation?
• Which advertising responses (e.g., deliberative judgments, verbal attitudes) can LLMs plausibly simulate, and which (e.g., implicit memory, affective response, low-attention processing) remain out of reach?
• How do synthetic responses compare with human data across high- and low-involvement conditions, or across System 1 and System 2 dominant tasks?
• Do synthetic respondents exhibit persuasion knowledge, skepticism, or ad avoidance in ways that mirror or distort human patterns?
• What theoretical frameworks best explain where machine simulation of consumer response breaks down?
• Machines That Listen: AI-Moderated Qualitative Research - What happens to qualitative advertising research when the moderator, the coder, or both are machines?
• How does AI moderation compare with skilled human moderation in probing depth, laddering, and the elicitation of meaning?
• Do consumers disclose differently to AI interviewers, particularly for sensitive or socially undesirable topics relevant to advertisers?
• How reliable are LLMs as qualitative analysts relative to human coders, and what is lost or gained in machine-led thematic analysis?
• Does qualitative research at machine scale change what qualitative inquiry is for, or merely how much of it can be done?
• AI as Measurement Instrument - How trustworthy are machines as coders and predictors of advertising content and response?
• How valid are AI-predicted attention, emotion, and memorability scores relative to eye tracking, facial coding, and other biometric ground truths, and where do the predictions break down?
• Can LLMs and multimodal models reliably code advertising content at scale (e.g., creativity, emotional tone, brand prominence, message strategy), and how should such measures be validated?
• What can computational reanalysis of large advertising archives (e.g., tracking studies, open-ended verbatims, effectiveness case libraries) reveal that original analyses could not?
• Stress Tests and Stand-Ins: Studying What Could Not Be Studied - Can AI extend advertising research into territory that was previously impractical, or impermissible, to study?
• Can adversarial AI populations red-team creative before launch, surfacing misinterpretation, offense, and unintended meanings across segments?
• Under what conditions, if any, are synthetic stand-ins defensible for audiences that are restricted or difficult to research directly (e.g., children, patients, regulated categories)?
• How should the field confront the fidelity paradox: synthetic methods are most attractive precisely where human ground truth is least available?
• The Changing Research Pipeline - How is AI reshaping the practice, economics, and integrity of advertising research?
• When creative variants can be generated and scored predictively at scale, what becomes of the pretest as a discrete stage, and of the research function itself?
• How accurate are commercial AI research tools when evaluated independently, and what evaluation frameworks should the field adopt?
• How prevalent is AI contamination of human data (e.g., bots, LLM-assisted respondents), and how can it be detected, and what does it mean for the panel infrastructure advertising research depends on?
• What disclosure and reporting standards should journals, firms, and industry bodies require when AI participates in the research process?