Promoting Agency in Human-AI Interaction: Stanford CS547 HCI Seminar

The Shift from AI Assistants to AI Advisors

Human-AI interaction is undergoing a fundamental shift as users increasingly turn to Large Language Models (LLMs) for advice on deeply personal topics—such as health, careers, and relationships—rather than using them solely as task-automating assistants. While an assistant performs actions on a user's behalf (e.g., writing code), an advisor aims to augment the user by changing their beliefs and understanding so the user can take better actions themselves.

This distinction is critical because current LLM design and evaluation paradigms focus on automation. However, for personal growth and behavior change, the goal is not to automate the task—since an AI cannot exercise for a user—but to empower the user to act. This approach mirrors the early vision of human-computer interaction pioneered by Douglas Engelbart, who viewed computers as tools for human augmentation rather than mere calculators.

Designing for Agency in Health Behavior Change

Health behavior change serves as an ideal testbed for studying augmentative AI because success depends entirely on the user's own actions. To promote agency, AI systems must avoid being "prescriptive"—the act of blindly dispensing unsolicited advice or imposing solutions—which often undermines user autonomy and fails to produce sustainable change.

The Role of Qualitative Context

Effective coaching relies on eliciting qualitative context—the goals, values, and motivations that underlie a person's reasons for change. While traditional health technology emphasizes quantitative context (e.g., step counts, heart rate), qualitative context is what actually drives behavior change.

By asking open-ended questions (e.g., "Why do you want to be more active?"), an AI advisor achieves two goals simultaneously:

1. It gathers the information necessary to provide aligned, effective support.
2. It invites the user to take an active role in the process, thereby strengthening their own autonomy.

Implementing Non-Prescriptive Strategies

To move away from the prescriptive nature of instruction-tuned LLMs, the GPT Coach system was developed using strategies from Motivational Interviewing (MI). MI is an evidence-based communication style that emphasizes avoiding unsolicited advice and instead eliciting the client's own reasons for change.

GPT Coach utilizes three specific prompt chains to ensure non-prescriptive behavior:

• Dialogue State Chain: Keeps the model on task during long conversations.
• Motivational Interviewing Chain: Forces the model to select from a validated set of counseling strategies (e.g., support, open-ended questioning) before generating a response.
• Tool Use Chain: Determines when to augment the conversation with quantitative wearable sensor data to affirm user strengths rather than just dictate goals.

Evaluating LLM-Augmented Interactions: GPT Coach and Bloom

Research into these systems demonstrates that non-prescriptive AI can significantly impact user psychology and engagement.

GPT Coach Findings

• High Fidelity: Expert coding showed that GPT Coach used consistent or neutral MI strategies over 93% of the time.
• Agency Support: Participants reported that the system helped them reflect on their own motivations and made their ideas more specific without providing unsolicited advice.
• Comparison to Vanilla LLMs: Counterfactual analysis revealed that vanilla GPT-4 is significantly more prone to persuasion and unsolicited advice than the structured GPT Coach system.

Bloom: Integrating Coaching into an Ecosystem

Bloom expanded the coaching agent into a full iOS application. It used the qualitative context elicited during coaching conversations to personalize downstream interactions, such as collaborative goal setting, push notifications, and data visualizations.

In a 4-week field study with 54 participants, Bloom showed:

• Mindset Shifts: Treatment participants expressed more nuanced changes in their beliefs about physical activity (e.g., moving from "working out sucks" to seeing it as an enjoyable part of their routine).
• Increased Engagement: Treatment users spent over five times more time in the app than the control group.
• Agency and Control: Participants attributed their positive mindset shifts to the feeling of being in control and having agency over their choices.

Algorithmic Approaches to Navigating Uncertainty

A primary limitation of prompt-based non-prescriptiveness is that it is often "hard-coded," leading to rigidity or excessive verbosity. To solve this, the research proposes a framework for Zero-Shot Bayes Adaptive Planning to help LLMs strategically navigate uncertainty about the user's latent state ($\theta$).

The Bayes Adaptive MDP Framework

In a standard Markov Decision Process (MDP), an agent treats all users the same. In a Bayes Adaptive MDP, the agent recognizes that every user is different and must learn about the specific user through interaction. The agent must strategically trade off the value of asking an informative question (reducing uncertainty about $\theta$) against the value of acting on current information.

The Three-Step Planning Pipeline

To implement this without extensive prior training data, a multi-stage prompting pipeline was developed:

1. Belief Elicitation: The LLM is prompted to output a posterior probability distribution over the user's latent attributes (e.g., stage of change).
2. Strategy Ranking: The LLM uses these beliefs to rank high-level strategies (e.g., "ask a question" vs. "make a recommendation") based on expected future returns.
3. Action Generation: The LLM generates the final utterance based on the selected strategy and the current beliefs.

Results across Environments

Testing across exercise recommendation, medical diagnosis (MedQ), and negotiation (Deal or No Deal) environments showed that this pipeline leads to substantial policy improvement. Notably, it allowed smaller models (like GPT-4o mini) to recover performance levels comparable to much larger frontier models by explicitly reasoning about uncertainty and beliefs.