Stanford CS547 HCI Seminar: Just-in-Time Objectives for Specialized AI Interactions

The "Spork Problem" in AI Interaction

Generic AI outputs are the result of what Michelle Lam calls the "spork problem." Much like a spork attempts to be both a spoon and a fork but fails to excel at either, modern Large Language Models (LLMs) are designed as all-in-one assistants, topic experts, and editors. Because developers cannot predict every user situation during training, they bake in broad assumptions and fine-tune models against diverse, often conflicting, preferences. This results in a "spork-like" interface: a single input method that produces generic outputs regardless of the user's specific goal.

This homogenization is not merely a matter of convenience; it can lead to shallow understandings of important texts, the amplification of political polarization in social feeds, and a general trend toward homogeneous thinking across populations relying on the same tools. While users can attempt to override these defaults through complex prompting, prompting is effortful, reactive, and challenging for the average user.

Just-in-Time (JIT) Objectives

To combat generic AI, Lam proposes Just-in-Time (JIT) objectives. Instead of relying on predefined system goals, JIT architectures induce a model of the user's in-the-moment goals by observing interaction traces—such as screenshots of the workspace or the raw DOM of a web page—at the time of interaction.

How JIT Objectives are Induced

JIT objectives are lightweight models consisting of a name, a detailed description, and a weight. The induction process follows these principles:

• Low-Effort Observation: Prioritizing ubiquitous inputs (screenshots, DOM) that are easy to gather.
• Instantaneous Bounding: Focusing on the immediate moment to better capture current activity.
• Uncertainty Reasoning: Allowing the LLM to state its uncertainty, which the pipeline then reasons over.

In a study with over 200 participants, JIT objectives were found to be highly accurate, with users selecting induced objectives in approximately 98% of cases.

Implementation via Generator-Evaluator Architectures

JIT objectives can be integrated into existing AI systems without requiring the retraining of base models. The architecture replaces the initial user prompt with an induced objective, which then steers two primary components:

1. Generators: The JIT objective is appended to generation prompts to tailor the artifact (e.g., a UI specification or a coding plan).
2. Evaluators: The objective is added to scoring rubrics, allowing the system to refine outputs based on stringent, user-specific criteria rather than general quality.

This approach allows for "hill climbing" on objectives, where the system iteratively improves the output to better align with the induced goal. In head-to-head comparisons, JIT generators produced preferred outputs approximately 70% of the time compared to baselines.

Practical Applications of JIT Frameworks

Poppins: On-Demand UI Generation

Poppins is a browser extension that demonstrates JIT objectives through on-demand UI generation. Instead of a chat interface, Poppins observes the user's screen and generates specialized software tools on the fly. Examples include:

• Presentation Flow Organizers: For users refining the structure of a research talk.
• Character Emotion Trackers: For authors tracking emotional trajectories in short stories to identify inconsistencies.
• Technical Method Explorers: For researchers comparing algorithms and their downstream applications.

Loom: Steerable Topic Modeling

For domain experts, Lam introduced Loom, a Python package for steerable and interpretable text analysis. Unlike traditional topic models that return vague keywords, Loom uses modular operators to surface high-level concepts defined by explicit inclusion criteria.

Loom's modular pipeline includes:

• Distill: Extracts key quotes and points from documents.
• Cluster: Groups related text spans using embeddings and HDBSCAN.
• Synthesize: Uses an LLM to draw unifying concepts across clusters.
• Score: Verifies the occurrence of concepts by mapping them back to documents.
• Seed: Allows users to steer any operator using a seed term to modify what the system attends to.

Societal Objective Functions for Social Media

Lam extends JIT objectives to social computing to address systemic issues like partisan animosity. Current social media feeds optimize for engagement (likes/clicks), which often correlates with increased polarization.

By translating social science constructs—such as the "Anti-Democratic Attitudes and Partisan Animosity" (APA) construct—into AI objective functions, researchers developed a feed re-ranking pipeline. This pipeline uses an LLM to rate posts based on APA variables and down-rank content that triggers partisan animosity. Experiments showed that this algorithmic intervention significantly reduced partisan animosity for both Democrats and Republicans.

Future Vision: User-Owned AI

Lam envisions a shift from "AI sporks" to a "Mary Poppins bag"—a user-owned AI system that possesses many functions but only surfaces the specific tool needed for the current task. This vision involves three primary pillars:

1. Adaptive Interfaces: Moving away from rigid, predefined interfaces toward computing environments that mold to the user in real-time, addressing immediate, longitudinal, and societal goals.
2. AI Interaction Observatories: Creating infrastructure to learn from large-scale end-user objectives to discover new tasks and problems for AI research to solve, moving beyond simple LLM logs.
3. Governance and Ownership: Developing technical and usability frameworks that allow users to fine-tune models on their own devices and share data/compute with trusted community members to achieve collective goals.