OpenAI Zelma: Making Education Data Accessible via GPT-4

Zelma is a GPT-4 powered research assistant that enables parents, teachers, school administrators, and policymakers to access and analyze U.S. standardized test data for students in grades 3-8. By converting scattered, multi-format education data into a uniform format and leveraging OpenAI's API, Zelma allows users to obtain tailored educational insights using plain language.

Technical Implementation of Zelma

Zelma was developed by Dr. Emily Oster and a team of student researchers at Brown University, who spent a year cleaning and unifying education data. The tool was developed in partnership with Novy to implement the following technical components:

GPT-4 and Function Calling

Novy used OpenAI's function calling capabilities to enable GPT-4 to select the appropriate visuals and data fields for displaying information to the user.

Fine-Tuning for Query Suggestions

To improve the user experience in the "Ask a Question" view, the team fine-tuned a model to create a data-aware type-ahead feature that suggests questions based on the available data.

Vector Database for Accuracy

Known-good example graphs were embedded and stored in a vector database. This approach wases used to improve the accuracy of the responses when handling difficult edge cases.

User Experience and Scope Management

To ensure users remain within the scope of Zelma's knowledge and to handle the variety of ways people phrase questions, the developers implemented several design choices:

• Example Prompt Suggestions: Because Zelma is specialized in a specific domain of education data, example prompts guide users on how to phrase queries and what the tool can answer.
• Public Question Logs: All questions asked of Zelma are made public, allowing users to learn from other queries and discouraging the irrelevant queries.
• SQL Code Transparency: Zelma displays the SQL code used to generate its answers, allowing users to verify the logic. For non-technical users, the same logic is explained in plain language.
• Contextual Explanations: The tool provides definitions and flags notable events, such as changes in state assessments, to help users understand external factors that may have influenced the data for a specific year.