ucinlp/autoprompt

What it solves

AutoPrompt is designed to automate the creation of prompts for various Natural Language Processing (NLP) tasks. Instead of manually crafting prompts, it allows users to elicit knowledge from masked language models (MLMs) like BERT and RoBERTa to perform tasks such as sentiment analysis, natural language inference (NLI), fact retrieval, and relation extraction.

How it works

The system uses a gradient-guided search to automatically find the best "trigger tokens" that, when added to a template, maximize the performance of the language model on a specific task.

Users define a template containing placeholders for the input text and a special [MASK] token (the placement [P]). The tool then searches for a set of trigger tokens [T] that are shared across all prompts to optimize the model's output at the mask position.

Who it’s for

Researchers and developers working with masked language models who want to optimize prompt engineering without manual trial-and-error.

Highlights

• Automated Prompt Generation: Uses gradient-guided search to replace manual prompt engineering.
• Diverse Task Support: Specifically designed for sentiment analysis, NLI, fact retrieval, and relation retrieval.
• Multimodel Compatibility: Supports both BERT and RoBERTa architectures.
• Integrated Evaluation: Includes scripts for evaluating prompts on fact retrieval and relation extraction tasks.