webmachinelearning/prompt-api

What it solves

This project proposes a standardized JavaScript API that allows web developers to access language models built directly into the browser or operating system. It eliminates the need for developers to rely on cloud-based AI APIs or manually bundle and run large models using WASM and WebGPU, reducing latency, lowering costs, and improving privacy by processing sensitive data locally.

How it works

The API provides a uniform interface (LanguageModel) to interact with a built-in model. Developers can create sessions, send prompts (either as single strings or structured messages with roles like "system", "user", and "assistant"), and receive responses. It supports both standard responses and streaming results for better user experience.

Key technical capabilities include:

• Sampling Parameters: Control over temperature and topK to tune model behavior.
• Tool Use: Ability to define external JavaScript functions (tools) that the model can invoke to fetch real-time data or perform actions.
• Multimodal Inputs: Support for passing images and audio clips alongside text prompts.
• Session Management: Ability to clone sessions for efficiency and manage context window limits.

Who it’s for

Web developers who want to integrate AI capabilities—such as text summarization, translation, translation, and content generation—into their web applications without the overhead of managing their own AI infrastructure or paying for cloud API tokens.

Highlights

• Local Execution: Enables offline usage and local processing of sensitive data.
• Model Agnostic: Encapsulates model management (downloads, updates) from the developer.
• Tool Integration: Allows models to call JavaScript functions to interact with the rest of the web application.
• Multimodal Support: Handles text, images, and audio as inputs to a single prompt.
• Standardized Interface: Aims for cross-browser interoperability via the Web Machine Learning Community Group.