zhengli97/Awesome-Prompt-Adapter-Learning-for-VLMs-CLIP

A curated list of awesome prompt/adapter learning methods for vision-language models like CLIP.

What is this project?

This is a curated list (a collection) of research papers and resources about prompt learning and adapter learning for vision-language models (VLMs) — models that understand both images and text, like CLIP. The list is maintained on GitHub and is meant to help researchers find relevant work in this area.

What does it contain?

The README is organized into several sections:

  • Surveys – Overview papers that summarize the field.
  • Foundation Models – Key VLMs (e.g., CLIP, ALIGN) that are used as base models.
  • Datasets – Standard datasets used for evaluating these methods (e.g., ImageNet, Caltech101).
  • General Prompt Learning – A large section with a comparison table and a paper list.
  • Other sections – For video understanding, continual learning, test-time prompt learning, and more.

Key details

  • Experimental comparison table: Shows average results (Base, Novel, HM) on 11 datasets for various methods, using the ViT-B/16 CLIP model. Only methods with open-source code are listed.
  • Paper list: Organized by year (2022–2025), each entry includes the paper title, publication venue, links to the paper and code (if available), and a badge indicating whether the method uses text prompts, image prompts, or both.
  • Tips: The maintainers invite contributions, ask for missing papers, and note that papers without code are not recommended.
  • Keywords: Badges (Text, Image, Image–Text) indicate what type of prompt/adapter a method uses.
  • Dataset links: The README provides links to standard datasets and also hosts a HuggingFace repository with all datasets (except ImageNet) for easy download.

Purpose

This project is a reference hub for researchers and practitioners working on adapting large vision-language models to new tasks using lightweight techniques like prompts or adapters. It saves time by collecting the most relevant papers, code, and benchmarks in one place.

Note: The README itself is the only source of information; no extra features are described beyond what is listed.

Related

  • Project
    sign/translate

    A real-time bidirectional translation system that converts sign language videos into spoken text and audio, and converts spoken language into sign language via avatars or GANs.

  • Project
    mll-lab-nu/Awesome-Spatial-Intelligence-in-VLM

    A curated list of research papers, datasets, and benchmarks on spatial intelligence in Vision-Language Models, organized as an awesome-list with tables of methods and links to code.

  • Project
    OpenVGLab/OmniLottie

    OmniLottie is a multimodal AI generator that creates complex vector Lottie animations from text, image, or video inputs using pre-trained Vision-Language Models.

  • Project
    mbzuai-oryx/LLaVA-pp

    LLaVA++ enhances LLaVA 1.5 by integrating LLaMA-3 and Phi-3 LLMs to improve visual instruction-following and academic task performance.