ASE (Agentic Software Engineering) — Project Explainer

What it is

ASE is a carefully chosen collection of 1,663 research papers about Agentic Software Engineering — that is, using AI agents / large language models (LLMs) to do software engineering tasks. The papers come from major conferences in software engineering, programming languages, security, and NLP. The project also ships an automated paper-labeling skill: a pipeline built on Claude Code that reads raw conference proceedings files (in BibTeX or HTML formats), picks out the papers that are relevant, classifies them by topic, and merges them into the database — with little manual effort.

The database / website

Papers can be browsed via a website (open web/index.html locally, or browse online). The interface supports:

Full-text search across titles and abstracts
Year and venue filters — independent single-select dropdowns; venue names are normalized (e.g., "ICSE" matches all ICSE years)
Label filter — pick one or more research topics from a sidebar or by clicking label pills on paper cards; multiple labels combine with AND logic
Expandable abstracts — click any paper card to reveal its abstract
Active filter summary — each active constraint shows as a removable tag below the toolbar

All filter dimensions (year, venue, labels) are optional and combine with AND logic: only papers satisfying every active constraint are shown.

Scope and tracked venues

Papers are systematically collected for all proceedings from 2023–2026 that have been publicly released, plus selected papers from earlier years (2020–2022) and other venues on a best-effort basis.

Tracked venues by field:

Software Engineering: ICSE (2023–2025), FSE (2023–2025), ASE (2023–2025), ISSTA (2022–2025), TSE (2023–2024), TOSEM (2023–2024)
Programming Languages: PLDI (2023, 2025), OOPSLA (2023–2025), POPL (2025), CC (2025), COLM (2025)
Security: S&P (2023–2025), USENIX Security (2023–2025), CCS (2023–2025), NDSS (2024–2026), RAID (2023)
NLP: ACL (2023–2025), EMNLP (2020, 2023–2025), NAACL (2024–2025)
Machine Learning: ICML (2021, 2023–2025), NeurIPS (2022–2024), ICLR (2021, 2023–2025)

Publication counts by venue for 2023–2026 (as shown in the README's tables):

Software Engineering (from table): ICSE 166, FSE 130, ASE 292, ISSTA 98 → total 686
Programming Languages (from table): PLDI 6, OOPSLA 34, POPL 1 → total 41
Security (from table): CCS 47, USENIXSec 41, S&P 22, NDSS 55 → total 165 (with 2026 data for NDSS)
NLP (from table): ACL 294, EMNLP 250, NAACL 22 → total 566
(Machine learning counts are not broken down in table form in the README)

How papers are organized (taxonomy)

Papers are classified with a two-level taxonomy: 9 top-level categories and 47 sub-categories. A paper can carry multiple labels. The taxonomy has three super-groups:

1. Agent for SE — papers where LLMs or AI agents are applied to core software engineering tasks:

Code Generation (750 papers) — sub-categories include Program Synthesis (410), Program Repair (223), Code Translation (69), Code Completion (68), Refactoring (37), Decompilation (23)
Static Analysis (446) — sub-categories include Bug Detection (249), Code Summarization (67), Code Search (51), Program Verification (44), and others
Dynamic Analysis (287) — sub-categories include Test Case Generation (118), Fuzzing (58), Domain-Specific Testing (56), Debugging (40), and others
Code Model (427) — Model Training (407), Binary and IR Model (35)
Other SE Tasks (96) — Doc/Comment/Commit Message Generation (35), Log Analysis (34), Code Review (29)

2. Agent Design and Analysis — research on agent architectures and safety/security of code-oriented LLMs:

Agent Design (328) — Planning (206), Tool Use (156), Multi-Agent (96), Memory Management (30)
Model Safety and Security (267) — Adversarial Attack (85), Jailbreaking (66), Secure Code Generation (65), Memorization (40), Backdoor Detection (36), Watermarking (35)
Agent Safety and Security (97) — Prompt Injection (73), Agent Defense (36), Access Control (12)

3. Evaluation — benchmarks, empirical studies, and surveys that assess LLM/agent capabilities for code:

Evaluation (914) — Empirical Study (620), Benchmark (392), Survey (26)

How papers get selected (four-stage pipeline)

Each venue's proceedings are processed through:

Extract — parse titles and abstracts from BibTeX or HTML files
Filter — keep papers whose title or abstract contains both LLM-related terms (e.g., "large language model," "GPT," "agent") and code-related terms (e.g., "program," "software," "testing," "verification"); this keyword pass is deliberately permissive (high recall)
Classify — send each candidate to the Claude API, which verifies relevance and assigns taxonomy labels; a paper is included only if LLMs or AI agents are a central contribution, not merely a baseline or comparison
Merge — add the classified papers to the canonical database (data/labeldata/labeldata.json) and regenerate the website

The automated paper-labeling skill

All scripts live in .claude/skills/paper-labeler/scripts/.

Prerequisites: pip install boto3 requests (boto3 for the Claude API via AWS Bedrock, requests for NDSS scraping). AWS credentials are needed for the labeling step; the filter-only step needs none.

Option A — Batch mode (recommended): scans a rawdata folder, skips venues already recorded in data/venues.json, runs the full pipeline for each new venue, and rebuilds the website. Examples:

python process_folder.py --dry-run — preview what would be processed
python process_folder.py — process all new venues under data/rawdata/
python process_folder.py data/rawdata/2025/ — process a specific year only
python process_folder.py --filter-only — keyword filter only, no API calls, no merge
Key options: --model, --region, --delay SECONDS, --no-rebuild

Option B — Manual per-venue pipeline (for finer control):

Extract — extract_papers.py reads BibTeX (most venues) or ACL Anthology HTML; NDSS HTML contains titles only, so abstracts are fetched separately with fetch_ndss_abstracts.py
Filter and label — label_papers.py with --phase filter (no credentials needed), --phase label (needs AWS credentials), or --phase all
Merge — merge_labeldata.py (supports --dry-run to preview without writing)
Rebuild website — build_site.py → outputs web/index.html

Using via Claude Code: the pipeline can be invoked conversationally, e.g., "Process the ASE2025 rawdata," "Label the papers in data/rawdata/2025/CCS2025.bib," "Rebuild the website" — Claude Code automatically invokes the skill and runs the right commands.

Supported input formats: BibTeX (.bib) for most venues with abstracts inline; ACL Anthology HTML (.html) for ACL, EMNLP, and some NAACL years; NDSS HTML (.html) with abstracts scraped separately.

Script reference: process_folder.py (batch mode), extract_papers.py (step 1), fetch_ndss_abstracts.py (step 1b), label_papers.py (step 2), merge_labeldata.py (step 3), build_site.py (step 4), import_original.py (one-time import of legacy papers).

Contributing

Add individual papers: append an entry to data/labeldata/labeldata.json (fields include title, author, year, abstract, URL, venue, and labels), ensure labels come from the taxonomy, rebuild the website, and open a pull request.
Add a new venue: place the .bib or .html file under data/rawdata/<year>/, run the batch pipeline, and open a pull request with the rawdata file and updated labeldata.json.
Report missing papers: open an issue with the venue name and a link to the proceedings; you can also suggest specific papers with labels.

Extending the taxonomy

To track a different research topic across the same venues, edit two sections in .claude/skills/paper-labeler/SKILL.md:

## Relevance Criteria — keyword lists and the natural-language prompt used by Claude to decide relevance
## Label Taxonomy — the two-level category hierarchy; after editing, keep the TAXONOMY dict in build_site.py and label_papers.py in sync

Then re-run the pipeline on existing rawdata to reclassify papers under the updated taxonomy.

Caveats / disclaimer

The repository is intended solely for research purposes. All metadata is sourced from publicly available proceedings pages on ACM, IEEE, and corresponding conference websites. Full-text PDFs are not included or redistributed.

Contact: stephenw.wangcp@gmail.com or wang6590@purdue.edu.

PurCL/ASE