logpai/loghub

What is Loghub?

Loghub is a public collection of system log files meant for research on AI-driven log analytics. The logs come from real systems — some are production data released in earlier studies, others were collected in the project's own lab. The authors deliberately avoid sanitizing, anonymizing, or modifying the logs wherever possible, so researchers get raw, realistic data.

The datasets are free to use for research or academic work, and have been downloaded by more than 450 organizations from both industry and academia.

What's inside?

The README lists 20 datasets, grouped into six categories:

• Distributed systems — HDFS (3 versions), Hadoop, Spark, Zookeeper, OpenStack
• Super computers — BGL, HPC, Thunderbird
• Operating systems — Windows, Linux, Mac
• Mobile systems — Android (2 versions), HealthApp
• Server applications — Apache, OpenSSH
• Standalone software — Proxifier

Each dataset entry shows:

• Description — what system produced the log (e.g., "Hadoop distributed file system log")
• Labeled? — whether the log comes with ground-truth labels (marked with a checkmark; some are unlabeled)
• Time span — how long the log covers (e.g., 38.7 hours, 214.7 days, or "N.A." if not stated)
• #Lines — number of log lines (ranging from ~21,000 for Proxifier to ~211 million for Thunderbird)
• Raw size — file size (from ~2 MiB to ~30 GiB)
• Download — a direct link to the raw log file

All downloads point to Zenodo records.

How to use it

• License: The datasets are free for research or academic work. If you use or distribute them, you must reference the repository URL (https://github.com/logpai/loghub) and cite the Loghub paper.
• Citation: Two papers are listed — the original Loghub paper (ISSRE 2023) and a follow-up evaluation paper (ISSTA 2024).
• Discussion: Questions and discussion happen via GitHub Discussions on the repository.

Key takeaway

Loghub is a ready-made, freely downloadable set of real system logs, organized by system type, with metadata (size, line count, labeling status) so researchers can pick the right dataset for log-parsing, anomaly detection, or other AI-driven log analysis work.