nfstream/nfstream

What it solves

NFStream simplifies the process of analyzing network traffic by providing a high-level Python framework for both online (live) and offline (pcap file) data analysis. It addresses the difficulty of extracting meaningful, reproducible features from raw network packets, especially for encrypted traffic, and provides a standardized way to prepare this data for machine learning models.

How it works

NFStream processes network data using a high-performance computation engine based on CFFI and supports multiplatform execution. It uses nDPI for deep packet inspection to identify encrypted applications and extract metadata (like TLS or SSH fingerprints). It can also probe the system kernel to link network flows to specific local processes (PIDs and process names). For feature extraction, it calculates post-mortem statistics (mean, std dev of packet sizes/inter-arrival times) and early flow features (SPLT analysis of the first n packets).

Who it’s for

It is designed for network researchers and security engineers who need to perform practical network flow analysis, create reproducible experiments, or deploy machine learning models for traffic management and categorization.

Highlights

• Encrypted Visibility: Identifies encrypted applications and extracts metadata fingerprints using nDPI.
• System Integration: Maps network flows to actual system processes via kernel probing.
• ML-Ready: Provides standardized statistical feature extraction and native export to Pandas and CSV for easy model training.
• High Performance: Utilizes AF_PACKET_V3/FANOUT on Linux and supports PyPy for faster execution.
• Extensible: Allows users to create custom flow features using a plugin system (NFPlugins).