clips/pattern

Web mining module for Python, with tools for scraping, natural language processing, machine learning, network analysis and visualization.

What it is

Pattern is a Python module for web mining — pulling data from the web and analyzing it. It was built for Python 2.7 and 3.6.

Main capabilities

Data mining: connect to web services (Google, Twitter, Wikipedia), crawl websites, parse HTML.
Natural language processing: tag parts of speech, search n-grams, analyze sentiment, use WordNet.
Machine learning: vector space models, clustering, classification (KNN, SVM, Perceptron).
Network analysis: graph centrality and visualization.

Example use case

The README shows a Twitter sentiment classifier: it collects tweets containing #win or #fail, extracts only adjectives from each tweet, converts them into word-count vectors, and trains a KNN classifier to predict whether new tweets are more like WIN or FAIL.

Installation

Install with pip:

pip install pattern

Or download and run python setup.py install. You can also place the pattern folder next to your script or add it to sys.path manually.

Status — archived

The repository is no longer maintained. It will not receive updates, bug fixes, or security patches. Issues and pull requests may not be reviewed. Use at your own risk.

License

BSD (3-clause).

Bundled components

Pattern ships with several third-party tools, including Brill taggers for English, Dutch, German, Spanish, French, and Italian; LIBSVM and LIBLINEAR for machine learning; NetworkX for graph analysis; and a spelling corrector.

Documentation

User documentation and examples are available on the project's GitHub wiki.

Reference

De Smedt, T., Daelemans, W. (2012). Pattern for Python. Journal of Machine Learning Research, 13, 2031–2035.