GaoQ1/rasa_nlu_gq

What it solves

This project provides a set of custom components and extensions for Rasa NLU, specifically optimized for Chinese language processing. It allows developers to integrate more advanced entity extraction and intent classification models into their Rasa pipelines without needing to modify the core Rasa source code.

How it works

It functions as an addon that can be loaded into the Rasa NLU pipeline via configuration files. It introduces several specialized modules:

• Entity Extractors: Implements BiLSTM+CRF and IDCNN+CRF (dilated convolution) models for more accurate entity recognition.
• Feature Extraction: Integrates BERT for generating word vector features.
• Intent Classification: Adds BERT-based intent classifiers and a TensorFlow-based estimator classifier.
• POS Tagging: Adds a Jieba-based part-of-speech (POS) extractor to identify names, locations, and organizations.
• Intent Modification: Allows the system to reverse-modify intents based on detected entities.

Who it’s for

Developers building Chinese-language chatbots or NLU systems using the Rasa framework who need more sophisticated NLP tools than the default offerings.

Highlights

• Modular Integration: Works as a plugin/addon, ensuring compatibility with Rasa updates.
• Advanced NER: Supports BiLSTM+CRF and IDCNN+CRF models for entity extraction.
• BERT Integration: Enables the use of BERT vectors for both feature extraction and intent classification.
• Chinese Language Support: Specifically includes Jieba tokenizer and POS tagging for Chinese text.
• Hardware Optimization: Provides configuration options to manage CPU and GPU utilization for TensorFlow components.