Natural language processing and modelling using Scikitlearn Pipelines

In this repository I have stored my workflow for natural language processing using Gradient Boosting, Random Forest, Naive Bayes and Grid Search functionalities in sklearn. I used normal approach and pipeline and used featured union in the pipeline.

Motivation

This notebook is my reference for doing a common natural language processing and modelling workflow. I wanted to apply both procedural workflow and Scikitlearn Pipline approach.

Usage example

Can be used for natural language processing

Requirements

The libraries used in this note put are:

1. re
2. pandas,
3. numpy,
4. sckitlearn,
5. nltk

Disclaimer:

File structure

The file is structured into following sections

Part 0: Loading data

Part 1: Defining methods for preprocessing texts

Part 2: Splitting data and applying processing methods on them

Part 3: Naive bayes method

Part 4: K-fold hold out

Part 5: Random Forest, Gradient Boosting and modelling by GridSearch

Part 6: Pipeline

Part 7: Feature union in pipeline

References

I used my learnings from the bellow into this repository

1. https://www.lynda.com/Python-tutorials/NLP-Python-Machine-Learning-Essential-Training/622075-2.html

2. https://www.udacity.com/course/data-scientist-nanodegree--nd025