A Tour of Python NLP Libraries

NLP, or Pure Language Processing, is a area inside Synthetic Intelligence that focuses on the interplay between human language and computer systems. It tries to discover and apply textual content information so computer systems can perceive the textual content meaningfully.

Because the NLP area analysis progresses, how we course of textual content information in computer systems has advanced. Fashionable instances, we have now used Python to assist discover and course of information simply.

With Python changing into the go-to language for exploring textual content information, many libraries have been developed particularly for the NLP area. On this article, we’ll discover numerous unimaginable and helpful NLP libraries.

So, let’s get into it.
 

NLTK

 
NLTK, or Pure Language Instrument Equipment, is an NLP Python library with many text-processing APIs and industrial-grade wrappers. It’s one of many largest NLP Python libraries utilized by researchers, information scientists, engineers, and others. It’s a normal NLP Python library for NLP duties.

Let’s attempt to discover what NLTK may do. First, we would want to put in the library with the next code.

Subsequent, we might see what NLTK may do. First, NLTK can carry out the tokenization course of utilizing the next code:

import nltk from nltk.tokenize
import word_tokenize

# Obtain the mandatory assets
nltk.obtain('punkt')

textual content = "The fruit in the table is a banana"
tokens = word_tokenize(textual content)

print(tokens)

Output>> 
['The', 'fruit', 'in', 'the', 'table', 'is', 'a', 'banana']

Tokenization principally would divide every phrase in a sentence into particular person information.

With NLTK, we will additionally carry out Half-of-Speech (POS) Tags on the textual content pattern.

from nltk.tag import pos_tag

nltk.obtain('averaged_perceptron_tagger')

textual content = "The fruit in the table is a banana"
pos_tags = pos_tag(tokens)

print(pos_tags)

Output>>
[('The', 'DT'), ('fruit', 'NN'), ('in', 'IN'), ('the', 'DT'), ('table', 'NN'), ('is', 'VBZ'), ('a', 'DT'), ('banana', 'NN')]

The output of the POS tagger with NLTK is every token and its meant POS tags. For instance, the phrase Fruit is Noun (NN), and the phrase ‘a’ is Determinant (DT).

It’s additionally attainable to carry out Stemming and Lemmatization with NLTK. Stemming is decreasing a phrase to its base kind by chopping its prefixes and suffixes, whereas Lemmatization additionally transforms to the bottom kind by contemplating the phrases’ POS and morphological evaluation.

from nltk.stem import PorterStemmer, WordNetLemmatizer
nltk.obtain('wordnet')
nltk.obtain('punkt')

textual content = "The striped bats are hanging on their feet for best"
tokens = word_tokenize(textual content)

# Stemming
stemmer = PorterStemmer()
stems = [stemmer.stem(token) for token in tokens]
print("Stems:", stems)

# Lemmatization
lemmatizer = WordNetLemmatizer()
lemmas = [lemmatizer.lemmatize(token) for token in tokens]
print("Lemmas:", lemmas)

Output>> 
Stems: ['the', 'stripe', 'bat', 'are', 'hang', 'on', 'their', 'feet', 'for', 'best']
Lemmas: ['The', 'striped', 'bat', 'are', 'hanging', 'on', 'their', 'foot', 'for', 'best']

You possibly can see that the stemming and lentmatization processes have barely completely different outcomes from the phrases.

That’s the easy utilization of NLTK. You possibly can nonetheless do many issues with them, however the above APIs are essentially the most generally used.
 

SpaCy

 
SpaCy is an NLP Python library that’s designed particularly for manufacturing use. It’s a complicated library, and SpaCy is understood for its efficiency and talent to deal with massive quantities of textual content information. It’s a preferable library for business use in lots of NLP circumstances.

To put in SpaCy, you may take a look at their utilization web page. Relying in your necessities, there are numerous mixtures to select from.

Let’s strive utilizing SpaCy for the NLP activity. First, we might strive performing Named Entity Recognition (NER) with the library. NER is a technique of figuring out and classifying named entities in textual content into predefined classes, akin to particular person, handle, location, and extra.

import spacy

nlp = spacy.load("en_core_web_sm")

textual content = "Brad is working in the U.K. Startup called AIForLife for 7 Months."
doc = nlp(textual content)
#Carry out the NER
for ent in doc.ents:
    print(ent.textual content, ent.label_)

Output>>
Brad PERSON
the U.Ok. Startup ORG
7 Months DATE

As you may see, the SpaCy pre-trained mannequin understands which phrase inside the doc might be categorised.

Subsequent, we will use SpaCy to carry out Dependency Parsing and visualize them. Dependency Parsing is a technique of understanding how every phrase pertains to the opposite by forming a tree construction.

import spacy
from spacy import displacy

nlp = spacy.load("en_core_web_sm")

textual content = "SpaCy excels at dependency parsing."
doc = nlp(textual content)
for token in doc:
    print(f"{token.text}: {token.dep_}, {token.head.text}")

displacy.render(doc, jupyter=True)

Output>> 
Brad: nsubj, working
is: aux, working
working: ROOT, working
in: prep, working
the: det, Startup
U.Ok.: compound, Startup
Startup: pobj, in
known as: advcl, working
AIForLife: oprd, known as
for: prep, known as
7: nummod, Months
Months: pobj, for
.: punct, working

The output ought to embrace all of the phrases with their POS and the place they’re associated. The code above would additionally present tree visualization in your Jupyter Pocket book.

Lastly, let’s strive performing textual content similarity with SpaCy. Textual content similarity measures how comparable or associated two items of textual content are. It has many strategies and measurements, however we’ll strive the only one.

import spacy

nlp = spacy.load("en_core_web_sm")

doc1 = nlp("I like pizza")
doc2 = nlp("I love hamburger")

# Calculate similarity
similarity = doc1.similarity(doc2)
print("Similarity:", similarity)

Output>>
Similarity: 0.6159097609586724

The similarity measure measures the similarity between texts by offering an output rating, often between 0 and 1. The nearer the rating is to 1, the extra comparable each texts are.

There are nonetheless many issues you are able to do with SpaCy. Discover the documentation to search out one thing helpful to your work.
 

TextBlob

 
TextBlob is an NLP Python library for processing textual information constructed on high of NLTK. It simplifies lots of NLTK’s utilization and might streamline textual content processing duties.

You possibly can set up TextBlob utilizing the next code:

pip set up -U textblob
python -m textblob.download_corpora

First, let’s attempt to use TextBlob for NLP duties. The primary one we might strive is to do sentiment evaluation with TextBlob. We will try this with the code beneath.

from textblob import TextBlob

textual content = "I am in the top of the world"
blob = TextBlob(textual content)
sentiment = blob.sentiment

print(sentiment)

Output>>
Sentiment(polarity=0.5, subjectivity=0.5)

The output is a polarity and subjectivity rating. Polarity is the sentiment of the textual content the place the rating ranges from -1 (detrimental) to 1 (optimistic). On the identical time, the subjectivity rating ranges from 0 (goal) to 1 (subjective).

We will additionally use TextBlob for textual content correction duties. You are able to do that with the next code.

from textblob import TextBlob

textual content = "I havv goood speling."
blob = TextBlob(textual content)

# Spelling Correction
corrected_blob = blob.right()
print("Corrected Text:", corrected_blob)

Output>>
Corrected Textual content: I've good spelling.

Attempt to discover the TextBlob packages to search out the APIs to your textual content duties.
 

Gensim

 
Gensim is an open-source Python NLP library specializing in subject modeling and doc similarity evaluation, particularly for large and streaming information. It focuses extra on industrial real-time functions.

Let’s strive the library. First, we will set up them utilizing the next code:

After the set up is completed, we will strive the Gensim functionality. Let’s attempt to do subject modeling with LDA utilizing Gensim.

import gensim
from gensim import corpora
from gensim.fashions import LdaModel

# Pattern paperwork
paperwork = [
    "Tennis is my favorite sport to play.",
    "Football is a popular competition in certain country.",
    "There are many athletes currently training for the olympic."
]

# Preprocess paperwork
texts = [[word for word in document.lower().split()] for doc in paperwork]

dictionary = corpora.Dictionary(texts)
corpus = [dictionary.doc2bow(text) for text in texts]


#The LDA mannequin
lda_model = LdaModel(corpus, num_topics=2, id2word=dictionary, passes=15)

matters = lda_model.print_topics()
for subject in matters:
    print(subject)

Output>>
(0, '0.073*"there" + 0.073*"currently" + 0.073*"olympic." + 0.073*"the" + 0.073*"athletes" + 0.073*"for" + 0.073*"training" + 0.073*"many" + 0.073*"are" + 0.025*"is"')
(1, '0.094*"is" + 0.057*"football" + 0.057*"certain" + 0.057*"popular" + 0.057*"a" + 0.057*"competition" + 0.057*"country." + 0.057*"in" + 0.057*"favorite" + 0.057*"tennis"')

The output is a mix of phrases from the doc samples that cohesively change into a subject. You possibly can consider whether or not the end result is sensible or not.

Gensim additionally gives a means for customers to embed content material. For instance, we use Word2Vec to create embedding from phrases.

import gensim
from gensim.fashions import Word2Vec

# Pattern sentences
sentences = [
    ['machine', 'learning'],
    ['deep', 'learning', 'models'],
    ['natural', 'language', 'processing']
]

# Practice Word2Vec mannequin
mannequin = Word2Vec(sentences, vector_size=20, window=5, min_count=1, staff=4)

vector = mannequin.wv['machine']
print(vector)

Output>>
[ 0.01174188 -0.02259516  0.04194366 -0.04929082  0.0338232   0.01457208
 -0.02466416  0.02199094 -0.00869787  0.03355692  0.04982425 -0.02181222
 -0.00299669 -0.02847819  0.01925411  0.01393313  0.03445538  0.03050548
  0.04769249  0.04636709]

There are nonetheless many functions you should utilize with Gensim. Attempt to see the documentation and consider your wants.
 

Conclusion

On this article, we explored a number of Python NLP libraries important for a lot of textual content duties. All of those libraries can be helpful to your work, from Textual content Tokenization to Phrase Embedding. The libraries we’re discussing are:

1. NLTK
2. SpaCy
3. TextBlob
4. Gensim

I hope it helps
 
 

Cornellius Yudha Wijaya is a knowledge science assistant supervisor and information author. Whereas working full-time at Allianz Indonesia, he likes to share Python and information ideas through social media and writing media. Cornellius writes on a wide range of AI and machine studying matters.