N-grams are all possible combinations of “N” words from the text. When two words are combined at a time, they are known as Bigrams, when three words are combined at a time, they are known as Trigrams, so on and so forth. They are very useful when we are trying to do NLP because combinations of words are more meaningful as compared to individual words.
There are two ways to generate N-grams, either by writing the logic yourself or by using the nltk library function.
|
1 2 3 4 5 6 7 8 9 10 |
# Creating a function to generate N-Grams def generate_ngrams(text, WordsToCombine): words = text.split() output = [] for i in range(len(words)- WordsToCombine+1): output.append(words[i:i+WordsToCombine]) return output # Calling the function generate_ngrams(text='this is a very good book to study', WordsToCombine=3) |
Sample Output
|
1 2 3 4 5 6 7 8 |
# NLTK function to generate ngrams import nltk from nltk.util import ngrams samplText='this is a very good book to study' NGRAMS=ngrams(sequence=nltk.word_tokenize(samplText), n=3) for grams in NGRAMS: print(grams) |
Sample Output
Author Details
Lead Data Scientist
Farukh is an innovator in solving industry problems using Artificial intelligence. His expertise is backed with 10 years of industry experience. Being a senior data scientist he is responsible for designing the AI/ML solution to provide maximum gains for the clients. As a thought leader, his focus is on solving the key business problems of the CPG Industry. He has worked across different domains like Telecom, Insurance, and Logistics. He has worked with global tech leaders including Infosys, IBM, and Persistent systems. His passion to teach inspired him to create this website!