عزیز قارئین،
In this blog, I will be discussing chunking both theoretically and practically in Python.
تو، چلو شروع کرتے ہیں…
نوٹ: For the implementation, its better to use the Python IDLE as the output is a drawing of a tree which pops up in a separate window.
ایجنڈا
- What is chunking?
- Where is chunking used?
- Types of chunking
- Implementation of chunking in Python
- نتائج کی نمائش
What is chunking?
Chunking is defined as the process of natural language processing used to identify parts of speech and short phrases present in a given sentence.
Recalling our good old English grammar classes back in school, note that there are eight parts of speech namely the noun, verb, adjective, adverb, preposition, conjunction, pronoun, and interjection. Also, in the above definition of chunking, short phrases refer to the phrases formed by including any of these parts of speech.
For example, chunking can be done to identify and thus group noun phrases or nouns alone, adjectives or adjective phrases, and so on. Consider the sentence below:
“I had burgers and pastries for breakfast.”
In this case, if we wish to group or chunk noun phrases, we will get “burgers”, “pastries” and “lunch” which are the nouns or noun groups of the sentence.
Where is chunking used?
Why would we want to learn something without knowing where it is widely used?! Looking at the applications discussed in this section of the blog will help you stay curious till the end!
Chunking is used to get the required phrases from a given sentence. However, POS tagging can be used only to spot the parts of speech that every word of the sentence belongs to.
When we have loads of descriptions or modifications around a particular word or the phrase of our interest, we use chunking to grab the required phrase alone, ignoring the rest around it. Hence, chunking paves a way to group the required phrases and exclude all the modifiers around them which are not necessary for our analysis. Summing up, chunking helps us extract the important words alone from lengthy descriptions. Thus, chunking is a step in information extraction.
Interestingly, this process of chunking in NLP is extended to various other applications; for instance, to group fruits of a specific category, say, fruits rich in proteins as a group, fruits rich in vitamins as another group, and so on. Besides, chunking can also be used to group similar cars, say, cars supporting auto-gear into one group and the others which support manual gear into another chunk and so on.
Types of Chunking
There are, broadly, two types of chunking:
- Chunking up
- Chunking down
Chunking up:
Here, we don’t dive deep; instead, we are happy with just an overview of the information. It just helps us get a brief idea of the given data.
Chunking down:
Unlike the previous type of chunking, chunking down helps us get detailed information.
So, if you just want an insight, consider “chunking up” otherwise prefer “chunking down”.
Implementation of chunking in Python
Imagine a situation in which you want to extract all the verbs from the given text for your analysis. Thus, in this case, we must consider the chunking of verb phrases. This is because our objective is to extract all verb phrases from the given piece of text. Chunking is done with the help of regular expressions.
Don’t worry if it’s the first time you are coming across the term, “regular expressions”. The below table is here, at your rescue:
|
|
|
|
|
|
|
|
|
|
|
|
The above table includes the most common regular expressions used. Regular expressions are very useful in the command line especially while deleting, locating, renaming, or moving files.
Anyways, for this implementation, we will only be using *. Feel free to look at the above table to familiarize yourself with the symbol!
We will be performing chunking using nltk, the most popular NLP library. So, let us first import it.
درآمد nltk
Let’s consider the below sample text which I created on my own. Feel free to replace the below with any sample text you like to implement chunking!
sample_text=""" Rama killed Ravana to save Sita from Lanka.The legend of the Ramayan is the most popular Indian epic.A lot of movies and serials have already been shot in several languages here in India based on the Ramayana. """
Clearly, the data has to be sentence tokenized and then word tokenized before we proceed. Tokenization is nothing but the process of breaking down the given piece of text into smaller units such as sentences, in the case of sentence tokenization and words, in the case of word tokenization.
Followed by tokenization, POS(part-of-speech) tagging is done for each word, in which the part-of-speech of every word will be identified. Now, we are interested only in the verb part-of-speech and wish to extract the same.
Hence, specify the part-of-speech of our interest using the required regular expression as follows:
VB: {}
tokenized=nltk.sent_tokenize(sample_text) for i in tokenized: words=nltk.word_tokenize(i) # print(words) tagged_words=nltk.pos_tag(words) # print(tagged_words) chunkGram=r"""VB: {}""" chunkParser=nltk.RegexpParser(chunkGram) chunked=chunkParser.parse(tagged_words) chunked.draw()
The regular expression(RE) is enclosed within angular brackets() which in turn are enclosed within curly brackets({ and }).
نوٹ: Specify the RE according to the required POS
VB stands for the verb POS. The dot succeeding the VB means to match any character following VB. The question mark after the dot specifies that any character after B must occur only once or must not occur at all. However, from the table which we saw previously, this character is optional. We have framed the regular expression in this manner because, in NLTK, verb phrases include the following POS tags:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Thus, verb phrases can belong to any of the above POS. That’s why the regular expression is framed as VB.? which includes all of the above categories. RegexpParser package is used to check if a POS satisfies our required pattern which we have mentioned using the RE previously.
The entire code can be seen as follows:
import nltk nltk.download('averaged_perceptron_tagger') sample_text=""" Rama killed Ravana to save Sita from Lanka.The legend of the Ramayan is the most popular Indian epic.A lot of movies and serials have already been shot in several languages here in India based on the Ramayana. """ tokenized=nltk.sent_tokenize(sample_text) for i in tokenized: words=nltk.word_tokenize(i) # print(words) tagged_words=nltk.pos_tag(words) # print(tagged_words) chunkGram=r"""VB: {}""" chunkParser=nltk.RegexpParser(chunkGram) chunked=chunkParser.parse(tagged_words) chunked.draw()
نتائج کی نمائش
Finally, we obtain a tree form of the POS of the words along with the words whose POS matches with the given RE. The snapshot of the output obtained for the sample text passed by us can be seen in the above figures.
Observe that the words which satisfy our RE for verb phrases alone are clearly highlighted in the output. Hence, chunking of verb phrases has been performed successfully.
Hope you found my article useful.
آپ کا شکریہ!
حوالہ جات
1. Implementing chunking in Python
3. Full list of POS available in NLP
میرے بارے میں
میں نتھی شری وی ہوں، بی ٹیک کمپیوٹر سائنس اور انجینئرنگ کے آخری سال کی طالبہ ہوں۔ مجھے ایسی ٹھنڈی ٹیکنالوجیز سیکھنا اور ان کو عملی جامہ پہنانا پسند ہے، خاص طور پر یہ دیکھنا کہ وہ معاشرے کے مشکل مسائل کو حل کرنے میں ہماری مدد کیسے کرتی ہیں۔ میری دلچسپی کے شعبوں میں مصنوعی ذہانت، ڈیٹا سائنس، اور نیچرل لینگویج پروسیسنگ شامل ہیں۔
یہ میرا لنکڈ ان پروفائل ہے: My LinkedIn
آپ تجزیات ودھیا سے متعلق میرے دوسرے مضامین پڑھ سکتے ہیں۔ یہاں.
- "
- 7
- تمام
- تجزیہ
- تجزیاتی
- ایپلی کیشنز
- ارد گرد
- مضمون
- مضامین
- مصنوعی ذہانت
- بلاگ
- ناشتا
- کاریں
- کوڈ
- آنے والے
- کامن
- کمپیوٹر سائنس
- اعداد و شمار
- ڈیٹا سائنس
- انجنیئرنگ
- انگریزی
- نکالنے
- پہلا
- پہلی بار
- فارم
- مفت
- گئر
- اچھا
- قبضہ
- گرائمر
- گروپ
- یہاں
- روشنی ڈالی گئی
- کس طرح
- HTTPS
- خیال
- شناخت
- سمیت
- بھارت
- معلومات
- معلومات نکالنا
- انٹیلی جنس
- دلچسپی
- IT
- زبان
- زبانیں
- جانیں
- سیکھنے
- لائبریری
- لائن
- لنکڈ
- لسٹ
- محبت
- نشان
- میچ
- میڈیا
- سب سے زیادہ مقبول
- فلم
- یعنی
- قدرتی زبان
- قدرتی زبان عملیات
- ویزا
- دیگر
- دیگر
- پاٹرن
- جملے
- مقبول
- پو
- حال (-)
- پروفائل
- ازگر
- RE
- قارئین
- باقی
- سکول
- سائنس
- مختصر
- سنیپشاٹ
- So
- حل
- کمرشل
- رہنا
- طالب علم
- حمایت
- ٹیکنالوجی
- وقت
- ٹوکن بنانا
- us
- کیا ہے
- کے اندر
- الفاظ
- سال
- صفر