Finding Negation in a text is a very tough job. Don't worry, Spacy is there

Like many more NLP tasks, finding negation in a text is a very tough job, which I have faced a lot of time in my career. I tried different syntactical approaches to capture it, but nothing worked well upto my expectation level. The accuracy was pretty low. But after I came to know about Spacy, I gave a try with it. And surprisingly I met with what I was expecting. Lets make this discussion short and bang on with technical stuffs.

I have extensively used Spacy and purely my logics to identify the negation in a sentence. The main technique I have used for the negation identification is Dependency Parsing for the words in a sentence. Using Spacy NLP model, if it is easy to identify which word is getting negatted in a particular sentence structure, then it will be very easy to access where exactly the negation actually is taking place in the sentence structure.

I have pushed one ipython notebook for your reference and code. You can get the notebook from here.

There are 4 different functions working behind to actually do the job.

• parse_dependency_new

• findKeys

• findKeys_K

• negation_existance

The main function is negation_existance. And other 3 functions are the helping functions.

The dependency tree of the four functions looks like following:

Functionalities:

1) parse_dependency_new :

• Purpose: Create a parsed dependency tree structure of word_ids in dictionary format maintaing the parent-child relationship from Spacy NLP dependency structure.

• Input: text you want to parse

• Output: dictionary of word_ids* explaining the dependency tree in the form of key-value pair structure (key as the head and value as the children)

*Note: In spacy if you pass a text through the NLP model, it will provide you unique integer ids for each of the word in the text, using which you can easily identify and access a word in a text.

Example:

code:

doc = nlp("I like tea, but I don't drink coffee")

for token in doc:

print(token.text,': ',token.idx)

Outcome:

How much I have understood, this id generation is probably based upon the character indexing in text only.

2) findKeys_K (this function is actually build upon findKeys function):

• Purpose: Finding the child tree of a key_word in a dependency tree structure

• Input: a) output of parse_dependency_new b) key_word c) word_id_dic (this is a dict object with id and word as key-value pair)

• Output: parsed child tree

3) negation_existance:

• Purpose: Finding either there is negation present for a given key_word or not

• Input: text , key_word

• Output: 0 or 1 ( 0 = Negation doesn't exist for the key word , 1 = Negation exists for the key word )

For an example, say text = "I like tea, but I don't drink coffee"

Is there any negation with 'tea'?

negation_existance("I like tea, but I don't drink coffee",'tea') -> 0

Hence, No Negation with key_word 'tea'.

Is there any negation with 'coffee'?

negation_existance("I like tea, but I don't drink coffee",'coffee') -> 1

Hence, Negation exists with key_word 'coffee'.

This is how simply the negation_existance function works.

Lets follow the notebook and try your hands with the code, as explaining every line of code is out of scope here. Though my code is not perfectly working as the logic I have followed is not fully refined, I will request to apply your own logic of finding negation for a specific keyword, and if you feel that is working better than mine, feel free to go with that. I have given 2 examples in the notebook, from where you will be able to get how the negation_existance is working and how you can use it further.

Thanks for reading. For any query or question please leave a comment below or mail us. Happy Learning!