J.Doe February 2016

Unable to use word.tokenize() functionc from nltk package

Unable to call the word_tokenize() from the nltk package. Even though I had performed the following steps in terminal

>>python
>>import nltk
>>nltk.download('all')

and following is the simple code for tokenizing

import nltk
sentence = """At eight o'clock on Thursday morning. Arthur didn't feel very good."""
tokens = nltk.word_tokenize(sentence)
tokens

but in spyder console it shows

import nltk
File "nltk.py", line 3, in <module>
tokens = nltk.word_tokenize(sentence)
AttributeError: 'module' object has no attribute 'word_tokenize'
>>> 

Answers


Pranav Waila February 2016

Please check if nltk installation is complete or incomplete. I experienced similar case earlier though it was due to NLTK dataset.


mhawke February 2016

It looks like you named your script nltk.py, so your code is importing itself instead of the nltk module as you expect.

Try changing your script to use another name.


Arqam February 2016

There must be some problem with the installation it seems. If you are using Ubuntu, then see to it that you have installed NLTK in the latest version of Python, because by default Ubuntu has version 2.7 and you have to specify the version you want to install NLTK module separately.

Also check that the pip command that you are using is the one installed for Python 3.4 and not the 2.7 version.

Post Status

Asked in February 2016
Viewed 1,657 times
Voted 4
Answered 3 times

Search




Leave an answer