Hi,
I'm try to write my own analyzer for Naive Bayes classification.
When I run the seq2sparse command and called -a MyANalyzer, The output
shows the algorithim used my filter and analyzer:
### Using length filter
### Using stopword fileter
### Using my filter
### Using my keyword filter
But if I check the dictionary-file-0 generated in this step, I still find
all the tokens (not filtered!). Does that mean my analyzer didn't work, or
the dictionary-file-0 just record all tokens but the algorithm actually use
the filtered data?
Thanks
Mengfei Ren
I'm try to write my own analyzer for Naive Bayes classification.
When I run the seq2sparse command and called -a MyANalyzer, The output
shows the algorithim used my filter and analyzer:
### Using length filter
### Using stopword fileter
### Using my filter
### Using my keyword filter
But if I check the dictionary-file-0 generated in this step, I still find
all the tokens (not filtered!). Does that mean my analyzer didn't work, or
the dictionary-file-0 just record all tokens but the algorithm actually use
the filtered data?
Thanks
Mengfei Ren