Quantcast
Channel: Apache Timeline
Viewing all articles
Browse latest Browse all 5648

Write my Analyzer and Fillter, what would the dictionary-file-0 be like?

$
0
0
Hi,

I'm try to write my own analyzer for Naive Bayes classification.
When I run the seq2sparse command and called -a MyANalyzer, The output
shows the algorithim used my filter and analyzer:

### Using length filter
### Using stopword fileter
### Using my filter
### Using my keyword filter

But if I check the dictionary-file-0 generated in this step, I still find
all the tokens (not filtered!). Does that mean my analyzer didn't work, or
the dictionary-file-0 just record all tokens but the algorithm actually use
the filtered data?

Thanks

Mengfei Ren

Viewing all articles
Browse latest Browse all 5648

Trending Articles