Skip to content
Commit 02dab418 authored by Pol Dellaiera's avatar Pol Dellaiera Committed by Arkadiusz Kondas
Browse files

Provide a new NGramTokenizer with minGram and maxGram support (#350)

* Issue #349: Provide a new NGramTokenizer.

* Issue #349: Add tests.

* Fixes from code review.

* Implement NGramTokenizer with min and max gram support

* Add missing tests for ngram

* Add info about NGramTokenizer to docs and readme

* Add performance test for tokenization
parent b3fe9dae
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment