Methods
tokenizer
tokenizer() → {Tokenizer}
Creates an instance of Tokenizer.
Example
// Load wink tokenizer.
var tokenizer = require( 'wink-tokenizer' );
// Create your instance of wink tokenizer.
var myTokenizer = tokenizer();Returns
object conatining set of API methods for tokenizing a sentence and defining configuration, plugin etc.
- Type
- Tokenizer