Uses of Interface
weka.core.tokenizers.cleaners.TokenCleaner

Packages that use TokenCleaner

Package	Description
weka.core.tokenizers
weka.core.tokenizers.cleaners

Uses of TokenCleaner in weka.core.tokenizers

Fields in weka.core.tokenizers declared as TokenCleaner

Modifier and Type	Field	Description
protected TokenCleaner	PreCleanedTokenizer.m_Cleaner	the cleaner to use.

Methods in weka.core.tokenizers that return TokenCleaner

Modifier and Type	Method	Description
TokenCleaner	PreCleanedTokenizer.getCleaner()	Returns the cleaner to use for cleaning the tokens from the initial tokenization.
protected TokenCleaner	PreCleanedTokenizer.getDefaultCleaner()	Returns the default cleaner.

Methods in weka.core.tokenizers with parameters of type TokenCleaner

Modifier and Type	Method	Description
void	PreCleanedTokenizer.setCleaner(TokenCleaner value)	Sets the cleaner to use for cleaning the tokens from the initial tokenization.

Uses of TokenCleaner in weka.core.tokenizers.cleaners

Classes in weka.core.tokenizers.cleaners that implement TokenCleaner

Modifier and Type	Class	Description
class	AbstractTokenCleaner	Ancestor for cleaning tokens.
class	MultiCleaner	Combines multiple cleaners, applies them sequentially.
class	NormalizeDuplicateChars	Replaces all duplicate characters with a single one.
class	PassThrough	Dummy, never removes a token.
class	RemoveNonWordCharTokens	Removes tokens that contain non-word characters.

Fields in weka.core.tokenizers.cleaners declared as TokenCleaner

Modifier and Type	Field	Description
protected TokenCleaner[]	MultiCleaner.m_Cleaners	the cleaners to use.

Methods in weka.core.tokenizers.cleaners that return TokenCleaner

Modifier and Type	Method	Description
TokenCleaner[]	MultiCleaner.getCleaners()	Returns the cleaners to use.
protected TokenCleaner[]	MultiCleaner.getDefaultCleaners()	Returns the default token cleaners.

Methods in weka.core.tokenizers.cleaners with parameters of type TokenCleaner

Modifier and Type	Method	Description
void	MultiCleaner.setCleaners(TokenCleaner[] value)	Sets the cleaners to use.