Uses of Interface
weka.core.tokenizers.cleaners.TokenCleaner
-
Packages that use TokenCleaner Package Description weka.core.tokenizers weka.core.tokenizers.cleaners -
-
Uses of TokenCleaner in weka.core.tokenizers
Fields in weka.core.tokenizers declared as TokenCleaner Modifier and Type Field Description protected TokenCleaner
PreCleanedTokenizer. m_Cleaner
the cleaner to use.Methods in weka.core.tokenizers that return TokenCleaner Modifier and Type Method Description TokenCleaner
PreCleanedTokenizer. getCleaner()
Returns the cleaner to use for cleaning the tokens from the initial tokenization.protected TokenCleaner
PreCleanedTokenizer. getDefaultCleaner()
Returns the default cleaner.Methods in weka.core.tokenizers with parameters of type TokenCleaner Modifier and Type Method Description void
PreCleanedTokenizer. setCleaner(TokenCleaner value)
Sets the cleaner to use for cleaning the tokens from the initial tokenization. -
Uses of TokenCleaner in weka.core.tokenizers.cleaners
Classes in weka.core.tokenizers.cleaners that implement TokenCleaner Modifier and Type Class Description class
AbstractTokenCleaner
Ancestor for cleaning tokens.class
MultiCleaner
Combines multiple cleaners, applies them sequentially.class
NormalizeDuplicateChars
Replaces all duplicate characters with a single one.class
PassThrough
Dummy, never removes a token.class
RemoveNonWordCharTokens
Removes tokens that contain non-word characters.Fields in weka.core.tokenizers.cleaners declared as TokenCleaner Modifier and Type Field Description protected TokenCleaner[]
MultiCleaner. m_Cleaners
the cleaners to use.Methods in weka.core.tokenizers.cleaners that return TokenCleaner Modifier and Type Method Description TokenCleaner[]
MultiCleaner. getCleaners()
Returns the cleaners to use.protected TokenCleaner[]
MultiCleaner. getDefaultCleaners()
Returns the default token cleaners.Methods in weka.core.tokenizers.cleaners with parameters of type TokenCleaner Modifier and Type Method Description void
MultiCleaner. setCleaners(TokenCleaner[] value)
Sets the cleaners to use.
-