Package adams.flow.transformer
Class WekaFilter
-
- All Implemented Interfaces:
AdditionalInformationHandler,CleanUpHandler,Destroyable,GlobalInfoSupporter,ModelFileHandler,LoggingLevelHandler,LoggingSupporter,OptionHandler,QuickInfoSupporter,ShallowCopySupporter<Actor>,SizeOfHandler,Stoppable,StoppableWithFeedback,VariablesInspectionHandler,VariableChangeListener,OptionalContainerOutput,StorageUser,Actor,ErrorHandler,InputConsumer,ModelLoaderSupporter,OptionalOneTimeInitializer,OutputProducer,PropertiesUpdater,JobRunnerSupporter,Serializable,Comparable
public class WekaFilter extends AbstractTransformerWithPropertiesUpdating implements OptionalContainerOutput, ModelFileHandler, StorageUser, ModelLoaderSupporter, OptionalOneTimeInitializer, JobRunnerSupporter
Filters Instances/Instance objects using the specified filter.
When re-using a trained filter, ensure that 'initializeOnce' is checked.
The following order is used to obtain the model (when using AUTO):
1. model file present?
2. source actor present?
3. storage item present?
4. use specified filter definition
Input/output:
- accepts:
weka.core.Instance
weka.core.Instances
adams.data.instance.Instance
- generates:
weka.core.Instance
weka.core.Instances
adams.data.instance.Instance
-logging-level <OFF|SEVERE|WARNING|INFO|CONFIG|FINE|FINER|FINEST> (property: loggingLevel) The logging level for outputting errors and debugging output. default: WARNING min-user-mode: Expert
-name <java.lang.String> (property: name) The name of the actor. default: WekaFilter
-annotation <adams.core.base.BaseAnnotation> (property: annotations) The annotations to attach to this actor. default:
-skip <boolean> (property: skip) If set to true, transformation is skipped and the input token is just forwarded as it is. default: false
-stop-flow-on-error <boolean> (property: stopFlowOnError) If set to true, the flow execution at this level gets stopped in case this actor encounters an error; the error gets propagated; useful for critical actors. default: false min-user-mode: Expert
-silent <boolean> (property: silent) If enabled, then no errors are output in the console; Note: the enclosing actor handler must have this enabled as well. default: false min-user-mode: Expert
-property <adams.core.base.BaseString> [-property ...] (property: properties) The properties to update with the values associated with the specified values. default:
-variable <adams.core.VariableName> [-variable ...] (property: variableNames) The names of the variables to update the properties with. default:
-filter <weka.filters.Filter> (property: filter) The filter to use for filtering the Instances/Instance objects. default: weka.filters.AllFilter
-model-loading-type <AUTO|FILE|SOURCE_ACTOR|STORAGE> (property: modelLoadingType) Determines how to load the model, in case of AUTO, first the model file is checked, then the callable actor and then the storage. default: AUTO
-model <adams.core.io.PlaceholderFile> (property: modelFile) The file to load the model from, ignored if pointing to a directory. default: ${CWD}-source <adams.flow.core.CallableActorReference> (property: modelActor) The callable actor (source) to obtain the model from, ignored if not present. default:
-storage <adams.flow.control.StorageName> (property: modelStorage) The storage item to obtain the model from, ignored if not present. default: storage
-init-once <boolean> (property: initializeOnce) If set to true, then the filter will get initialized only with the first batch of data; otherwise every time data gets passed through; only applies when using the filter definition, the others (model file, source, storage ) assume the filter to be built. default: false
-keep <boolean> (property: keepRelationName) If set to true, then the filter won't change the relation name of the incoming dataset. default: false
-prefer-jobrunner <boolean> (property: preferJobRunner) If enabled, tries to offload the processing onto a adams.flow.standalone.JobRunnerInstance; applies only to batch filtering. default: false
-output-container <boolean> (property: outputContainer) If enabled, a adams.flow.container.WekaFilterContainer is output with the filter and the filtered data (Instance or Instances). default: false
- Author:
- fracpete (fracpete at waikato dot ac dot nz)
- See Also:
- Serialized Form
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static classWekaFilter.BatchFilterJob
-
Field Summary
Fields Modifier and Type Field Description static StringBACKUP_INITIALIZEDthe key for storing the current initialized state in the backup.protected weka.filters.Filterm_ActualFilterthe actual filter used.protected weka.filters.Filterm_Filterthe filter to apply.protected booleanm_FlowContextUpdatedwhether the flow context has been updated.protected booleanm_Initializedwhether the filter has been initialized.protected booleanm_InitializeOncewhether to initialize filter only with the first batch.protected JobRunnerInstancem_JobRunnerInstancethe JobRunnerInstance to use.protected booleanm_KeepRelationNamewhether to keep the incoming relation name.protected WekaFilterModelLoaderm_ModelLoaderthe model loader.protected booleanm_OutputContainerwhether to output a container.protected booleanm_PreferJobRunnerwhether to offload filtering into a JobRunnerInstance.-
Fields inherited from class adams.flow.transformer.AbstractTransformerWithPropertiesUpdating
m_Containers, m_Properties, m_VariableNames
-
Fields inherited from class adams.flow.transformer.AbstractTransformer
BACKUP_INPUT, BACKUP_OUTPUT, m_InputToken, m_OutputToken
-
Fields inherited from class adams.flow.core.AbstractActor
m_Annotations, m_BackupState, m_DetectedObjectVariables, m_DetectedVariables, m_ErrorHandler, m_Executed, m_Executing, m_ExecutionListeningSupporter, m_FullName, m_LoggingPrefix, m_Name, m_Parent, m_ScopeHandler, m_Self, m_Silent, m_Skip, m_StopFlowOnError, m_StopMessage, m_Stopped, m_StorageHandler, m_VariablesUpdated
-
Fields inherited from class adams.core.option.AbstractOptionHandler
m_OptionManager
-
Fields inherited from class adams.core.logging.LoggingObject
m_Logger, m_LoggingIsEnabled, m_LoggingLevel
-
Fields inherited from interface adams.flow.core.Actor
FILE_EXTENSION, FILE_EXTENSION_GZ
-
-
Constructor Summary
Constructors Constructor Description WekaFilter()
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description Class[]accepts()Returns the class that the consumer accepts.protected Hashtable<String,Object>backupState()Backs up the current state of the actor before update the variables.protected TokencreateToken(Object input, Object data)Creates a token with the data.voiddefineOptions()Adds options to the internal list of options.protected StringdoExecute()Executes the flow item.StringfilterTipText()Returns the tip text for this property.Class[]generates()Returns the class of objects that it generates.weka.filters.FiltergetFilter()Returns the filter in use.booleangetInitializeOnce()Returns whether the filter gets initialized only with the first batch.booleangetKeepRelationName()Returns whether the filter doesn't change the relation name.CallableActorReferencegetModelActor()Returns the filter source actor.PlaceholderFilegetModelFile()Returns the file to load the model from.AbstractModelLoader.ModelLoadingTypegetModelLoadingType()Returns the loading type.StorageNamegetModelStorage()Returns the filter storage item.booleangetOutputContainer()Returns whether to output a container with the filter alongside the filtered data or just the filtered data.booleangetPreferJobRunner()Returns whether to offload processing to a JobRunner instance if available.StringgetQuickInfo()Returns a quick info about the actor, which will be displayed in the GUI.StringglobalInfo()Returns a string describing the object.protected StringinitActualFilter(weka.core.Instances data)Initializes the actual filter to use.protected voidinitialize()Initializes the members.StringinitializeOnceTipText()Returns the tip text for this property.booleanisUsingStorage()Returns whether storage items are being used.StringkeepRelationNameTipText()Returns the tip text for this property.StringmodelActorTipText()Returns the tip text for this property.StringmodelFileTipText()Returns the tip text for this property.StringmodelLoadingTypeTipText()Returns the tip text for this property.StringmodelStorageTipText()Returns the tip text for this property.StringoutputContainerTipText()Returns the tip text for this property.StringpreferJobRunnerTipText()Returns the tip text for this property.protected voidpruneBackup()Removes entries from the backup.protected voidreset()Resets the scheme.protected voidrestoreState(Hashtable<String,Object> state)Restores the state of the actor before the variables got updated.voidsetFilter(weka.filters.Filter value)Sets the filter to use.voidsetInitializeOnce(boolean value)Sets whether the filter gets initialized only with the first batch.voidsetKeepRelationName(boolean value)Sets whether the filter doesn't change the relation name.voidsetLoggingLevel(LoggingLevel value)Sets the logging level.voidsetModelActor(CallableActorReference value)Sets the filter source actor.voidsetModelFile(PlaceholderFile value)Sets the file to load the model from.voidsetModelLoadingType(AbstractModelLoader.ModelLoadingType value)Sets the loading type.voidsetModelStorage(StorageName value)Sets the filter storage item.voidsetOutputContainer(boolean value)Sets whether to output a container with the filter alongside the filtered data or just the filtered data.voidsetPreferJobRunner(boolean value)Sets whether to offload processing to a JobRunner instance if available.StringsetUp()Initializes the item for flow execution.-
Methods inherited from class adams.flow.transformer.AbstractTransformerWithPropertiesUpdating
getProperties, getVariableNames, propertiesTipText, setProperties, setUpContainers, setUpContainersIfNecessary, setVariableNames, updateObject, variableNamesTipText
-
Methods inherited from class adams.flow.transformer.AbstractTransformer
currentInput, execute, hasInput, hasPendingOutput, input, output, postExecute, wrapUp
-
Methods inherited from class adams.flow.core.AbstractActor
annotationsTipText, canInspectOptions, canPerformSetUpCheck, cleanUp, compareTo, configureLogger, destroy, equals, finalUpdateVariables, findVariables, findVariables, forceVariables, forCommandLine, forName, forName, getAdditionalInformation, getAnnotations, getDefaultName, getDetectedVariables, getErrorHandler, getFlowActors, getFlowExecutionListeningSupporter, getFullName, getName, getNextSibling, getParent, getParentComponent, getPreviousSibling, getRoot, getScopeHandler, getSilent, getSkip, getStopFlowOnError, getStopMessage, getStorageHandler, getVariables, handleError, handleException, hasErrorHandler, hasStopMessage, index, isBackedUp, isExecuted, isExecuting, isFinished, isHeadless, isStopped, nameTipText, performSetUpChecks, performVariableChecks, preExecute, pruneBackup, setAnnotations, setErrorHandler, setName, setParent, setSilent, setSkip, setStopFlowOnError, setVariables, shallowCopy, shallowCopy, silentTipText, sizeOf, skipTipText, stopExecution, stopExecution, stopFlowOnErrorTipText, updateDetectedVariables, updatePrefix, updateVariables, variableChanged
-
Methods inherited from class adams.core.option.AbstractOptionHandler
cleanUpOptions, finishInit, getDefaultLoggingLevel, getOptionManager, loggingLevelTipText, newOptionManager, toCommandLine, toString
-
Methods inherited from class adams.core.logging.LoggingObject
getLogger, getLoggingLevel, initializeLogging, isLoggingEnabled
-
Methods inherited from class java.lang.Object
clone, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
-
Methods inherited from interface adams.flow.core.Actor
cleanUp, compareTo, destroy, equals, execute, findVariables, getAnnotations, getDefaultName, getDetectedVariables, getErrorHandler, getFlowExecutionListeningSupporter, getFullName, getName, getNextSibling, getParent, getParentComponent, getPreviousSibling, getRoot, getScopeHandler, getSilent, getSkip, getStopFlowOnError, getStopMessage, getStorageHandler, getVariables, handleError, hasErrorHandler, hasStopMessage, index, isExecuted, isFinished, isHeadless, isStopped, setAnnotations, setErrorHandler, setName, setParent, setSilent, setSkip, setStopFlowOnError, setVariables, shallowCopy, shallowCopy, sizeOf, stopExecution, stopExecution, toCommandLine, variableChanged, wrapUp
-
Methods inherited from interface adams.core.AdditionalInformationHandler
getAdditionalInformation
-
Methods inherited from interface adams.core.logging.LoggingLevelHandler
getLoggingLevel
-
Methods inherited from interface adams.core.logging.LoggingSupporter
getLogger, isLoggingEnabled
-
Methods inherited from interface adams.core.option.OptionHandler
cleanUpOptions, getOptionManager
-
Methods inherited from interface adams.core.VariablesInspectionHandler
canInspectOptions
-
-
-
-
Field Detail
-
BACKUP_INITIALIZED
public static final String BACKUP_INITIALIZED
the key for storing the current initialized state in the backup.- See Also:
- Constant Field Values
-
m_Filter
protected weka.filters.Filter m_Filter
the filter to apply.
-
m_ActualFilter
protected weka.filters.Filter m_ActualFilter
the actual filter used.
-
m_InitializeOnce
protected boolean m_InitializeOnce
whether to initialize filter only with the first batch.
-
m_KeepRelationName
protected boolean m_KeepRelationName
whether to keep the incoming relation name.
-
m_OutputContainer
protected boolean m_OutputContainer
whether to output a container.
-
m_Initialized
protected boolean m_Initialized
whether the filter has been initialized.
-
m_FlowContextUpdated
protected boolean m_FlowContextUpdated
whether the flow context has been updated.
-
m_ModelLoader
protected WekaFilterModelLoader m_ModelLoader
the model loader.
-
m_PreferJobRunner
protected boolean m_PreferJobRunner
whether to offload filtering into a JobRunnerInstance.
-
m_JobRunnerInstance
protected transient JobRunnerInstance m_JobRunnerInstance
the JobRunnerInstance to use.
-
-
Method Detail
-
globalInfo
public String globalInfo()
Returns a string describing the object.- Specified by:
globalInfoin interfaceGlobalInfoSupporter- Specified by:
globalInfoin classAbstractOptionHandler- Returns:
- a description suitable for displaying in the gui
-
defineOptions
public void defineOptions()
Adds options to the internal list of options.- Specified by:
defineOptionsin interfaceOptionHandler- Overrides:
defineOptionsin classAbstractTransformerWithPropertiesUpdating
-
initialize
protected void initialize()
Initializes the members.- Overrides:
initializein classAbstractActor
-
setLoggingLevel
public void setLoggingLevel(LoggingLevel value)
Sets the logging level.- Specified by:
setLoggingLevelin interfaceLoggingLevelHandler- Overrides:
setLoggingLevelin classAbstractOptionHandler- Parameters:
value- the level
-
setFilter
public void setFilter(weka.filters.Filter value)
Sets the filter to use.- Parameters:
value- the filter
-
getFilter
public weka.filters.Filter getFilter()
Returns the filter in use.- Returns:
- the filter
-
filterTipText
public String filterTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setModelLoadingType
public void setModelLoadingType(AbstractModelLoader.ModelLoadingType value)
Sets the loading type. In case ofAbstractModelLoader.ModelLoadingType.AUTO, first file, then callable actor, then storage.- Specified by:
setModelLoadingTypein interfaceModelLoaderSupporter- Parameters:
value- the type
-
getModelLoadingType
public AbstractModelLoader.ModelLoadingType getModelLoadingType()
Returns the loading type. In case ofAbstractModelLoader.ModelLoadingType.AUTO, first file, then callable actor, then storage.- Specified by:
getModelLoadingTypein interfaceModelLoaderSupporter- Returns:
- the type
-
modelLoadingTypeTipText
public String modelLoadingTypeTipText()
Returns the tip text for this property.- Specified by:
modelLoadingTypeTipTextin interfaceModelLoaderSupporter- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setModelFile
public void setModelFile(PlaceholderFile value)
Sets the file to load the model from.- Specified by:
setModelFilein interfaceModelFileHandler- Specified by:
setModelFilein interfaceModelLoaderSupporter- Parameters:
value- the model file
-
getModelFile
public PlaceholderFile getModelFile()
Returns the file to load the model from.- Specified by:
getModelFilein interfaceModelFileHandler- Specified by:
getModelFilein interfaceModelLoaderSupporter- Returns:
- the model file
-
modelFileTipText
public String modelFileTipText()
Returns the tip text for this property.- Specified by:
modelFileTipTextin interfaceModelFileHandler- Specified by:
modelFileTipTextin interfaceModelLoaderSupporter- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setModelActor
public void setModelActor(CallableActorReference value)
Sets the filter source actor.- Specified by:
setModelActorin interfaceModelLoaderSupporter- Parameters:
value- the source
-
getModelActor
public CallableActorReference getModelActor()
Returns the filter source actor.- Specified by:
getModelActorin interfaceModelLoaderSupporter- Returns:
- the source
-
modelActorTipText
public String modelActorTipText()
Returns the tip text for this property.- Specified by:
modelActorTipTextin interfaceModelLoaderSupporter- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setModelStorage
public void setModelStorage(StorageName value)
Sets the filter storage item.- Specified by:
setModelStoragein interfaceModelLoaderSupporter- Parameters:
value- the storage item
-
getModelStorage
public StorageName getModelStorage()
Returns the filter storage item.- Specified by:
getModelStoragein interfaceModelLoaderSupporter- Returns:
- the storage item
-
modelStorageTipText
public String modelStorageTipText()
Returns the tip text for this property.- Specified by:
modelStorageTipTextin interfaceModelLoaderSupporter- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setInitializeOnce
public void setInitializeOnce(boolean value)
Sets whether the filter gets initialized only with the first batch.- Specified by:
setInitializeOncein interfaceOptionalOneTimeInitializer- Parameters:
value- true if the filter gets only initialized once
-
getInitializeOnce
public boolean getInitializeOnce()
Returns whether the filter gets initialized only with the first batch.- Specified by:
getInitializeOncein interfaceOptionalOneTimeInitializer- Returns:
- true if the filter gets only initialized once
-
initializeOnceTipText
public String initializeOnceTipText()
Returns the tip text for this property.- Specified by:
initializeOnceTipTextin interfaceOptionalOneTimeInitializer- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setKeepRelationName
public void setKeepRelationName(boolean value)
Sets whether the filter doesn't change the relation name.- Parameters:
value- true if the filter won't change the relation name
-
getKeepRelationName
public boolean getKeepRelationName()
Returns whether the filter doesn't change the relation name.- Returns:
- true if the filter doesn't change the relation name
-
keepRelationNameTipText
public String keepRelationNameTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setPreferJobRunner
public void setPreferJobRunner(boolean value)
Sets whether to offload processing to a JobRunner instance if available.- Specified by:
setPreferJobRunnerin interfaceJobRunnerSupporter- Parameters:
value- if true try to find/use a JobRunner instance
-
getPreferJobRunner
public boolean getPreferJobRunner()
Returns whether to offload processing to a JobRunner instance if available.- Specified by:
getPreferJobRunnerin interfaceJobRunnerSupporter- Returns:
- if true try to find/use a JobRunner instance
-
preferJobRunnerTipText
public String preferJobRunnerTipText()
Returns the tip text for this property.- Specified by:
preferJobRunnerTipTextin interfaceJobRunnerSupporter- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setOutputContainer
public void setOutputContainer(boolean value)
Sets whether to output a container with the filter alongside the filtered data or just the filtered data.- Specified by:
setOutputContainerin interfaceOptionalContainerOutput- Parameters:
value- true if to output the container
-
getOutputContainer
public boolean getOutputContainer()
Returns whether to output a container with the filter alongside the filtered data or just the filtered data.- Specified by:
getOutputContainerin interfaceOptionalContainerOutput- Returns:
- true if to output the container
-
outputContainerTipText
public String outputContainerTipText()
Returns the tip text for this property.- Specified by:
outputContainerTipTextin interfaceOptionalContainerOutput- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
isUsingStorage
public boolean isUsingStorage()
Returns whether storage items are being used.- Specified by:
isUsingStoragein interfaceStorageUser- Returns:
- true if storage items are used
-
generates
public Class[] generates()
Returns the class of objects that it generates.- Specified by:
generatesin interfaceOutputProducer- Returns:
- weka.core.Instance, weka.core.Instances, adams.data.instance.Instance
-
getQuickInfo
public String getQuickInfo()
Returns a quick info about the actor, which will be displayed in the GUI.- Specified by:
getQuickInfoin interfaceActor- Specified by:
getQuickInfoin interfaceQuickInfoSupporter- Overrides:
getQuickInfoin classAbstractTransformerWithPropertiesUpdating- Returns:
- null if no info available, otherwise short string
-
pruneBackup
protected void pruneBackup()
Removes entries from the backup.- Overrides:
pruneBackupin classAbstractActor- See Also:
AbstractActor.reset()
-
backupState
protected Hashtable<String,Object> backupState()
Backs up the current state of the actor before update the variables.- Overrides:
backupStatein classAbstractTransformer- Returns:
- the backup
- See Also:
AbstractActor.updateVariables(),AbstractActor.restoreState(Hashtable)
-
restoreState
protected void restoreState(Hashtable<String,Object> state)
Restores the state of the actor before the variables got updated.- Overrides:
restoreStatein classAbstractTransformer- Parameters:
state- the backup of the state to restore from- See Also:
AbstractActor.updateVariables(),AbstractActor.backupState()
-
reset
protected void reset()
Resets the scheme.- Overrides:
resetin classAbstractTransformerWithPropertiesUpdating
-
accepts
public Class[] accepts()
Returns the class that the consumer accepts.- Specified by:
acceptsin interfaceInputConsumer- Returns:
- weka.core.Instance, weka.core.Instances, adams.data.instance.Instance
-
createToken
protected Token createToken(Object input, Object data)
Creates a token with the data. If required creates a container with the filter.- Parameters:
data- the data to output in the token- Returns:
- the generated token
-
initActualFilter
protected String initActualFilter(weka.core.Instances data) throws Exception
Initializes the actual filter to use.- Parameters:
data- the data to initialize the filter when using the filter definition- Returns:
- null if successful, otherwise error message
- Throws:
Exception- if initialization, copying etc fails
-
setUp
public String setUp()
Initializes the item for flow execution.- Specified by:
setUpin interfaceActor- Overrides:
setUpin classAbstractActor- Returns:
- null if everything is fine, otherwise error message
- See Also:
AbstractActor.reset()
-
doExecute
protected String doExecute()
Executes the flow item.- Specified by:
doExecutein classAbstractActor- Returns:
- null if everything is fine, otherwise error message
-
-