Package adams.flow.transformer
Class WekaFilter
- java.lang.Object
-
- adams.core.logging.LoggingObject
-
- adams.core.logging.CustomLoggingLevelObject
-
- adams.core.option.AbstractOptionHandler
-
- adams.flow.core.AbstractActor
-
- adams.flow.transformer.AbstractTransformer
-
- adams.flow.transformer.AbstractTransformerWithPropertiesUpdating
-
- adams.flow.transformer.WekaFilter
-
- All Implemented Interfaces:
adams.core.AdditionalInformationHandler
,adams.core.CleanUpHandler
,adams.core.Destroyable
,adams.core.GlobalInfoSupporter
,adams.core.io.ModelFileHandler
,adams.core.logging.LoggingLevelHandler
,adams.core.logging.LoggingSupporter
,adams.core.option.OptionHandler
,adams.core.QuickInfoSupporter
,adams.core.ShallowCopySupporter<adams.flow.core.Actor>
,adams.core.SizeOfHandler
,adams.core.Stoppable
,adams.core.StoppableWithFeedback
,adams.core.VariablesInspectionHandler
,adams.event.VariableChangeListener
,adams.flow.container.OptionalContainerOutput
,adams.flow.control.StorageUser
,adams.flow.core.Actor
,adams.flow.core.ErrorHandler
,adams.flow.core.InputConsumer
,adams.flow.core.ModelLoaderSupporter
,adams.flow.core.OptionalOneTimeInitializer
,adams.flow.core.OutputProducer
,adams.flow.core.PropertiesUpdater
,adams.multiprocess.JobRunnerSupporter
,Serializable
,Comparable
public class WekaFilter extends adams.flow.transformer.AbstractTransformerWithPropertiesUpdating implements adams.flow.container.OptionalContainerOutput, adams.core.io.ModelFileHandler, adams.flow.control.StorageUser, adams.flow.core.ModelLoaderSupporter, adams.flow.core.OptionalOneTimeInitializer, adams.multiprocess.JobRunnerSupporter
Filters Instances/Instance objects using the specified filter.
When re-using a trained filter, ensure that 'initializeOnce' is checked.
The following order is used to obtain the model (when using AUTO):
1. model file present?
2. source actor present?
3. storage item present?
4. use specified filter definition
Input/output:
- accepts:
weka.core.Instance
weka.core.Instances
adams.data.instance.Instance
- generates:
weka.core.Instance
weka.core.Instances
adams.data.instance.Instance
-logging-level <OFF|SEVERE|WARNING|INFO|CONFIG|FINE|FINER|FINEST> (property: loggingLevel) The logging level for outputting errors and debugging output. default: WARNING min-user-mode: Expert
-name <java.lang.String> (property: name) The name of the actor. default: WekaFilter
-annotation <adams.core.base.BaseAnnotation> (property: annotations) The annotations to attach to this actor. default:
-skip <boolean> (property: skip) If set to true, transformation is skipped and the input token is just forwarded as it is. default: false
-stop-flow-on-error <boolean> (property: stopFlowOnError) If set to true, the flow execution at this level gets stopped in case this actor encounters an error; the error gets propagated; useful for critical actors. default: false min-user-mode: Expert
-silent <boolean> (property: silent) If enabled, then no errors are output in the console; Note: the enclosing actor handler must have this enabled as well. default: false min-user-mode: Expert
-property <adams.core.base.BaseString> [-property ...] (property: properties) The properties to update with the values associated with the specified values. default:
-variable <adams.core.VariableName> [-variable ...] (property: variableNames) The names of the variables to update the properties with. default:
-filter <weka.filters.Filter> (property: filter) The filter to use for filtering the Instances/Instance objects. default: weka.filters.AllFilter
-model-loading-type <AUTO|FILE|SOURCE_ACTOR|STORAGE> (property: modelLoadingType) Determines how to load the model, in case of AUTO, first the model file is checked, then the callable actor and then the storage. default: AUTO
-model <adams.core.io.PlaceholderFile> (property: modelFile) The file to load the model from, ignored if pointing to a directory. default: ${CWD}
-source <adams.flow.core.CallableActorReference> (property: modelActor) The callable actor (source) to obtain the model from, ignored if not present. default:
-storage <adams.flow.control.StorageName> (property: modelStorage) The storage item to obtain the model from, ignored if not present. default: storage
-init-once <boolean> (property: initializeOnce) If set to true, then the filter will get initialized only with the first batch of data; otherwise every time data gets passed through; only applies when using the filter definition, the others (model file, source, storage ) assume the filter to be built. default: false
-keep <boolean> (property: keepRelationName) If set to true, then the filter won't change the relation name of the incoming dataset. default: false
-prefer-jobrunner <boolean> (property: preferJobRunner) If enabled, tries to offload the processing onto a adams.flow.standalone.JobRunnerInstance; applies only to batch filtering. default: false
-output-container <boolean> (property: outputContainer) If enabled, a adams.flow.container.WekaFilterContainer is output with the filter and the filtered data (Instance or Instances). default: false
- Author:
- fracpete (fracpete at waikato dot ac dot nz)
- See Also:
- Serialized Form
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
WekaFilter.BatchFilterJob
-
Field Summary
Fields Modifier and Type Field Description static String
BACKUP_INITIALIZED
the key for storing the current initialized state in the backup.protected weka.filters.Filter
m_ActualFilter
the actual filter used.protected weka.filters.Filter
m_Filter
the filter to apply.protected boolean
m_FlowContextUpdated
whether the flow context has been updated.protected boolean
m_Initialized
whether the filter has been initialized.protected boolean
m_InitializeOnce
whether to initialize filter only with the first batch.protected adams.flow.standalone.JobRunnerInstance
m_JobRunnerInstance
the JobRunnerInstance to use.protected boolean
m_KeepRelationName
whether to keep the incoming relation name.protected WekaFilterModelLoader
m_ModelLoader
the model loader.protected boolean
m_OutputContainer
whether to output a container.protected boolean
m_PreferJobRunner
whether to offload filtering into a JobRunnerInstance.-
Fields inherited from class adams.flow.transformer.AbstractTransformerWithPropertiesUpdating
m_Containers, m_Properties, m_VariableNames
-
Fields inherited from class adams.flow.transformer.AbstractTransformer
BACKUP_INPUT, BACKUP_OUTPUT, m_InputToken, m_OutputToken
-
Fields inherited from class adams.flow.core.AbstractActor
m_Annotations, m_BackupState, m_DetectedObjectVariables, m_DetectedVariables, m_ErrorHandler, m_Executed, m_Executing, m_ExecutionListeningSupporter, m_FullName, m_LoggingPrefix, m_Name, m_Parent, m_ScopeHandler, m_Self, m_Silent, m_Skip, m_StopFlowOnError, m_StopMessage, m_Stopped, m_StorageHandler, m_VariablesUpdated
-
-
Constructor Summary
Constructors Constructor Description WekaFilter()
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description Class[]
accepts()
Returns the class that the consumer accepts.protected Hashtable<String,Object>
backupState()
Backs up the current state of the actor before update the variables.protected adams.flow.core.Token
createToken(Object input, Object data)
Creates a token with the data.void
defineOptions()
Adds options to the internal list of options.protected String
doExecute()
Executes the flow item.String
filterTipText()
Returns the tip text for this property.Class[]
generates()
Returns the class of objects that it generates.weka.filters.Filter
getFilter()
Returns the filter in use.boolean
getInitializeOnce()
Returns whether the filter gets initialized only with the first batch.boolean
getKeepRelationName()
Returns whether the filter doesn't change the relation name.adams.flow.core.CallableActorReference
getModelActor()
Returns the filter source actor.adams.core.io.PlaceholderFile
getModelFile()
Returns the file to load the model from.adams.flow.core.AbstractModelLoader.ModelLoadingType
getModelLoadingType()
Returns the loading type.adams.flow.control.StorageName
getModelStorage()
Returns the filter storage item.boolean
getOutputContainer()
Returns whether to output a container with the filter alongside the filtered data or just the filtered data.boolean
getPreferJobRunner()
Returns whether to offload processing to a JobRunner instance if available.String
getQuickInfo()
Returns a quick info about the actor, which will be displayed in the GUI.String
globalInfo()
Returns a string describing the object.protected String
initActualFilter(weka.core.Instances data)
Initializes the actual filter to use.protected void
initialize()
Initializes the members.String
initializeOnceTipText()
Returns the tip text for this property.boolean
isUsingStorage()
Returns whether storage items are being used.String
keepRelationNameTipText()
Returns the tip text for this property.String
modelActorTipText()
Returns the tip text for this property.String
modelFileTipText()
Returns the tip text for this property.String
modelLoadingTypeTipText()
Returns the tip text for this property.String
modelStorageTipText()
Returns the tip text for this property.String
outputContainerTipText()
Returns the tip text for this property.String
preferJobRunnerTipText()
Returns the tip text for this property.protected void
pruneBackup()
Removes entries from the backup.protected void
reset()
Resets the scheme.protected void
restoreState(Hashtable<String,Object> state)
Restores the state of the actor before the variables got updated.void
setFilter(weka.filters.Filter value)
Sets the filter to use.void
setInitializeOnce(boolean value)
Sets whether the filter gets initialized only with the first batch.void
setKeepRelationName(boolean value)
Sets whether the filter doesn't change the relation name.void
setLoggingLevel(adams.core.logging.LoggingLevel value)
Sets the logging level.void
setModelActor(adams.flow.core.CallableActorReference value)
Sets the filter source actor.void
setModelFile(adams.core.io.PlaceholderFile value)
Sets the file to load the model from.void
setModelLoadingType(adams.flow.core.AbstractModelLoader.ModelLoadingType value)
Sets the loading type.void
setModelStorage(adams.flow.control.StorageName value)
Sets the filter storage item.void
setOutputContainer(boolean value)
Sets whether to output a container with the filter alongside the filtered data or just the filtered data.void
setPreferJobRunner(boolean value)
Sets whether to offload processing to a JobRunner instance if available.String
setUp()
Initializes the item for flow execution.-
Methods inherited from class adams.flow.transformer.AbstractTransformerWithPropertiesUpdating
getProperties, getVariableNames, propertiesTipText, setProperties, setUpContainers, setUpContainersIfNecessary, setVariableNames, updateObject, variableNamesTipText
-
Methods inherited from class adams.flow.transformer.AbstractTransformer
currentInput, execute, hasInput, hasPendingOutput, input, output, postExecute, wrapUp
-
Methods inherited from class adams.flow.core.AbstractActor
annotationsTipText, canInspectOptions, canPerformSetUpCheck, cleanUp, compareTo, configureLogger, destroy, equals, finalUpdateVariables, findVariables, findVariables, forceVariables, forCommandLine, forName, forName, getAdditionalInformation, getAnnotations, getDefaultName, getDetectedVariables, getErrorHandler, getFlowActors, getFlowExecutionListeningSupporter, getFullName, getName, getNextSibling, getParent, getParentComponent, getPreviousSibling, getRoot, getScopeHandler, getSilent, getSkip, getStopFlowOnError, getStopMessage, getStorageHandler, getVariables, handleError, handleException, hasErrorHandler, hasStopMessage, index, isBackedUp, isExecuted, isExecuting, isFinished, isHeadless, isStopped, nameTipText, performSetUpChecks, performVariableChecks, preExecute, pruneBackup, setAnnotations, setErrorHandler, setName, setParent, setSilent, setSkip, setStopFlowOnError, setVariables, shallowCopy, shallowCopy, silentTipText, sizeOf, skipTipText, stopExecution, stopExecution, stopFlowOnErrorTipText, updateDetectedVariables, updatePrefix, updateVariables, variableChanged
-
Methods inherited from class adams.core.option.AbstractOptionHandler
cleanUpOptions, finishInit, getDefaultLoggingLevel, getOptionManager, loggingLevelTipText, newOptionManager, toCommandLine, toString
-
Methods inherited from class adams.core.logging.LoggingObject
getLogger, getLoggingLevel, initializeLogging, isLoggingEnabled
-
Methods inherited from class java.lang.Object
clone, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
-
Methods inherited from interface adams.flow.core.Actor
cleanUp, compareTo, destroy, equals, execute, findVariables, getAnnotations, getDefaultName, getDetectedVariables, getErrorHandler, getFlowExecutionListeningSupporter, getFullName, getName, getNextSibling, getParent, getParentComponent, getPreviousSibling, getRoot, getScopeHandler, getSilent, getSkip, getStopFlowOnError, getStopMessage, getStorageHandler, getVariables, handleError, hasErrorHandler, hasStopMessage, index, isExecuted, isFinished, isHeadless, isStopped, setAnnotations, setErrorHandler, setName, setParent, setSilent, setSkip, setStopFlowOnError, setVariables, shallowCopy, shallowCopy, sizeOf, stopExecution, stopExecution, toCommandLine, variableChanged, wrapUp
-
-
-
-
Field Detail
-
BACKUP_INITIALIZED
public static final String BACKUP_INITIALIZED
the key for storing the current initialized state in the backup.- See Also:
- Constant Field Values
-
m_Filter
protected weka.filters.Filter m_Filter
the filter to apply.
-
m_ActualFilter
protected weka.filters.Filter m_ActualFilter
the actual filter used.
-
m_InitializeOnce
protected boolean m_InitializeOnce
whether to initialize filter only with the first batch.
-
m_KeepRelationName
protected boolean m_KeepRelationName
whether to keep the incoming relation name.
-
m_OutputContainer
protected boolean m_OutputContainer
whether to output a container.
-
m_Initialized
protected boolean m_Initialized
whether the filter has been initialized.
-
m_FlowContextUpdated
protected boolean m_FlowContextUpdated
whether the flow context has been updated.
-
m_ModelLoader
protected WekaFilterModelLoader m_ModelLoader
the model loader.
-
m_PreferJobRunner
protected boolean m_PreferJobRunner
whether to offload filtering into a JobRunnerInstance.
-
m_JobRunnerInstance
protected transient adams.flow.standalone.JobRunnerInstance m_JobRunnerInstance
the JobRunnerInstance to use.
-
-
Method Detail
-
globalInfo
public String globalInfo()
Returns a string describing the object.- Specified by:
globalInfo
in interfaceadams.core.GlobalInfoSupporter
- Specified by:
globalInfo
in classadams.core.option.AbstractOptionHandler
- Returns:
- a description suitable for displaying in the gui
-
defineOptions
public void defineOptions()
Adds options to the internal list of options.- Specified by:
defineOptions
in interfaceadams.core.option.OptionHandler
- Overrides:
defineOptions
in classadams.flow.transformer.AbstractTransformerWithPropertiesUpdating
-
initialize
protected void initialize()
Initializes the members.- Overrides:
initialize
in classadams.flow.core.AbstractActor
-
setLoggingLevel
public void setLoggingLevel(adams.core.logging.LoggingLevel value)
Sets the logging level.- Specified by:
setLoggingLevel
in interfaceadams.core.logging.LoggingLevelHandler
- Overrides:
setLoggingLevel
in classadams.core.option.AbstractOptionHandler
- Parameters:
value
- the level
-
setFilter
public void setFilter(weka.filters.Filter value)
Sets the filter to use.- Parameters:
value
- the filter
-
getFilter
public weka.filters.Filter getFilter()
Returns the filter in use.- Returns:
- the filter
-
filterTipText
public String filterTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setModelLoadingType
public void setModelLoadingType(adams.flow.core.AbstractModelLoader.ModelLoadingType value)
Sets the loading type. In case ofAbstractModelLoader.ModelLoadingType.AUTO
, first file, then callable actor, then storage.- Specified by:
setModelLoadingType
in interfaceadams.flow.core.ModelLoaderSupporter
- Parameters:
value
- the type
-
getModelLoadingType
public adams.flow.core.AbstractModelLoader.ModelLoadingType getModelLoadingType()
Returns the loading type. In case ofAbstractModelLoader.ModelLoadingType.AUTO
, first file, then callable actor, then storage.- Specified by:
getModelLoadingType
in interfaceadams.flow.core.ModelLoaderSupporter
- Returns:
- the type
-
modelLoadingTypeTipText
public String modelLoadingTypeTipText()
Returns the tip text for this property.- Specified by:
modelLoadingTypeTipText
in interfaceadams.flow.core.ModelLoaderSupporter
- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setModelFile
public void setModelFile(adams.core.io.PlaceholderFile value)
Sets the file to load the model from.- Specified by:
setModelFile
in interfaceadams.core.io.ModelFileHandler
- Specified by:
setModelFile
in interfaceadams.flow.core.ModelLoaderSupporter
- Parameters:
value
- the model file
-
getModelFile
public adams.core.io.PlaceholderFile getModelFile()
Returns the file to load the model from.- Specified by:
getModelFile
in interfaceadams.core.io.ModelFileHandler
- Specified by:
getModelFile
in interfaceadams.flow.core.ModelLoaderSupporter
- Returns:
- the model file
-
modelFileTipText
public String modelFileTipText()
Returns the tip text for this property.- Specified by:
modelFileTipText
in interfaceadams.core.io.ModelFileHandler
- Specified by:
modelFileTipText
in interfaceadams.flow.core.ModelLoaderSupporter
- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setModelActor
public void setModelActor(adams.flow.core.CallableActorReference value)
Sets the filter source actor.- Specified by:
setModelActor
in interfaceadams.flow.core.ModelLoaderSupporter
- Parameters:
value
- the source
-
getModelActor
public adams.flow.core.CallableActorReference getModelActor()
Returns the filter source actor.- Specified by:
getModelActor
in interfaceadams.flow.core.ModelLoaderSupporter
- Returns:
- the source
-
modelActorTipText
public String modelActorTipText()
Returns the tip text for this property.- Specified by:
modelActorTipText
in interfaceadams.flow.core.ModelLoaderSupporter
- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setModelStorage
public void setModelStorage(adams.flow.control.StorageName value)
Sets the filter storage item.- Specified by:
setModelStorage
in interfaceadams.flow.core.ModelLoaderSupporter
- Parameters:
value
- the storage item
-
getModelStorage
public adams.flow.control.StorageName getModelStorage()
Returns the filter storage item.- Specified by:
getModelStorage
in interfaceadams.flow.core.ModelLoaderSupporter
- Returns:
- the storage item
-
modelStorageTipText
public String modelStorageTipText()
Returns the tip text for this property.- Specified by:
modelStorageTipText
in interfaceadams.flow.core.ModelLoaderSupporter
- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setInitializeOnce
public void setInitializeOnce(boolean value)
Sets whether the filter gets initialized only with the first batch.- Specified by:
setInitializeOnce
in interfaceadams.flow.core.OptionalOneTimeInitializer
- Parameters:
value
- true if the filter gets only initialized once
-
getInitializeOnce
public boolean getInitializeOnce()
Returns whether the filter gets initialized only with the first batch.- Specified by:
getInitializeOnce
in interfaceadams.flow.core.OptionalOneTimeInitializer
- Returns:
- true if the filter gets only initialized once
-
initializeOnceTipText
public String initializeOnceTipText()
Returns the tip text for this property.- Specified by:
initializeOnceTipText
in interfaceadams.flow.core.OptionalOneTimeInitializer
- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setKeepRelationName
public void setKeepRelationName(boolean value)
Sets whether the filter doesn't change the relation name.- Parameters:
value
- true if the filter won't change the relation name
-
getKeepRelationName
public boolean getKeepRelationName()
Returns whether the filter doesn't change the relation name.- Returns:
- true if the filter doesn't change the relation name
-
keepRelationNameTipText
public String keepRelationNameTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setPreferJobRunner
public void setPreferJobRunner(boolean value)
Sets whether to offload processing to a JobRunner instance if available.- Specified by:
setPreferJobRunner
in interfaceadams.multiprocess.JobRunnerSupporter
- Parameters:
value
- if true try to find/use a JobRunner instance
-
getPreferJobRunner
public boolean getPreferJobRunner()
Returns whether to offload processing to a JobRunner instance if available.- Specified by:
getPreferJobRunner
in interfaceadams.multiprocess.JobRunnerSupporter
- Returns:
- if true try to find/use a JobRunner instance
-
preferJobRunnerTipText
public String preferJobRunnerTipText()
Returns the tip text for this property.- Specified by:
preferJobRunnerTipText
in interfaceadams.multiprocess.JobRunnerSupporter
- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setOutputContainer
public void setOutputContainer(boolean value)
Sets whether to output a container with the filter alongside the filtered data or just the filtered data.- Specified by:
setOutputContainer
in interfaceadams.flow.container.OptionalContainerOutput
- Parameters:
value
- true if to output the container
-
getOutputContainer
public boolean getOutputContainer()
Returns whether to output a container with the filter alongside the filtered data or just the filtered data.- Specified by:
getOutputContainer
in interfaceadams.flow.container.OptionalContainerOutput
- Returns:
- true if to output the container
-
outputContainerTipText
public String outputContainerTipText()
Returns the tip text for this property.- Specified by:
outputContainerTipText
in interfaceadams.flow.container.OptionalContainerOutput
- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
isUsingStorage
public boolean isUsingStorage()
Returns whether storage items are being used.- Specified by:
isUsingStorage
in interfaceadams.flow.control.StorageUser
- Returns:
- true if storage items are used
-
generates
public Class[] generates()
Returns the class of objects that it generates.- Specified by:
generates
in interfaceadams.flow.core.OutputProducer
- Returns:
- weka.core.Instance, weka.core.Instances, adams.data.instance.Instance
-
getQuickInfo
public String getQuickInfo()
Returns a quick info about the actor, which will be displayed in the GUI.- Specified by:
getQuickInfo
in interfaceadams.flow.core.Actor
- Specified by:
getQuickInfo
in interfaceadams.core.QuickInfoSupporter
- Overrides:
getQuickInfo
in classadams.flow.transformer.AbstractTransformerWithPropertiesUpdating
- Returns:
- null if no info available, otherwise short string
-
pruneBackup
protected void pruneBackup()
Removes entries from the backup.- Overrides:
pruneBackup
in classadams.flow.core.AbstractActor
-
backupState
protected Hashtable<String,Object> backupState()
Backs up the current state of the actor before update the variables.- Overrides:
backupState
in classadams.flow.transformer.AbstractTransformer
- Returns:
- the backup
-
restoreState
protected void restoreState(Hashtable<String,Object> state)
Restores the state of the actor before the variables got updated.- Overrides:
restoreState
in classadams.flow.transformer.AbstractTransformer
- Parameters:
state
- the backup of the state to restore from
-
reset
protected void reset()
Resets the scheme.- Overrides:
reset
in classadams.flow.transformer.AbstractTransformerWithPropertiesUpdating
-
accepts
public Class[] accepts()
Returns the class that the consumer accepts.- Specified by:
accepts
in interfaceadams.flow.core.InputConsumer
- Returns:
- weka.core.Instance, weka.core.Instances, adams.data.instance.Instance
-
createToken
protected adams.flow.core.Token createToken(Object input, Object data)
Creates a token with the data. If required creates a container with the filter.- Parameters:
data
- the data to output in the token- Returns:
- the generated token
-
initActualFilter
protected String initActualFilter(weka.core.Instances data) throws Exception
Initializes the actual filter to use.- Parameters:
data
- the data to initialize the filter when using the filter definition- Returns:
- null if successful, otherwise error message
- Throws:
Exception
- if initialization, copying etc fails
-
setUp
public String setUp()
Initializes the item for flow execution.- Specified by:
setUp
in interfaceadams.flow.core.Actor
- Overrides:
setUp
in classadams.flow.core.AbstractActor
- Returns:
- null if everything is fine, otherwise error message
-
doExecute
protected String doExecute()
Executes the flow item.- Specified by:
doExecute
in classadams.flow.core.AbstractActor
- Returns:
- null if everything is fine, otherwise error message
-
-