Package adams.flow.transformer
Class WekaFilter
- java.lang.Object
-
- adams.core.logging.LoggingObject
-
- adams.core.logging.CustomLoggingLevelObject
-
- adams.core.option.AbstractOptionHandler
-
- adams.flow.core.AbstractActor
-
- adams.flow.transformer.AbstractTransformer
-
- adams.flow.transformer.AbstractTransformerWithPropertiesUpdating
-
- adams.flow.transformer.WekaFilter
-
- All Implemented Interfaces:
adams.core.AdditionalInformationHandler,adams.core.CleanUpHandler,adams.core.Destroyable,adams.core.GlobalInfoSupporter,adams.core.io.ModelFileHandler,adams.core.logging.LoggingLevelHandler,adams.core.logging.LoggingSupporter,adams.core.option.OptionHandler,adams.core.QuickInfoSupporter,adams.core.ShallowCopySupporter<adams.flow.core.Actor>,adams.core.SizeOfHandler,adams.core.Stoppable,adams.core.StoppableWithFeedback,adams.core.VariablesInspectionHandler,adams.event.VariableChangeListener,adams.flow.container.OptionalContainerOutput,adams.flow.control.StorageUser,adams.flow.core.Actor,adams.flow.core.ErrorHandler,adams.flow.core.InputConsumer,adams.flow.core.ModelLoaderSupporter,adams.flow.core.OptionalOneTimeInitializer,adams.flow.core.OutputProducer,adams.flow.core.PropertiesUpdater,adams.multiprocess.JobRunnerSupporter,Serializable,Comparable
public class WekaFilter extends adams.flow.transformer.AbstractTransformerWithPropertiesUpdating implements adams.flow.container.OptionalContainerOutput, adams.core.io.ModelFileHandler, adams.flow.control.StorageUser, adams.flow.core.ModelLoaderSupporter, adams.flow.core.OptionalOneTimeInitializer, adams.multiprocess.JobRunnerSupporterFilters Instances/Instance objects using the specified filter.
When re-using a trained filter, ensure that 'initializeOnce' is checked.
The following order is used to obtain the model (when using AUTO):
1. model file present?
2. source actor present?
3. storage item present?
4. use specified filter definition
Input/output:
- accepts:
weka.core.Instance
weka.core.Instances
adams.data.instance.Instance
- generates:
weka.core.Instance
weka.core.Instances
adams.data.instance.Instance
-logging-level <OFF|SEVERE|WARNING|INFO|CONFIG|FINE|FINER|FINEST> (property: loggingLevel) The logging level for outputting errors and debugging output. default: WARNING min-user-mode: Expert
-name <java.lang.String> (property: name) The name of the actor. default: WekaFilter
-annotation <adams.core.base.BaseAnnotation> (property: annotations) The annotations to attach to this actor. default:
-skip <boolean> (property: skip) If set to true, transformation is skipped and the input token is just forwarded as it is. default: false
-stop-flow-on-error <boolean> (property: stopFlowOnError) If set to true, the flow execution at this level gets stopped in case this actor encounters an error; the error gets propagated; useful for critical actors. default: false min-user-mode: Expert
-silent <boolean> (property: silent) If enabled, then no errors are output in the console; Note: the enclosing actor handler must have this enabled as well. default: false min-user-mode: Expert
-property <adams.core.base.BaseString> [-property ...] (property: properties) The properties to update with the values associated with the specified values. default:
-variable <adams.core.VariableName> [-variable ...] (property: variableNames) The names of the variables to update the properties with. default:
-filter <weka.filters.Filter> (property: filter) The filter to use for filtering the Instances/Instance objects. default: weka.filters.AllFilter
-model-loading-type <AUTO|FILE|SOURCE_ACTOR|STORAGE> (property: modelLoadingType) Determines how to load the model, in case of AUTO, first the model file is checked, then the callable actor and then the storage. default: AUTO
-model <adams.core.io.PlaceholderFile> (property: modelFile) The file to load the model from, ignored if pointing to a directory. default: ${CWD}-source <adams.flow.core.CallableActorReference> (property: modelActor) The callable actor (source) to obtain the model from, ignored if not present. default:
-storage <adams.flow.control.StorageName> (property: modelStorage) The storage item to obtain the model from, ignored if not present. default: storage
-init-once <boolean> (property: initializeOnce) If set to true, then the filter will get initialized only with the first batch of data; otherwise every time data gets passed through; only applies when using the filter definition, the others (model file, source, storage ) assume the filter to be built. default: false
-keep <boolean> (property: keepRelationName) If set to true, then the filter won't change the relation name of the incoming dataset. default: false
-prefer-jobrunner <boolean> (property: preferJobRunner) If enabled, tries to offload the processing onto a adams.flow.standalone.JobRunnerInstance; applies only to batch filtering. default: false
-output-container <boolean> (property: outputContainer) If enabled, a adams.flow.container.WekaFilterContainer is output with the filter and the filtered data (Instance or Instances). default: false
- Author:
- fracpete (fracpete at waikato dot ac dot nz)
- See Also:
- Serialized Form
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static classWekaFilter.BatchFilterJob
-
Field Summary
Fields Modifier and Type Field Description static StringBACKUP_INITIALIZEDthe key for storing the current initialized state in the backup.protected weka.filters.Filterm_ActualFilterthe actual filter used.protected weka.filters.Filterm_Filterthe filter to apply.protected booleanm_FlowContextUpdatedwhether the flow context has been updated.protected booleanm_Initializedwhether the filter has been initialized.protected booleanm_InitializeOncewhether to initialize filter only with the first batch.protected adams.flow.standalone.JobRunnerInstancem_JobRunnerInstancethe JobRunnerInstance to use.protected booleanm_KeepRelationNamewhether to keep the incoming relation name.protected WekaFilterModelLoaderm_ModelLoaderthe model loader.protected booleanm_OutputContainerwhether to output a container.protected booleanm_PreferJobRunnerwhether to offload filtering into a JobRunnerInstance.-
Fields inherited from class adams.flow.transformer.AbstractTransformerWithPropertiesUpdating
m_Containers, m_Properties, m_VariableNames
-
Fields inherited from class adams.flow.transformer.AbstractTransformer
BACKUP_INPUT, BACKUP_OUTPUT, m_InputToken, m_OutputToken
-
Fields inherited from class adams.flow.core.AbstractActor
m_Annotations, m_BackupState, m_DetectedObjectVariables, m_DetectedVariables, m_ErrorHandler, m_Executed, m_Executing, m_ExecutionListeningSupporter, m_FullName, m_LoggingPrefix, m_Name, m_Parent, m_ScopeHandler, m_Self, m_Silent, m_Skip, m_StopFlowOnError, m_StopMessage, m_Stopped, m_StorageHandler, m_VariablesUpdated
-
-
Constructor Summary
Constructors Constructor Description WekaFilter()
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description Class[]accepts()Returns the class that the consumer accepts.protected Hashtable<String,Object>backupState()Backs up the current state of the actor before update the variables.protected adams.flow.core.TokencreateToken(Object input, Object data)Creates a token with the data.voiddefineOptions()Adds options to the internal list of options.protected StringdoExecute()Executes the flow item.StringfilterTipText()Returns the tip text for this property.Class[]generates()Returns the class of objects that it generates.weka.filters.FiltergetFilter()Returns the filter in use.booleangetInitializeOnce()Returns whether the filter gets initialized only with the first batch.booleangetKeepRelationName()Returns whether the filter doesn't change the relation name.adams.flow.core.CallableActorReferencegetModelActor()Returns the filter source actor.adams.core.io.PlaceholderFilegetModelFile()Returns the file to load the model from.adams.flow.core.AbstractModelLoader.ModelLoadingTypegetModelLoadingType()Returns the loading type.adams.flow.control.StorageNamegetModelStorage()Returns the filter storage item.booleangetOutputContainer()Returns whether to output a container with the filter alongside the filtered data or just the filtered data.booleangetPreferJobRunner()Returns whether to offload processing to a JobRunner instance if available.StringgetQuickInfo()Returns a quick info about the actor, which will be displayed in the GUI.StringglobalInfo()Returns a string describing the object.protected StringinitActualFilter(weka.core.Instances data)Initializes the actual filter to use.protected voidinitialize()Initializes the members.StringinitializeOnceTipText()Returns the tip text for this property.booleanisUsingStorage()Returns whether storage items are being used.StringkeepRelationNameTipText()Returns the tip text for this property.StringmodelActorTipText()Returns the tip text for this property.StringmodelFileTipText()Returns the tip text for this property.StringmodelLoadingTypeTipText()Returns the tip text for this property.StringmodelStorageTipText()Returns the tip text for this property.StringoutputContainerTipText()Returns the tip text for this property.StringpreferJobRunnerTipText()Returns the tip text for this property.protected voidpruneBackup()Removes entries from the backup.protected voidreset()Resets the scheme.protected voidrestoreState(Hashtable<String,Object> state)Restores the state of the actor before the variables got updated.voidsetFilter(weka.filters.Filter value)Sets the filter to use.voidsetInitializeOnce(boolean value)Sets whether the filter gets initialized only with the first batch.voidsetKeepRelationName(boolean value)Sets whether the filter doesn't change the relation name.voidsetLoggingLevel(adams.core.logging.LoggingLevel value)Sets the logging level.voidsetModelActor(adams.flow.core.CallableActorReference value)Sets the filter source actor.voidsetModelFile(adams.core.io.PlaceholderFile value)Sets the file to load the model from.voidsetModelLoadingType(adams.flow.core.AbstractModelLoader.ModelLoadingType value)Sets the loading type.voidsetModelStorage(adams.flow.control.StorageName value)Sets the filter storage item.voidsetOutputContainer(boolean value)Sets whether to output a container with the filter alongside the filtered data or just the filtered data.voidsetPreferJobRunner(boolean value)Sets whether to offload processing to a JobRunner instance if available.StringsetUp()Initializes the item for flow execution.-
Methods inherited from class adams.flow.transformer.AbstractTransformerWithPropertiesUpdating
getProperties, getVariableNames, propertiesTipText, setProperties, setUpContainers, setUpContainersIfNecessary, setVariableNames, updateObject, variableNamesTipText
-
Methods inherited from class adams.flow.transformer.AbstractTransformer
currentInput, execute, hasInput, hasPendingOutput, input, output, postExecute, wrapUp
-
Methods inherited from class adams.flow.core.AbstractActor
annotationsTipText, canInspectOptions, canPerformSetUpCheck, cleanUp, compareTo, configureLogger, destroy, equals, finalUpdateVariables, findVariables, findVariables, forceVariables, forCommandLine, forName, forName, getAdditionalInformation, getAnnotations, getDefaultName, getDetectedVariables, getErrorHandler, getFlowActors, getFlowExecutionListeningSupporter, getFullName, getName, getNextSibling, getParent, getParentComponent, getPreviousSibling, getRoot, getScopeHandler, getSilent, getSkip, getStopFlowOnError, getStopMessage, getStorageHandler, getVariables, handleError, handleException, hasErrorHandler, hasStopMessage, index, isBackedUp, isExecuted, isExecuting, isFinished, isHeadless, isStopped, nameTipText, performSetUpChecks, performVariableChecks, preExecute, pruneBackup, setAnnotations, setErrorHandler, setName, setParent, setSilent, setSkip, setStopFlowOnError, setVariables, shallowCopy, shallowCopy, silentTipText, sizeOf, skipTipText, stopExecution, stopExecution, stopFlowOnErrorTipText, updateDetectedVariables, updatePrefix, updateVariables, variableChanged
-
Methods inherited from class adams.core.option.AbstractOptionHandler
cleanUpOptions, finishInit, getDefaultLoggingLevel, getOptionManager, loggingLevelTipText, newOptionManager, toCommandLine, toString
-
Methods inherited from class adams.core.logging.LoggingObject
getLogger, getLoggingLevel, initializeLogging, isLoggingEnabled
-
Methods inherited from class java.lang.Object
clone, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
-
Methods inherited from interface adams.flow.core.Actor
cleanUp, compareTo, destroy, equals, execute, findVariables, getAnnotations, getDefaultName, getDetectedVariables, getErrorHandler, getFlowExecutionListeningSupporter, getFullName, getName, getNextSibling, getParent, getParentComponent, getPreviousSibling, getRoot, getScopeHandler, getSilent, getSkip, getStopFlowOnError, getStopMessage, getStorageHandler, getVariables, handleError, hasErrorHandler, hasStopMessage, index, isExecuted, isFinished, isHeadless, isStopped, setAnnotations, setErrorHandler, setName, setParent, setSilent, setSkip, setStopFlowOnError, setVariables, shallowCopy, shallowCopy, sizeOf, stopExecution, stopExecution, toCommandLine, variableChanged, wrapUp
-
-
-
-
Field Detail
-
BACKUP_INITIALIZED
public static final String BACKUP_INITIALIZED
the key for storing the current initialized state in the backup.- See Also:
- Constant Field Values
-
m_Filter
protected weka.filters.Filter m_Filter
the filter to apply.
-
m_ActualFilter
protected weka.filters.Filter m_ActualFilter
the actual filter used.
-
m_InitializeOnce
protected boolean m_InitializeOnce
whether to initialize filter only with the first batch.
-
m_KeepRelationName
protected boolean m_KeepRelationName
whether to keep the incoming relation name.
-
m_OutputContainer
protected boolean m_OutputContainer
whether to output a container.
-
m_Initialized
protected boolean m_Initialized
whether the filter has been initialized.
-
m_FlowContextUpdated
protected boolean m_FlowContextUpdated
whether the flow context has been updated.
-
m_ModelLoader
protected WekaFilterModelLoader m_ModelLoader
the model loader.
-
m_PreferJobRunner
protected boolean m_PreferJobRunner
whether to offload filtering into a JobRunnerInstance.
-
m_JobRunnerInstance
protected transient adams.flow.standalone.JobRunnerInstance m_JobRunnerInstance
the JobRunnerInstance to use.
-
-
Method Detail
-
globalInfo
public String globalInfo()
Returns a string describing the object.- Specified by:
globalInfoin interfaceadams.core.GlobalInfoSupporter- Specified by:
globalInfoin classadams.core.option.AbstractOptionHandler- Returns:
- a description suitable for displaying in the gui
-
defineOptions
public void defineOptions()
Adds options to the internal list of options.- Specified by:
defineOptionsin interfaceadams.core.option.OptionHandler- Overrides:
defineOptionsin classadams.flow.transformer.AbstractTransformerWithPropertiesUpdating
-
initialize
protected void initialize()
Initializes the members.- Overrides:
initializein classadams.flow.core.AbstractActor
-
setLoggingLevel
public void setLoggingLevel(adams.core.logging.LoggingLevel value)
Sets the logging level.- Specified by:
setLoggingLevelin interfaceadams.core.logging.LoggingLevelHandler- Overrides:
setLoggingLevelin classadams.core.option.AbstractOptionHandler- Parameters:
value- the level
-
setFilter
public void setFilter(weka.filters.Filter value)
Sets the filter to use.- Parameters:
value- the filter
-
getFilter
public weka.filters.Filter getFilter()
Returns the filter in use.- Returns:
- the filter
-
filterTipText
public String filterTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setModelLoadingType
public void setModelLoadingType(adams.flow.core.AbstractModelLoader.ModelLoadingType value)
Sets the loading type. In case ofAbstractModelLoader.ModelLoadingType.AUTO, first file, then callable actor, then storage.- Specified by:
setModelLoadingTypein interfaceadams.flow.core.ModelLoaderSupporter- Parameters:
value- the type
-
getModelLoadingType
public adams.flow.core.AbstractModelLoader.ModelLoadingType getModelLoadingType()
Returns the loading type. In case ofAbstractModelLoader.ModelLoadingType.AUTO, first file, then callable actor, then storage.- Specified by:
getModelLoadingTypein interfaceadams.flow.core.ModelLoaderSupporter- Returns:
- the type
-
modelLoadingTypeTipText
public String modelLoadingTypeTipText()
Returns the tip text for this property.- Specified by:
modelLoadingTypeTipTextin interfaceadams.flow.core.ModelLoaderSupporter- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setModelFile
public void setModelFile(adams.core.io.PlaceholderFile value)
Sets the file to load the model from.- Specified by:
setModelFilein interfaceadams.core.io.ModelFileHandler- Specified by:
setModelFilein interfaceadams.flow.core.ModelLoaderSupporter- Parameters:
value- the model file
-
getModelFile
public adams.core.io.PlaceholderFile getModelFile()
Returns the file to load the model from.- Specified by:
getModelFilein interfaceadams.core.io.ModelFileHandler- Specified by:
getModelFilein interfaceadams.flow.core.ModelLoaderSupporter- Returns:
- the model file
-
modelFileTipText
public String modelFileTipText()
Returns the tip text for this property.- Specified by:
modelFileTipTextin interfaceadams.core.io.ModelFileHandler- Specified by:
modelFileTipTextin interfaceadams.flow.core.ModelLoaderSupporter- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setModelActor
public void setModelActor(adams.flow.core.CallableActorReference value)
Sets the filter source actor.- Specified by:
setModelActorin interfaceadams.flow.core.ModelLoaderSupporter- Parameters:
value- the source
-
getModelActor
public adams.flow.core.CallableActorReference getModelActor()
Returns the filter source actor.- Specified by:
getModelActorin interfaceadams.flow.core.ModelLoaderSupporter- Returns:
- the source
-
modelActorTipText
public String modelActorTipText()
Returns the tip text for this property.- Specified by:
modelActorTipTextin interfaceadams.flow.core.ModelLoaderSupporter- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setModelStorage
public void setModelStorage(adams.flow.control.StorageName value)
Sets the filter storage item.- Specified by:
setModelStoragein interfaceadams.flow.core.ModelLoaderSupporter- Parameters:
value- the storage item
-
getModelStorage
public adams.flow.control.StorageName getModelStorage()
Returns the filter storage item.- Specified by:
getModelStoragein interfaceadams.flow.core.ModelLoaderSupporter- Returns:
- the storage item
-
modelStorageTipText
public String modelStorageTipText()
Returns the tip text for this property.- Specified by:
modelStorageTipTextin interfaceadams.flow.core.ModelLoaderSupporter- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setInitializeOnce
public void setInitializeOnce(boolean value)
Sets whether the filter gets initialized only with the first batch.- Specified by:
setInitializeOncein interfaceadams.flow.core.OptionalOneTimeInitializer- Parameters:
value- true if the filter gets only initialized once
-
getInitializeOnce
public boolean getInitializeOnce()
Returns whether the filter gets initialized only with the first batch.- Specified by:
getInitializeOncein interfaceadams.flow.core.OptionalOneTimeInitializer- Returns:
- true if the filter gets only initialized once
-
initializeOnceTipText
public String initializeOnceTipText()
Returns the tip text for this property.- Specified by:
initializeOnceTipTextin interfaceadams.flow.core.OptionalOneTimeInitializer- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setKeepRelationName
public void setKeepRelationName(boolean value)
Sets whether the filter doesn't change the relation name.- Parameters:
value- true if the filter won't change the relation name
-
getKeepRelationName
public boolean getKeepRelationName()
Returns whether the filter doesn't change the relation name.- Returns:
- true if the filter doesn't change the relation name
-
keepRelationNameTipText
public String keepRelationNameTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setPreferJobRunner
public void setPreferJobRunner(boolean value)
Sets whether to offload processing to a JobRunner instance if available.- Specified by:
setPreferJobRunnerin interfaceadams.multiprocess.JobRunnerSupporter- Parameters:
value- if true try to find/use a JobRunner instance
-
getPreferJobRunner
public boolean getPreferJobRunner()
Returns whether to offload processing to a JobRunner instance if available.- Specified by:
getPreferJobRunnerin interfaceadams.multiprocess.JobRunnerSupporter- Returns:
- if true try to find/use a JobRunner instance
-
preferJobRunnerTipText
public String preferJobRunnerTipText()
Returns the tip text for this property.- Specified by:
preferJobRunnerTipTextin interfaceadams.multiprocess.JobRunnerSupporter- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setOutputContainer
public void setOutputContainer(boolean value)
Sets whether to output a container with the filter alongside the filtered data or just the filtered data.- Specified by:
setOutputContainerin interfaceadams.flow.container.OptionalContainerOutput- Parameters:
value- true if to output the container
-
getOutputContainer
public boolean getOutputContainer()
Returns whether to output a container with the filter alongside the filtered data or just the filtered data.- Specified by:
getOutputContainerin interfaceadams.flow.container.OptionalContainerOutput- Returns:
- true if to output the container
-
outputContainerTipText
public String outputContainerTipText()
Returns the tip text for this property.- Specified by:
outputContainerTipTextin interfaceadams.flow.container.OptionalContainerOutput- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
isUsingStorage
public boolean isUsingStorage()
Returns whether storage items are being used.- Specified by:
isUsingStoragein interfaceadams.flow.control.StorageUser- Returns:
- true if storage items are used
-
generates
public Class[] generates()
Returns the class of objects that it generates.- Specified by:
generatesin interfaceadams.flow.core.OutputProducer- Returns:
- weka.core.Instance, weka.core.Instances, adams.data.instance.Instance
-
getQuickInfo
public String getQuickInfo()
Returns a quick info about the actor, which will be displayed in the GUI.- Specified by:
getQuickInfoin interfaceadams.flow.core.Actor- Specified by:
getQuickInfoin interfaceadams.core.QuickInfoSupporter- Overrides:
getQuickInfoin classadams.flow.transformer.AbstractTransformerWithPropertiesUpdating- Returns:
- null if no info available, otherwise short string
-
pruneBackup
protected void pruneBackup()
Removes entries from the backup.- Overrides:
pruneBackupin classadams.flow.core.AbstractActor
-
backupState
protected Hashtable<String,Object> backupState()
Backs up the current state of the actor before update the variables.- Overrides:
backupStatein classadams.flow.transformer.AbstractTransformer- Returns:
- the backup
-
restoreState
protected void restoreState(Hashtable<String,Object> state)
Restores the state of the actor before the variables got updated.- Overrides:
restoreStatein classadams.flow.transformer.AbstractTransformer- Parameters:
state- the backup of the state to restore from
-
reset
protected void reset()
Resets the scheme.- Overrides:
resetin classadams.flow.transformer.AbstractTransformerWithPropertiesUpdating
-
accepts
public Class[] accepts()
Returns the class that the consumer accepts.- Specified by:
acceptsin interfaceadams.flow.core.InputConsumer- Returns:
- weka.core.Instance, weka.core.Instances, adams.data.instance.Instance
-
createToken
protected adams.flow.core.Token createToken(Object input, Object data)
Creates a token with the data. If required creates a container with the filter.- Parameters:
data- the data to output in the token- Returns:
- the generated token
-
initActualFilter
protected String initActualFilter(weka.core.Instances data) throws Exception
Initializes the actual filter to use.- Parameters:
data- the data to initialize the filter when using the filter definition- Returns:
- null if successful, otherwise error message
- Throws:
Exception- if initialization, copying etc fails
-
setUp
public String setUp()
Initializes the item for flow execution.- Specified by:
setUpin interfaceadams.flow.core.Actor- Overrides:
setUpin classadams.flow.core.AbstractActor- Returns:
- null if everything is fine, otherwise error message
-
doExecute
protected String doExecute()
Executes the flow item.- Specified by:
doExecutein classadams.flow.core.AbstractActor- Returns:
- null if everything is fine, otherwise error message
-
-