Package adams.flow.transformer

Class WekaTestSetEvaluator

  • All Implemented Interfaces:
    adams.core.AdditionalInformationHandler, adams.core.CleanUpHandler, adams.core.Destroyable, adams.core.GlobalInfoSupporter, adams.core.logging.LoggingLevelHandler, adams.core.logging.LoggingSupporter, adams.core.option.OptionHandler, adams.core.QuickInfoSupporter, adams.core.ShallowCopySupporter<adams.flow.core.Actor>, adams.core.SizeOfHandler, adams.core.Stoppable, adams.core.StoppableWithFeedback, adams.core.VariablesInspectionHandler, adams.event.VariableChangeListener, adams.flow.core.Actor, adams.flow.core.ErrorHandler, adams.flow.core.InputConsumer, adams.flow.core.OutputProducer, adams.multiprocess.JobRunnerSupporter, Serializable, Comparable
    public class WekaTestSetEvaluator
extends AbstractWekaClassifierEvaluator
implements adams.multiprocess.JobRunnerSupporter
    Evaluates a trained classifier (obtained from input) on the dataset obtained from the callable actor.

    Input/output:
    - accepts:
       weka.classifiers.Classifier
       adams.flow.container.WekaModelContainer
    - generates:
       adams.flow.container.WekaEvaluationContainer


    Container information:
    - adams.flow.container.WekaModelContainer: Model, Header, Dataset
    - adams.flow.container.WekaEvaluationContainer: Evaluation, Model, Prediction output, Original indices, Test data 

    -logging-level <OFF|SEVERE|WARNING|INFO|CONFIG|FINE|FINER|FINEST> (property: loggingLevel)
    The logging level for outputting errors and debugging output.
    default: WARNING
    min-user-mode: Expert
 
    -name <java.lang.String> (property: name)
    The name of the actor.
    default: WekaTestSetEvaluator
 
    -annotation <adams.core.base.BaseAnnotation> (property: annotations)
    The annotations to attach to this actor.
    default:
 
    -skip <boolean> (property: skip)
    If set to true, transformation is skipped and the input token is just forwarded
    as it is.
    default: false
 
    -stop-flow-on-error <boolean> (property: stopFlowOnError)
    If set to true, the flow execution at this level gets stopped in case this
    actor encounters an error; the error gets propagated; useful for critical
    actors.
    default: false
    min-user-mode: Expert
 
    -silent <boolean> (property: silent)
    If enabled, then no errors are output in the console; Note: the enclosing
    actor handler must have this enabled as well.
    default: false
    min-user-mode: Expert
 
    -output <weka.classifiers.evaluation.output.prediction.AbstractOutput> (property: output)
    The class for generating prediction output; if 'Null' is used, then an Evaluation
    object is forwarded instead of a String.
    default: weka.classifiers.evaluation.output.prediction.Null
 
    -always-use-container <boolean> (property: alwaysUseContainer)
    If enabled, always outputs an evaluation container.
    default: false
 
    -testset <adams.flow.core.CallableActorReference> (property: testset)
    The callable actor to use for obtaining the test set.
    default: Testset
 
    -no-predictions <boolean> (property: discardPredictions)
    If enabled, the collection of predictions during evaluation is suppressed,
     which will conserve memory.
    default: false
 
    -prefer-jobrunner <boolean> (property: preferJobRunner)
    If enabled, tries to offload the processing onto a adams.flow.standalone.JobRunnerInstance;
     applies only to training.
    default: false
    Author:
    fracpete (fracpete at waikato dot ac dot nz)
    See Also:
    Serialized Form

    • Field Summary

      Fields 
      Modifier and Type Field Description
      protected StoppableEvaluation m_CurrentEvaluation
      the current evaluation.
      protected boolean m_DiscardPredictions
      whether to discard predictions.
      protected adams.flow.standalone.JobRunnerInstance m_JobRunnerInstance
      the JobRunnerInstance to use.
      protected boolean m_PreferJobRunner
      whether to offload training into a JobRunnerInstance.
      protected adams.flow.core.CallableActorReference m_Testset
      the name of the callable trainset provider.

      • Fields inherited from class adams.flow.transformer.AbstractTransformer

        BACKUP_INPUT, BACKUP_OUTPUT, m_InputToken, m_OutputToken

      • Fields inherited from class adams.flow.core.AbstractActor

        m_Annotations, m_BackupState, m_DetectedObjectVariables, m_DetectedVariables, m_ErrorHandler, m_Executed, m_Executing, m_ExecutionListeningSupporter, m_FullName, m_LoggingPrefix, m_Name, m_Parent, m_ScopeHandler, m_Self, m_Silent, m_Skip, m_StopFlowOnError, m_StopMessage, m_Stopped, m_StorageHandler, m_VariablesUpdated

      • Fields inherited from class adams.core.option.AbstractOptionHandler

        m_OptionManager

      • Fields inherited from class adams.core.logging.LoggingObject

        m_Logger, m_LoggingIsEnabled, m_LoggingLevel

      • Fields inherited from interface adams.flow.core.Actor

        FILE_EXTENSION, FILE_EXTENSION_GZ

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      Class[] accepts()
      Returns the class that the consumer accepts.
      void defineOptions()
      Adds options to the internal list of options.
      String discardPredictionsTipText()
      Returns the tip text for this property.
      protected String doExecute()
      Executes the flow item.
      boolean getDiscardPredictions()
      Returns whether to discard the predictions in order to preserve memory.
      boolean getPreferJobRunner()
      Returns whether to offload processing to a JobRunner instance if available.
      String getQuickInfo()
      Returns a quick info about the actor, which will be displayed in the GUI.
      adams.flow.core.CallableActorReference getTestset()
      Returns the name of the callable classifier in use.
      String globalInfo()
      Returns a string describing the object.
      String preferJobRunnerTipText()
      Returns the tip text for this property.
      void setDiscardPredictions​(boolean value)
      Sets whether to discard the predictions instead of collecting them for future use, in order to conserve memory.
      void setPreferJobRunner​(boolean value)
      Sets whether to offload processing to a JobRunner instance if available.
      void setTestset​(adams.flow.core.CallableActorReference value)
      Sets the name of the callable classifier to use.
      void stopExecution()
      Stops the execution.
      String testsetTipText()
      Returns the tip text for this property.

      • Methods inherited from class adams.flow.transformer.AbstractTransformer

        backupState, currentInput, execute, hasInput, hasPendingOutput, input, output, postExecute, restoreState

      • Methods inherited from class adams.flow.core.AbstractActor

        annotationsTipText, canInspectOptions, canPerformSetUpCheck, cleanUp, compareTo, configureLogger, destroy, equals, finalUpdateVariables, findVariables, findVariables, forceVariables, forCommandLine, forName, forName, getAdditionalInformation, getAnnotations, getDefaultName, getDetectedVariables, getErrorHandler, getFlowActors, getFlowExecutionListeningSupporter, getFullName, getName, getNextSibling, getParent, getParentComponent, getPreviousSibling, getRoot, getScopeHandler, getSilent, getSkip, getStopFlowOnError, getStopMessage, getStorageHandler, getVariables, handleError, handleException, hasErrorHandler, hasStopMessage, index, initialize, isBackedUp, isExecuted, isExecuting, isFinished, isHeadless, isStopped, nameTipText, performSetUpChecks, performVariableChecks, preExecute, pruneBackup, pruneBackup, setAnnotations, setErrorHandler, setName, setParent, setSilent, setSkip, setStopFlowOnError, setUp, setVariables, shallowCopy, shallowCopy, silentTipText, sizeOf, skipTipText, stopExecution, stopFlowOnErrorTipText, updateDetectedVariables, updatePrefix, updateVariables, variableChanged

      • Methods inherited from class adams.core.option.AbstractOptionHandler

        cleanUpOptions, finishInit, getDefaultLoggingLevel, getOptionManager, loggingLevelTipText, newOptionManager, setLoggingLevel, toCommandLine, toString

      • Methods inherited from class adams.core.logging.LoggingObject

        getLogger, getLoggingLevel, initializeLogging, isLoggingEnabled

      • Methods inherited from interface adams.flow.core.Actor

        cleanUp, compareTo, destroy, equals, findVariables, getAnnotations, getDefaultName, getDetectedVariables, getErrorHandler, getFlowExecutionListeningSupporter, getFullName, getName, getNextSibling, getParent, getParentComponent, getPreviousSibling, getRoot, getScopeHandler, getSilent, getSkip, getStopFlowOnError, getStopMessage, getStorageHandler, getVariables, handleError, hasErrorHandler, hasStopMessage, index, isExecuted, isFinished, isHeadless, isStopped, setAnnotations, setErrorHandler, setName, setParent, setSilent, setSkip, setStopFlowOnError, setUp, setVariables, shallowCopy, shallowCopy, sizeOf, stopExecution, toCommandLine, variableChanged

      • Methods inherited from interface adams.core.AdditionalInformationHandler

        getAdditionalInformation

      • Methods inherited from interface adams.core.logging.LoggingLevelHandler

        getLoggingLevel, setLoggingLevel

      • Methods inherited from interface adams.core.logging.LoggingSupporter

        getLogger, isLoggingEnabled

      • Methods inherited from interface adams.core.option.OptionHandler

        cleanUpOptions, getOptionManager

      • Methods inherited from interface adams.core.VariablesInspectionHandler

        canInspectOptions

    • Field Detail

      • m_Testset

        protected adams.flow.core.CallableActorReference m_Testset
        the name of the callable trainset provider.

      • m_DiscardPredictions

        protected boolean m_DiscardPredictions
        whether to discard predictions.

      • m_PreferJobRunner

        protected boolean m_PreferJobRunner
        whether to offload training into a JobRunnerInstance.

      • m_JobRunnerInstance

        protected transient adams.flow.standalone.JobRunnerInstance m_JobRunnerInstance
        the JobRunnerInstance to use.

      • m_CurrentEvaluation

        protected transient StoppableEvaluation m_CurrentEvaluation
        the current evaluation.

    • Constructor Detail

      • WekaTestSetEvaluator

        public WekaTestSetEvaluator()

    • Method Detail

      • globalInfo

        public String globalInfo()
        Returns a string describing the object.
        Specified by:
        globalInfo in interface adams.core.GlobalInfoSupporter
        Specified by:
        globalInfo in class adams.core.option.AbstractOptionHandler
        Returns:
        a description suitable for displaying in the gui

      • defineOptions

        public void defineOptions()
        Adds options to the internal list of options.
        Specified by:
        defineOptions in interface adams.core.option.OptionHandler
        Overrides:
        defineOptions in class AbstractWekaClassifierEvaluator

      • getQuickInfo

        public String getQuickInfo()
        Returns a quick info about the actor, which will be displayed in the GUI.
        Specified by:
        getQuickInfo in interface adams.flow.core.Actor
        Specified by:
        getQuickInfo in interface adams.core.QuickInfoSupporter
        Overrides:
        getQuickInfo in class adams.flow.core.AbstractActor
        Returns:
        null if no info available, otherwise short string

      • setTestset

        public void setTestset​(adams.flow.core.CallableActorReference value)
        Sets the name of the callable classifier to use.
        Parameters:
        value - the name

      • getTestset

        public adams.flow.core.CallableActorReference getTestset()
        Returns the name of the callable classifier in use.
        Returns:
        the name

      • testsetTipText

        public String testsetTipText()
        Returns the tip text for this property.
        Returns:
        tip text for this property suitable for displaying in the GUI or for listing the options.

      • setDiscardPredictions

        public void setDiscardPredictions​(boolean value)
        Sets whether to discard the predictions instead of collecting them for future use, in order to conserve memory.
        Parameters:
        value - true if to discard predictions

      • getDiscardPredictions

        public boolean getDiscardPredictions()
        Returns whether to discard the predictions in order to preserve memory.
        Returns:
        true if predictions discarded

      • discardPredictionsTipText

        public String discardPredictionsTipText()
        Returns the tip text for this property.
        Returns:
        tip text for this property suitable for displaying in the GUI or for listing the options.

      • setPreferJobRunner

        public void setPreferJobRunner​(boolean value)
        Sets whether to offload processing to a JobRunner instance if available.
        Specified by:
        setPreferJobRunner in interface adams.multiprocess.JobRunnerSupporter
        Parameters:
        value - if true try to find/use a JobRunner instance

      • getPreferJobRunner

        public boolean getPreferJobRunner()
        Returns whether to offload processing to a JobRunner instance if available.
        Specified by:
        getPreferJobRunner in interface adams.multiprocess.JobRunnerSupporter
        Returns:
        if true try to find/use a JobRunner instance

      • preferJobRunnerTipText

        public String preferJobRunnerTipText()
        Returns the tip text for this property.
        Specified by:
        preferJobRunnerTipText in interface adams.multiprocess.JobRunnerSupporter
        Returns:
        tip text for this property suitable for displaying in the GUI or for listing the options.

      • accepts

        public Class[] accepts()
        Returns the class that the consumer accepts.
        Specified by:
        accepts in interface adams.flow.core.InputConsumer
        Returns:
        weka.classifiers.Classifier.class, adams.flow.container.WekaModelContainer.class

      • doExecute

        protected String doExecute()
        Executes the flow item.
        Specified by:
        doExecute in class adams.flow.core.AbstractActor
        Returns:
        null if everything is fine, otherwise error message

      • stopExecution

        public void stopExecution()
        Stops the execution. No message set.
        Specified by:
        stopExecution in interface adams.flow.core.Actor
        Specified by:
        stopExecution in interface adams.core.Stoppable
        Overrides:
        stopExecution in class adams.flow.core.AbstractActor