Package adams.flow.transformer

Class WekaClassifierRanker

  • All Implemented Interfaces:
    adams.core.AdditionalInformationHandler, adams.core.CleanUpHandler, adams.core.Destroyable, adams.core.GlobalInfoSupporter, adams.core.logging.LoggingLevelHandler, adams.core.logging.LoggingSupporter, adams.core.option.OptionHandler, adams.core.Pausable, adams.core.QuickInfoSupporter, adams.core.Randomizable, adams.core.ShallowCopySupporter<adams.flow.core.Actor>, adams.core.SizeOfHandler, adams.core.Stoppable, adams.core.StoppableWithFeedback, adams.core.ThreadLimiter, adams.core.VariablesInspectionHandler, adams.event.FlowPauseStateListener, adams.event.VariableChangeListener, adams.flow.core.Actor, adams.flow.core.ErrorHandler, adams.flow.core.InputConsumer, adams.flow.core.OutputProducer, Serializable, Comparable
    public class WekaClassifierRanker
extends adams.flow.transformer.AbstractTransformer
implements adams.core.Randomizable, adams.core.Pausable, adams.event.FlowPauseStateListener, adams.core.ThreadLimiter
    Performs a quick evaluation using cross-validation on a single dataset (or evaluation on a separate test set if the number of folds is less than 2) to rank the classifiers received on the input and forwarding the x best ones. Further evaluation can be performed using the Experimenter.

    Input/output:
    - accepts:
       weka.classifiers.Classifier[]
    - generates:
       weka.classifiers.Classifier[]


    -logging-level <OFF|SEVERE|WARNING|INFO|CONFIG|FINE|FINER|FINEST> (property: loggingLevel)
    The logging level for outputting errors and debugging output.
    default: WARNING
 
    -name <java.lang.String> (property: name)
    The name of the actor.
    default: WekaClassifierRanker
 
    -annotation <adams.core.base.BaseText> (property: annotations)
    The annotations to attach to this actor.
    default: 
 
    -skip <boolean> (property: skip)
    If set to true, transformation is skipped and the input token is just forwarded 
    as it is.
    default: false
 
    -stop-flow-on-error <boolean> (property: stopFlowOnError)
    If set to true, the flow gets stopped in case this actor encounters an error;
     useful for critical actors.
    default: false
 
    -max <int> (property: max)
    The maximum number of top-ranked classifiers to forward; use -1 to forward 
    all of them (ranked array).
    default: 3
    minimum: -1
 
    -seed <long> (property: seed)
    The seed value to use in the cross-validation.
    default: 1
 
    -folds <int> (property: folds)
    The number of folds to use in cross-validation.
    default: 10
    minimum: 1
 
    -measure <CC|RMSE|RRSE|MAE|RAE|COMBINED|ACC|KAPPA> (property: measure)
    The measure used for ranking the classifiers.
    default: CC
 
    -train <adams.flow.core.CallableActorReference> (property: train)
    The name of the callable actor that is used for obtaining the training set.
    default: train
 
    -test <adams.flow.core.CallableActorReference> (property: test)
    The name of the callable actor that is used for obtaining the test set (
    only if folds <2).
    default: test
 
    -output-best <boolean> (property: outputBestSetup)
    If true, then for optimizers like GridSearch and MultiSearch the best setup 
    that was found will be output instead of the optimizer setup.
    default: false
 
    -num-threads <int> (property: numThreads)
    The number of threads to use for evaluating the classifiers in parallel 
    (-1 means one for each core/cpu).
    default: -1
    minimum: -1
    Author:
    fracpete (fracpete at waikato dot ac dot nz)
    See Also:
    Serialized Form

    • Field Summary

      Fields 
      Modifier and Type Field Description
      protected WekaLabelIndex m_ClassLabel
      the class label (in case of class-specific measures).
      protected int m_Folds
      the number of folds to use in cross-validation.
      protected adams.flow.core.CallableActorHelper m_Helper
      the helper class.
      protected adams.multiprocess.JobRunner m_JobRunner
      the job runner for evaluating the setups.
      protected adams.flow.standalone.JobRunnerSetup m_JobRunnerSetup
      the jobrunner setup.
      protected int m_Max
      the maximum number of top-ranked classifiers to forward.
      protected WekaClassifierRanker.Measure m_Measure
      the measure for the evaluation.
      protected int m_NumThreads
      the number of threads to use for parallel execution.
      protected boolean m_OutputBestSetup
      whether to output the best setup in case of GridSearch/MultiSearch.
      protected long m_Seed
      the random seed to use.
      protected adams.flow.core.CallableActorReference m_Test
      the callable actor to obtain the test dataset for train/test evaluation from.
      protected adams.flow.core.CallableActorReference m_Train
      the callable actor to obtain the training dataset from.

      • Fields inherited from class adams.flow.transformer.AbstractTransformer

        BACKUP_INPUT, BACKUP_OUTPUT, m_InputToken, m_OutputToken

      • Fields inherited from class adams.flow.core.AbstractActor

        m_Annotations, m_BackupState, m_DetectedObjectVariables, m_DetectedVariables, m_ErrorHandler, m_Executed, m_Executing, m_ExecutionListeningSupporter, m_FullName, m_LoggingPrefix, m_Name, m_Parent, m_ScopeHandler, m_Self, m_Silent, m_Skip, m_StopFlowOnError, m_StopMessage, m_Stopped, m_StorageHandler, m_VariablesUpdated

      • Fields inherited from class adams.core.option.AbstractOptionHandler

        m_OptionManager

      • Fields inherited from class adams.core.logging.LoggingObject

        m_Logger, m_LoggingIsEnabled, m_LoggingLevel

      • Fields inherited from interface adams.flow.core.Actor

        FILE_EXTENSION, FILE_EXTENSION_GZ

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      Class[] accepts()
      Returns the class that the consumer accepts.
      String classLabelTipText()
      Returns the tip text for this property.
      void cleanUp()
      Frees up memory in a "destructive" non-reversible way.
      void defineOptions()
      Adds options to the internal list of options.
      protected String doExecute()
      Executes the flow item.
      void flowPauseStateChanged​(adams.event.FlowPauseStateEvent e)
      Gets called when the pause state of the flow changes.
      String foldsTipText()
      Returns the tip text for this property.
      Class[] generates()
      Returns the class of objects that it generates.
      WekaLabelIndex getClassLabel()
      Returns the class label index to use for class-specific measures.
      int getFolds()
      Returns the number of folds to use.
      int getMax()
      Returns the maximum number of top-ranked classifiers to forward.
      WekaClassifierRanker.Measure getMeasure()
      Returns the measure used for ranking the classifiers.
      int getNumThreads()
      Returns the number of threads in use.
      boolean getOutputBestSetup()
      Returns whether to output the best setup for optimizers like GridSearch and MultiSearch.
      String getQuickInfo()
      Returns a quick info about the actor, which will be displayed in the GUI.
      long getSeed()
      Returns the seed value.
      adams.flow.core.CallableActorReference getTest()
      Returns the name of the callable actor to obtain the test set.
      adams.flow.core.CallableActorReference getTrain()
      Returns the name of the callable actor to obtain the training set.
      String globalInfo()
      Returns a string describing the object.
      protected void initialize()
      Initializes the members.
      boolean isPaused()
      Returns whether the object is currently paused.
      String maxTipText()
      Returns the tip text for this property.
      String measureTipText()
      Returns the tip text for this property.
      String numThreadsTipText()
      Returns the tip text for this property.
      String outputBestSetupTipText()
      Returns the tip text for this property.
      void pauseExecution()
      Pauses the execution.
      void resumeExecution()
      Resumes the execution.
      String seedTipText()
      Returns the tip text for this property.
      void setClassLabel​(WekaLabelIndex value)
      Sets the class label index to use for class-specific measures.
      void setFolds​(int value)
      Sets the number of folds to use.
      void setMax​(int value)
      Sets the maximum number of top-ranked classifiers to forward.
      void setMeasure​(WekaClassifierRanker.Measure value)
      Sets the measure to use for ranking the classifiers.
      void setNumThreads​(int value)
      Sets the number of threads to use.
      void setOutputBestSetup​(boolean value)
      Sets whether to output the best setup found for optimizers like GridSearch and MultiSearch.
      void setSeed​(long value)
      Sets the seed value.
      void setTest​(adams.flow.core.CallableActorReference value)
      Sets the name of the callable actor to obtain the test set.
      void setTrain​(adams.flow.core.CallableActorReference value)
      Sets the name of the callable actor to obtain the training set.
      String setUp()
      Initializes the item for flow execution.
      void stopExecution()
      Stops the execution.
      String testTipText()
      Returns the tip text for this property.
      String trainTipText()
      Returns the tip text for this property.

      • Methods inherited from class adams.flow.transformer.AbstractTransformer

        backupState, currentInput, execute, hasInput, hasPendingOutput, input, output, postExecute, restoreState, wrapUp

      • Methods inherited from class adams.flow.core.AbstractActor

        annotationsTipText, canInspectOptions, canPerformSetUpCheck, compareTo, configureLogger, destroy, equals, finalUpdateVariables, findVariables, findVariables, forceVariables, forCommandLine, forName, forName, getAdditionalInformation, getAnnotations, getDefaultName, getDetectedVariables, getErrorHandler, getFlowActors, getFlowExecutionListeningSupporter, getFullName, getName, getNextSibling, getParent, getParentComponent, getPreviousSibling, getRoot, getScopeHandler, getSilent, getSkip, getStopFlowOnError, getStopMessage, getStorageHandler, getVariables, handleError, handleException, hasErrorHandler, hasStopMessage, index, isBackedUp, isExecuted, isExecuting, isFinished, isHeadless, isStopped, nameTipText, performSetUpChecks, performVariableChecks, preExecute, pruneBackup, pruneBackup, reset, setAnnotations, setErrorHandler, setName, setParent, setSilent, setSkip, setStopFlowOnError, setVariables, shallowCopy, shallowCopy, silentTipText, sizeOf, skipTipText, stopExecution, stopFlowOnErrorTipText, updateDetectedVariables, updatePrefix, updateVariables, variableChanged

      • Methods inherited from class adams.core.option.AbstractOptionHandler

        cleanUpOptions, finishInit, getDefaultLoggingLevel, getOptionManager, loggingLevelTipText, newOptionManager, setLoggingLevel, toCommandLine, toString

      • Methods inherited from class adams.core.logging.LoggingObject

        getLogger, getLoggingLevel, initializeLogging, isLoggingEnabled

      • Methods inherited from interface adams.flow.core.Actor

        compareTo, destroy, equals, findVariables, getAnnotations, getDefaultName, getDetectedVariables, getErrorHandler, getFlowExecutionListeningSupporter, getFullName, getName, getNextSibling, getParent, getParentComponent, getPreviousSibling, getRoot, getScopeHandler, getSilent, getSkip, getStopFlowOnError, getStopMessage, getStorageHandler, getVariables, handleError, hasErrorHandler, hasStopMessage, index, isExecuted, isFinished, isHeadless, isStopped, setAnnotations, setErrorHandler, setName, setParent, setSilent, setSkip, setStopFlowOnError, setVariables, shallowCopy, shallowCopy, sizeOf, stopExecution, toCommandLine, variableChanged

      • Methods inherited from interface adams.core.AdditionalInformationHandler

        getAdditionalInformation

      • Methods inherited from interface adams.core.logging.LoggingLevelHandler

        getLoggingLevel, setLoggingLevel

      • Methods inherited from interface adams.core.logging.LoggingSupporter

        getLogger, isLoggingEnabled

      • Methods inherited from interface adams.core.option.OptionHandler

        cleanUpOptions, getOptionManager

      • Methods inherited from interface adams.core.VariablesInspectionHandler

        canInspectOptions

    • Field Detail

      • m_Train

        protected adams.flow.core.CallableActorReference m_Train
        the callable actor to obtain the training dataset from.

      • m_Test

        protected adams.flow.core.CallableActorReference m_Test
        the callable actor to obtain the test dataset for train/test evaluation from.

      • m_Max

        protected int m_Max
        the maximum number of top-ranked classifiers to forward.

      • m_Seed

        protected long m_Seed
        the random seed to use.

      • m_Folds

        protected int m_Folds
        the number of folds to use in cross-validation.

      • m_ClassLabel

        protected WekaLabelIndex m_ClassLabel
        the class label (in case of class-specific measures).

      • m_OutputBestSetup

        protected boolean m_OutputBestSetup
        whether to output the best setup in case of GridSearch/MultiSearch.

      • m_NumThreads

        protected int m_NumThreads
        the number of threads to use for parallel execution.

      • m_Helper

        protected adams.flow.core.CallableActorHelper m_Helper
        the helper class.

      • m_JobRunnerSetup

        protected transient adams.flow.standalone.JobRunnerSetup m_JobRunnerSetup
        the jobrunner setup.

      • m_JobRunner

        protected adams.multiprocess.JobRunner m_JobRunner
        the job runner for evaluating the setups.

    • Constructor Detail

      • WekaClassifierRanker

        public WekaClassifierRanker()

    • Method Detail

      • globalInfo

        public String globalInfo()
        Returns a string describing the object.
        Specified by:
        globalInfo in interface adams.core.GlobalInfoSupporter
        Specified by:
        globalInfo in class adams.core.option.AbstractOptionHandler
        Returns:
        a description suitable for displaying in the gui

      • defineOptions

        public void defineOptions()
        Adds options to the internal list of options.
        Specified by:
        defineOptions in interface adams.core.option.OptionHandler
        Overrides:
        defineOptions in class adams.flow.core.AbstractActor

      • initialize

        protected void initialize()
        Initializes the members.
        Overrides:
        initialize in class adams.flow.core.AbstractActor

      • getQuickInfo

        public String getQuickInfo()
        Returns a quick info about the actor, which will be displayed in the GUI.
        Specified by:
        getQuickInfo in interface adams.flow.core.Actor
        Specified by:
        getQuickInfo in interface adams.core.QuickInfoSupporter
        Overrides:
        getQuickInfo in class adams.flow.core.AbstractActor
        Returns:
        null if no info available, otherwise short string

      • setMax

        public void setMax​(int value)
        Sets the maximum number of top-ranked classifiers to forward.
        Parameters:
        value - the maximum number, use -1 for all

      • getMax

        public int getMax()
        Returns the maximum number of top-ranked classifiers to forward.
        Returns:
        the maximum number, -1 if all returned

      • maxTipText

        public String maxTipText()
        Returns the tip text for this property.
        Returns:
        tip text for this property suitable for displaying in the GUI or for listing the options.

      • setSeed

        public void setSeed​(long value)
        Sets the seed value.
        Specified by:
        setSeed in interface adams.core.Randomizable
        Parameters:
        value - the seed

      • getSeed

        public long getSeed()
        Returns the seed value.
        Specified by:
        getSeed in interface adams.core.Randomizable
        Returns:
        the seed

      • seedTipText

        public String seedTipText()
        Returns the tip text for this property.
        Specified by:
        seedTipText in interface adams.core.Randomizable
        Returns:
        tip text for this property suitable for displaying in the GUI or for listing the options.

      • setFolds

        public void setFolds​(int value)
        Sets the number of folds to use.
        Parameters:
        value - the folds

      • getFolds

        public int getFolds()
        Returns the number of folds to use.
        Returns:
        the folds

      • foldsTipText

        public String foldsTipText()
        Returns the tip text for this property.
        Returns:
        tip text for this property suitable for displaying in the GUI or for listing the options.

      • setMeasure

        public void setMeasure​(WekaClassifierRanker.Measure value)
        Sets the measure to use for ranking the classifiers.
        Parameters:
        value - the ranking measure

      • getMeasure

        public WekaClassifierRanker.Measure getMeasure()
        Returns the measure used for ranking the classifiers.
        Returns:
        the ranking measure

      • measureTipText

        public String measureTipText()
        Returns the tip text for this property.
        Returns:
        tip text for this property suitable for displaying in the GUI or for listing the options.

      • setClassLabel

        public void setClassLabel​(WekaLabelIndex value)
        Sets the class label index to use for class-specific measures.
        Parameters:
        value - the label index

      • getClassLabel

        public WekaLabelIndex getClassLabel()
        Returns the class label index to use for class-specific measures.
        Returns:
        the label index

      • classLabelTipText

        public String classLabelTipText()
        Returns the tip text for this property.
        Returns:
        tip text for this property suitable for displaying in the GUI or for listing the options.

      • setTrain

        public void setTrain​(adams.flow.core.CallableActorReference value)
        Sets the name of the callable actor to obtain the training set.
        Parameters:
        value - the name of the callable actor

      • getTrain

        public adams.flow.core.CallableActorReference getTrain()
        Returns the name of the callable actor to obtain the training set.
        Returns:
        the name of the callable actor

      • trainTipText

        public String trainTipText()
        Returns the tip text for this property.
        Returns:
        tip text for this property suitable for displaying in the GUI or for listing the options.

      • setTest

        public void setTest​(adams.flow.core.CallableActorReference value)
        Sets the name of the callable actor to obtain the test set.
        Parameters:
        value - the name of the callable actor

      • getTest

        public adams.flow.core.CallableActorReference getTest()
        Returns the name of the callable actor to obtain the test set.
        Returns:
        the name of the callable actor

      • testTipText

        public String testTipText()
        Returns the tip text for this property.
        Returns:
        tip text for this property suitable for displaying in the GUI or for listing the options.

      • setOutputBestSetup

        public void setOutputBestSetup​(boolean value)
        Sets whether to output the best setup found for optimizers like GridSearch and MultiSearch.
        Parameters:
        value - true if the best setup is to be output

      • getOutputBestSetup

        public boolean getOutputBestSetup()
        Returns whether to output the best setup for optimizers like GridSearch and MultiSearch.
        Returns:
        true if the best setup is output

      • outputBestSetupTipText

        public String outputBestSetupTipText()
        Returns the tip text for this property.
        Returns:
        tip text for this property suitable for displaying in the GUI or for listing the options.

      • setNumThreads

        public void setNumThreads​(int value)
        Sets the number of threads to use.
        Specified by:
        setNumThreads in interface adams.core.ThreadLimiter
        Parameters:
        value - the number of threads

      • getNumThreads

        public int getNumThreads()
        Returns the number of threads in use.
        Specified by:
        getNumThreads in interface adams.core.ThreadLimiter
        Returns:
        the number of threads

      • numThreadsTipText

        public String numThreadsTipText()
        Returns the tip text for this property.
        Returns:
        tip text for this property suitable for displaying in the GUI or for listing the options.

      • setUp

        public String setUp()
        Initializes the item for flow execution.
        Specified by:
        setUp in interface adams.flow.core.Actor
        Overrides:
        setUp in class adams.flow.core.AbstractActor
        Returns:
        null if everything is fine, otherwise error message

      • accepts

        public Class[] accepts()
        Returns the class that the consumer accepts.
        Specified by:
        accepts in interface adams.flow.core.InputConsumer
        Returns:
        weka.classifiers.Classifier[].class

      • generates

        public Class[] generates()
        Returns the class of objects that it generates.
        Specified by:
        generates in interface adams.flow.core.OutputProducer
        Returns:
        weka.classifiers.Classifier[].class

      • doExecute

        protected String doExecute()
        Executes the flow item.
        Specified by:
        doExecute in class adams.flow.core.AbstractActor
        Returns:
        null if everything is fine, otherwise error message

      • flowPauseStateChanged

        public void flowPauseStateChanged​(adams.event.FlowPauseStateEvent e)
        Gets called when the pause state of the flow changes.
        Specified by:
        flowPauseStateChanged in interface adams.event.FlowPauseStateListener
        Parameters:
        e - the event

      • pauseExecution

        public void pauseExecution()
        Pauses the execution.
        Specified by:
        pauseExecution in interface adams.core.Pausable

      • isPaused

        public boolean isPaused()
        Returns whether the object is currently paused.
        Specified by:
        isPaused in interface adams.core.Pausable
        Returns:
        true if object is paused

      • resumeExecution

        public void resumeExecution()
        Resumes the execution.
        Specified by:
        resumeExecution in interface adams.core.Pausable

      • stopExecution

        public void stopExecution()
        Stops the execution.
        Specified by:
        stopExecution in interface adams.flow.core.Actor
        Specified by:
        stopExecution in interface adams.core.Stoppable
        Overrides:
        stopExecution in class adams.flow.core.AbstractActor

      • cleanUp

        public void cleanUp()
        Frees up memory in a "destructive" non-reversible way.
        Specified by:
        cleanUp in interface adams.flow.core.Actor
        Specified by:
        cleanUp in interface adams.core.CleanUpHandler
        Overrides:
        cleanUp in class adams.flow.core.AbstractActor