adams.flow.transformer
Class WekaClassifierRanker

java.lang.Object
  adams.core.ConsoleObject
      adams.core.option.AbstractOptionHandler
          adams.flow.core.AbstractActor
              adams.flow.transformer.AbstractTransformer
                  adams.flow.transformer.WekaClassifierRanker

All Implemented Interfaces:

AdditionalInformationHandler, CleanUpHandler, Debuggable, DebugOutputHandler, Destroyable, OptionHandler, Pausable, QuickInfoSupporter, Randomizable, ShallowCopySupporter<AbstractActor>, SizeOfHandler, Stoppable, FlowPauseStateListener, VariableChangeListener, ErrorHandler, InputConsumer, OutputProducer, Serializable, Comparable

public class WekaClassifierRanker
extends AbstractTransformer
implements Randomizable, Pausable, FlowPauseStateListener

Performs a quick evaluation using cross-validation on a single dataset (or evaluation on a separate test set if the number of folds is less than 2) to rank the classifiers received on the input and forwarding the x best ones. Further evaluation can be performed using the Experimenter.

Input/output:
- accepts:
weka.classifiers.Classifier[]
- generates:
weka.classifiers.Classifier[]

Valid options are:

-D <int> (property: debugLevel)
    The greater the number the more additional info the scheme may output to
    the console (0 = off).
    default: 0
    minimum: 0
 

-name <java.lang.String> (property: name)
    The name of the actor.
    default: WekaClassifierRanker
 

-annotation <adams.core.base.BaseText> (property: annotations)
    The annotations to attach to this actor.
    default:
 

-skip (property: skip)
    If set to true, transformation is skipped and the input token is just forwarded
    as it is.
 

-stop-flow-on-error (property: stopFlowOnError)
    If set to true, the flow gets stopped in case this actor encounters an error;
     useful for critical actors.
 

-max <int> (property: max)
    The maximum number of top-ranked classifiers to forward; use -1 to forward
    all of them (ranked array).
    default: 3
    minimum: -1
 

-seed <long> (property: seed)
    The seed value to use in the cross-validation.
    default: 1
 

-folds <int> (property: folds)
    The number of folds to use in cross-validation.
    default: 10
    minimum: 1
 

-measure <Correlation coefficient|Root mean squared error|Root relative squared error|Mean absolute error|Root absolute error|Combined: (1-abs(CC)) + RRSE + RAE|Accuracy|Kapp> (property: measure)
    The measure used for ranking the classifiers.
    default: CC
 

-train <adams.flow.core.GlobalActorReference> (property: train)
    The name of the global actor that is used for obtaining the training set.
    default: train
 

-test <adams.flow.core.GlobalActorReference> (property: test)
    The name of the global actor that is used for obtaining the test set (only
    if folds <2).
    default: test
 

-output-best (property: outputBestSetup)
    If true, then for optimizers like GridSearch and MultiSearch the best setup
    that was found will be output instead of the optimizer setup.
 

-num-threads <int> (property: numThreads)
    The number of threads to use for evaluating the classifiers in parallel
    (-1 means one for each core/cpu).
    default: -1
    minimum: -1
 

Version:

$Revision: 4665 $

Author:

fracpete (fracpete at waikato dot ac dot nz)

See Also:

Serialized Form

Nested Class Summary

static class	WekaClassifierRanker.Measure The performance measure to use.
static class	WekaClassifierRanker.RankingJob A job class specific to ranking classifiers.

Field Summary

protected int	m_Folds the number of folds to use in cross-validation.
protected GlobalActorHelper	m_Helper the helper class.
protected JobRunner<WekaClassifierRanker.RankingJob>	m_JobRunner the job runner for evaluating the setups.
protected int	m_Max the maximum number of top-ranked classifiers to forward.
protected WekaClassifierRanker.Measure	m_Measure the measure for the evaluation.
protected int	m_NumThreads the number of threads to use for parallel execution.
protected boolean	m_OutputBestSetup whether to output the best setup in case of GridSearch/MultiSearch.
protected long	m_Seed the random seed to use.
protected GlobalActorReference	m_Test the global actor to obtain the test dataset for train/test evaluation from.
protected GlobalActorReference	m_Train the global actor to obtain the training dataset from.

Fields inherited from class adams.flow.transformer.AbstractTransformer

BACKUP_INPUT, BACKUP_OUTPUT, m_InputToken, m_OutputToken

Fields inherited from class adams.flow.core.AbstractActor

FILE_EXTENSION, FILE_EXTENSION_GZ, m_Annotations, m_BackupState, m_DetectedObjectVariables, m_DetectedVariables, m_ErrorHandler, m_Executed, m_FullName, m_Headless, m_Name, m_Parent, m_Root, m_Self, m_Skip, m_StopFlowOnError, m_StopMessage, m_Stopped, m_StorageHandler, m_VariablesUpdated

Fields inherited from class adams.core.option.AbstractOptionHandler

m_DebugLevel, m_OptionManager

Constructor Summary

WekaClassifierRanker()

Method Summary

Class[]	accepts() Returns the class that the consumer accepts.
void	cleanUp() Frees up memory in a "destructive" non-reversible way.
void	defineOptions() Adds options to the internal list of options.
protected String	doExecute() Executes the flow item.
void	flowPauseStateChanged(FlowPauseStateEvent e) Gets called when the pause state of the flow changes.
String	foldsTipText() Returns the tip text for this property.
Class[]	generates() Returns the class of objects that it generates.
int	getFolds() Returns the number of folds to use.
int	getMax() Returns the maximum number of top-ranked classifiers to forward.
WekaClassifierRanker.Measure	getMeasure() Returns the measure used for ranking the classifiers.
int	getNumThreads() Returns the number of threads in use.
boolean	getOutputBestSetup() Returns whether to output the best setup for optimizers like GridSearch and MultiSearch.
String	getQuickInfo() Returns a quick info about the actor, which will be displayed in the GUI.
long	getSeed() Returns the seed value.
GlobalActorReference	getTest() Returns the name of the global actor to obtain the test set.
GlobalActorReference	getTrain() Returns the name of the global actor to obtain the training set.
String	globalInfo() Returns a string describing the object.
protected void	initialize() Initializes the members.
boolean	isPaused() Returns whether the object is currently paused.
String	maxTipText() Returns the tip text for this property.
String	measureTipText() Returns the tip text for this property.
String	numThreadsTipText() Returns the tip text for this property.
String	outputBestSetupTipText() Returns the tip text for this property.
void	pauseExecution() Pauses the execution.
void	resumeExecution() Resumes the execution.
String	seedTipText() Returns the tip text for this property.
void	setFolds(int value) Sets the number of folds to use.
void	setMax(int value) Sets the maximum number of top-ranked classifiers to forward.
void	setMeasure(WekaClassifierRanker.Measure value) Sets the measure to use for ranking the classifiers.
void	setNumThreads(int value) Sets the number of threads to use.
void	setOutputBestSetup(boolean value) Sets whether to output the best setup found for optimizers like GridSearch and MultiSearch.
void	setSeed(long value) Sets the seed value.
void	setTest(GlobalActorReference value) Sets the name of the global actor to obtain the test set.
void	setTrain(GlobalActorReference value) Sets the name of the global actor to obtain the training set.
String	setUp() Initializes the item for flow execution.
void	stopExecution() Stops the execution.
String	testTipText() Returns the tip text for this property.
String	trainTipText() Returns the tip text for this property.

Methods inherited from class adams.flow.transformer.AbstractTransformer

backupState, execute, hasPendingOutput, input, output, postExecute, reset, restoreState, wrapUp

Methods inherited from class adams.flow.core.AbstractActor

annotationsTipText, canInspectOptions, canPerformSetUpCheck, compareTo, debug, destroy, equals, findVariables, findVariables, findVariables, forceVariables, forCommandLine, forName, getAdditionalInformation, getAnnotations, getDefaultName, getDetectedVariables, getErrorHandler, getFlowActors, getFullName, getName, getNextSibling, getParent, getPreviousSibling, getRoot, getSkip, getStopFlowOnError, getStopMessage, getStorageHandler, getVariables, handleError, hasErrorHandler, hasStopMessage, index, isBackedUp, isExecuted, isFinished, isHeadless, isStopped, nameTipText, performSetUpChecks, preExecute, pruneBackup, pruneBackup, setAnnotations, setErrorHandler, setHeadless, setName, setParent, setSkip, setStopFlowOnError, setVariables, shallowCopy, shallowCopy, sizeOf, skipTipText, stopExecution, stopFlowOnErrorTipText, updateDetectedVariables, updatePrefix, updateVariables, variableChanged

Methods inherited from class adams.core.option.AbstractOptionHandler

cleanUpOptions, debug, debugLevelTipText, finishInit, getDebugLevel, getOptionManager, isDebugOn, newOptionManager, setDebugLevel, toCommandLine, toString

Methods inherited from class adams.core.ConsoleObject

getDebugging, getSystemErr, getSystemOut

Methods inherited from class java.lang.Object

clone, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

m_Train

protected GlobalActorReference m_Train

the global actor to obtain the training dataset from.

m_Test

protected GlobalActorReference m_Test

the global actor to obtain the test dataset for train/test evaluation from.

m_Max

protected int m_Max

the maximum number of top-ranked classifiers to forward.

m_Seed

protected long m_Seed

the random seed to use.

m_Folds

protected int m_Folds

the number of folds to use in cross-validation.

m_Measure

protected WekaClassifierRanker.Measure m_Measure

the measure for the evaluation.

m_OutputBestSetup

protected boolean m_OutputBestSetup

whether to output the best setup in case of GridSearch/MultiSearch.

m_NumThreads

protected int m_NumThreads

the number of threads to use for parallel execution.

m_Helper

protected GlobalActorHelper m_Helper

the helper class.

m_JobRunner

protected JobRunner<WekaClassifierRanker.RankingJob> m_JobRunner

the job runner for evaluating the setups.

Constructor Detail

WekaClassifierRanker

public WekaClassifierRanker()

Method Detail

globalInfo

public String globalInfo()

Returns a string describing the object.

Specified by:

globalInfo in class AbstractOptionHandler

Returns:

a description suitable for displaying in the gui

defineOptions

public void defineOptions()

Adds options to the internal list of options.

Specified by:

defineOptions in interface OptionHandler

Overrides:

defineOptions in class AbstractActor

initialize

protected void initialize()

Initializes the members.

Overrides:

initialize in class AbstractActor

getQuickInfo

public String getQuickInfo()

Returns a quick info about the actor, which will be displayed in the GUI.

Specified by:

getQuickInfo in interface QuickInfoSupporter

Overrides:

getQuickInfo in class AbstractActor

Returns:

null if no info available, otherwise short string

setMax

public void setMax(int value)

Sets the maximum number of top-ranked classifiers to forward.

Parameters:

value - the maximum number, use -1 for all

getMax

public int getMax()

Returns the maximum number of top-ranked classifiers to forward.

Returns:

the maximum number, -1 if all returned

maxTipText

public String maxTipText()

Returns the tip text for this property.

Returns:

tip text for this property suitable for displaying in the GUI or for listing the options.

setSeed

public void setSeed(long value)

Sets the seed value.

Specified by:

setSeed in interface Randomizable

Parameters:

value - the seed

getSeed

public long getSeed()

Returns the seed value.

Specified by:

getSeed in interface Randomizable

Returns:

the seed

seedTipText

public String seedTipText()

Returns the tip text for this property.

Specified by:

seedTipText in interface Randomizable

Returns:

tip text for this property suitable for displaying in the GUI or for listing the options.

setFolds

public void setFolds(int value)

Sets the number of folds to use.

Parameters:

value - the folds

getFolds

public int getFolds()

Returns the number of folds to use.

Returns:

the folds

foldsTipText

public String foldsTipText()

Returns the tip text for this property.

Returns:

tip text for this property suitable for displaying in the GUI or for listing the options.

setMeasure

public void setMeasure(WekaClassifierRanker.Measure value)

Sets the measure to use for ranking the classifiers.

Parameters:

value - the ranking measure

getMeasure

public WekaClassifierRanker.Measure getMeasure()

Returns the measure used for ranking the classifiers.

Returns:

the ranking measure

measureTipText

public String measureTipText()

Returns the tip text for this property.

Returns:

tip text for this property suitable for displaying in the GUI or for listing the options.

setTrain

public void setTrain(GlobalActorReference value)

Sets the name of the global actor to obtain the training set.

Parameters:

value - the name of the global actor

getTrain

public GlobalActorReference getTrain()

Returns the name of the global actor to obtain the training set.

Returns:

the name of the global actor

trainTipText

public String trainTipText()

Returns the tip text for this property.

Returns:

tip text for this property suitable for displaying in the GUI or for listing the options.

setTest

public void setTest(GlobalActorReference value)

Sets the name of the global actor to obtain the test set.

Parameters:

value - the name of the global actor

getTest

public GlobalActorReference getTest()

Returns the name of the global actor to obtain the test set.

Returns:

the name of the global actor

testTipText

public String testTipText()

Returns the tip text for this property.

Returns:

tip text for this property suitable for displaying in the GUI or for listing the options.

setOutputBestSetup

public void setOutputBestSetup(boolean value)

Sets whether to output the best setup found for optimizers like GridSearch and MultiSearch.

Parameters:

value - true if the best setup is to be output

getOutputBestSetup

public boolean getOutputBestSetup()

Returns whether to output the best setup for optimizers like GridSearch and MultiSearch.

Returns:

true if the best setup is output

outputBestSetupTipText

public String outputBestSetupTipText()

Returns the tip text for this property.

Returns:

tip text for this property suitable for displaying in the GUI or for listing the options.

setNumThreads

public void setNumThreads(int value)

Sets the number of threads to use.

Parameters:

value - the number of threads

getNumThreads

public int getNumThreads()

Returns the number of threads in use.

Returns:

the number of threads

numThreadsTipText

public String numThreadsTipText()

Returns the tip text for this property.

Returns:

tip text for this property suitable for displaying in the GUI or for listing the options.

setUp

public String setUp()

Initializes the item for flow execution.

Overrides:

setUp in class AbstractActor

Returns:

null if everything is fine, otherwise error message

accepts

public Class[] accepts()

Returns the class that the consumer accepts.

Specified by:

accepts in interface InputConsumer

Returns:

weka.classifiers.Classifier[].class

generates

public Class[] generates()

Returns the class of objects that it generates.

Specified by:

generates in interface OutputProducer

Returns:

weka.classifiers.Classifier[].class

doExecute

protected String doExecute()

Executes the flow item.

Specified by:

doExecute in class AbstractActor

Returns:

null if everything is fine, otherwise error message

flowPauseStateChanged

public void flowPauseStateChanged(FlowPauseStateEvent e)

Gets called when the pause state of the flow changes.

Specified by:

flowPauseStateChanged in interface FlowPauseStateListener

Parameters:

e - the event

pauseExecution

public void pauseExecution()

Pauses the execution.

Specified by:

pauseExecution in interface Pausable

isPaused

public boolean isPaused()

Returns whether the object is currently paused.

Specified by:

isPaused in interface Pausable

Returns:

true if object is paused

resumeExecution

public void resumeExecution()

Resumes the execution.

Specified by:

resumeExecution in interface Pausable

stopExecution

public void stopExecution()

Stops the execution.

Specified by:

stopExecution in interface Stoppable

Overrides:

stopExecution in class AbstractActor

cleanUp

public void cleanUp()

Frees up memory in a "destructive" non-reversible way.

Specified by:

cleanUp in interface CleanUpHandler

Overrides:

cleanUp in class AbstractActor