java.lang.Object
- adams.core.logging.LoggingObject
- - adams.core.logging.CustomLoggingLevelObject
  - - adams.core.option.AbstractOptionHandler
    - - adams.gui.tools.wekamultiexperimenter.experiment.AbstractExperiment

All Implemented Interfaces:

Destroyable, GlobalInfoSupporter, LoggingLevelHandler, LoggingSupporter, OptionHandler, SizeOfHandler, Stoppable, StoppableWithFeedback, SpreadSheetSupporter, ExperimentWithCustomizableRelationNames, ResettableExperiment, Serializable

Direct Known Subclasses:

CrossValidationExperiment, TrainTestSplitExperiment
```
public abstract class AbstractExperiment
extends AbstractOptionHandler
implements StoppableWithFeedback, ExperimentWithCustomizableRelationNames, ResettableExperiment, SpreadSheetSupporter
```
Ancestor for simple experiments.

Version:

$Revision$

Author:

FracPete (fracpete at waikato dot ac dot nz)

See Also:

Serialized Form

Nested Class Summary

Nested Classes
Modifier and Type	Class	Description
`static class`	`AbstractExperiment.AbstractExperimentJob<T extends AbstractExperiment>`	For evaluating a single classifier/dataset combination.

Field Summary

Fields
Modifier and Type	Field	Description
`protected JobRunner`	`m_ActualJobRunner`	JobRunner for the classifier/dataset combinations.
`protected AbstractClassAttributeHeuristic`	`m_ClassAttribute`	how to determine the class attribute.
`protected weka.classifiers.Classifier[]`	`m_Classifiers`	the classifiers to evaluate.
`protected Index`	`m_ClassLabelIndex`	the class label index for per-class stats.
`protected WekaCommandLineHandler`	`m_CommandLineHandler`	for handling commandlines.
`protected PlaceholderFile[]`	`m_Datasets`	the datasets to evaluate.
`protected List<SpreadSheet>`	`m_Generated`	the results generated by the evaluations.
`protected int`	`m_JobCounter`	the counter for finished jobs.
`protected JobRunner`	`m_JobRunner`	the JobRunner template.
`protected int`	`m_JobTotal`	the total number of jobs.
`protected BaseText`	`m_Notes`	the notes for the experiment.
`protected boolean`	`m_PrefixDatasetsWithIndex`	whether to prefix the relation names with the index.
`protected boolean`	`m_ResetResults`	whether to reset the results before starting the experiment.
`protected SpreadSheet`	`m_Results`	the generated results.
`protected AbstractResultsHandler`	`m_ResultsHandler`	the handler for the results.
`protected boolean`	`m_Running`	whether the experiment is running.
`protected int`	`m_Runs`	the number of runs.
`protected StatusMessageHandler`	`m_StatusMessageHandler`	for notifications.
`protected boolean`	`m_Stopped`	whether the experiment was stopped.
`protected boolean`	`m_UseFilename`	whether to use the filename (w/o path) instead of relationname.

Fields inherited from class adams.core.option.AbstractOptionHandler
m_OptionManager

Fields inherited from class adams.core.logging.LoggingObject
m_Logger, m_LoggingIsEnabled, m_LoggingLevel

Constructor Summary

Constructors
Constructor Description

AbstractExperiment()

Method Summary

All Methods Static Methods Instance Methods Abstract Methods Concrete Methods
Modifier and Type	Method	Description
`void`	`addClassifier(weka.classifiers.Classifier cls)`	Adds a classifier.
`void`	`addDataset(PlaceholderFile file)`	Adds a dataset.
`void`	`appendResults(SpreadSheet results)`	Adds the results to the existing ones.
`String`	`classAttributeTipText()`	Returns the tip text for this property.
`String`	`classifiersTipText()`	Returns the tip text for this property.
`String`	`classLabelIndexTipText()`	Returns the tip text for this property.
`protected void`	`clearProgress()`	Clears the progress of the experiment.
`protected MultiRowFinder`	`configureRowFinder(int currentRun, weka.classifiers.Classifier cls, weka.core.Instances data)`	Configures the row finder that determines whether the classifier/dataset combination is still required.
`String`	`datasetsTipText()`	Returns the tip text for this property.
`void`	`defineOptions()`	Adds options to the internal list of options.
`protected String`	`doExecute()`	Runs the actual experiment.
`protected abstract AbstractExperiment.AbstractExperimentJob<? extends AbstractExperiment>`	`evaluate(int currentRun, weka.classifiers.Classifier cls, weka.core.Instances data)`	Creates a runnabel to evaluate the classifier on the dataset.
`String`	`execute()`	Executes the experiment.
`AbstractClassAttributeHeuristic`	`getClassAttribute()`	Returns the heuristic for determining the class attribute (if not explicitly set).
`weka.classifiers.Classifier[]`	`getClassifiers()`	Returns the classifiers.
`Index`	`getClassLabelIndex()`	Returns the index of the class label to use when generating per-class statistics.
`PlaceholderFile[]`	`getDatasets()`	The datasets in use.
`JobRunner`	`getJobRunner()`	Returns the jobrunner for the experiment.
`BaseText`	`getNotes()`	Returns the notes for the experiment.
`boolean`	`getPrefixDatasetsWithIndex()`	Returns whether to prefix the datasets with the index.
`boolean`	`getResetResults()`	Returns whether to clear the results before starting the experiment.
`AbstractResultsHandler`	`getResultsHandler()`	Returns the results handler to use.
`int`	`getRuns()`	Returns the number of runs.
`StatusMessageHandler`	`getStatusMessageHandler()`	Returns the current status message handler in use.
`boolean`	`getUseFilename()`	Returns whether to use the filename (w/o path) instead of the relationname.
`void`	`incProgress()`	Increments and updates the progress.
`protected String`	`initExecute()`	Initializes the experiment.
`void`	`initProgress()`	Initializes progress.
`protected SpreadSheet`	`initResults()`	Initializes the results.
`protected boolean`	`isComplete(int[] rows)`	Checks whether the number of rows located in the current results are complete.
`protected boolean`	`isRequired(int currentRun, weka.classifiers.Classifier cls, weka.core.Instances data)`	Checks whether the classifier/dataset combination is required.
`boolean`	`isStopped()`	Returns whether the experiment has been stopped.
`String`	`jobRunnerTipText()`	Returns the tip text for this property.
`protected weka.core.Instances`	`loadDataset(int index)`	Loads the dataset.
`protected void`	`log(String msg)`	Displays the message.
`protected void`	`log(String msg, Throwable t)`	Displays the error.
`String`	`notesTipText()`	Returns the tip text for this property.
`protected void`	`postExecute(boolean success)`	Hook method just after the experiment was run.
`protected String`	`preExecute()`	Hook method just before the experiment is run (after initialization).
`String`	`prefixDatasetsWithIndexTipText()`	Returns the tip text for this property.
`protected void`	`removeIncomplete(int currentRun, weka.classifiers.Classifier cls, weka.core.Instances data)`	Removes the incomplete rows of the classifier/dataset combination.
`String`	`resetResultsTipText()`	Returns the tip text for this property.
`String`	`resultWriterTipText()`	Returns the tip text for this property.
`String`	`runsTipText()`	Returns the tip text for this property.
`void`	`setClassAttribute(AbstractClassAttributeHeuristic value)`	Sets the heuristic for determining the class attribute (if not explicitly set).
`void`	`setClassifiers(weka.classifiers.Classifier[] value)`	Sets the classifiers to use.
`void`	`setClassLabelIndex(Index value)`	Sets the index of the class label to use when generating per-class statistics.
`void`	`setDatasets(PlaceholderFile[] value)`	Sets the datasets to use.
`void`	`setJobRunner(JobRunner value)`	Sets the jobrunner for the experiment.
`void`	`setNotes(BaseText value)`	Sets the notes for the experiment.
`void`	`setPrefixDatasetsWithIndex(boolean value)`	Sets whether to prefix the datasets with the index.
`void`	`setResetResults(boolean value)`	Sets whether to clear the results before starting the experiment.
`void`	`setResultsHandler(AbstractResultsHandler value)`	Sets the results handler to use.
`void`	`setRuns(int value)`	Sets the number of runs.
`void`	`setStatusMessageHandler(StatusMessageHandler value)`	Sets the status message handler to use.
`void`	`setUseFilename(boolean value)`	Sets whether to use the filename (w/o path) instead of the relationname.
`static String`	`shortenCommandLine(weka.classifiers.Classifier cls)`	Returns a shortened commandline string for the classifier.
`protected void`	`showProgress()`	Updates the progress of the experiment.
`void`	`stopExecution()`	Stops the execution.
`weka.core.Instances`	`toInstances()`	Returns the collected results.
`SpreadSheet`	`toSpreadSheet()`	Returns the collected results.
`String`	`useFilenameTipText()`	Returns the tip text for this property.

Methods inherited from class adams.core.option.AbstractOptionHandler
cleanUpOptions, destroy, finishInit, getDefaultLoggingLevel, getOptionManager, globalInfo, initialize, loggingLevelTipText, newOptionManager, reset, setLoggingLevel, toCommandLine, toString

Methods inherited from class adams.core.logging.LoggingObject
configureLogger, getLogger, getLoggingLevel, initializeLogging, isLoggingEnabled, sizeOf

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface adams.core.logging.LoggingLevelHandler
getLoggingLevel

- Field Detail
  - m_ResetResults
```
protected boolean m_ResetResults
```
    whether to reset the results before starting the experiment.
  - m_Classifiers
```
protected weka.classifiers.Classifier[] m_Classifiers
```
    the classifiers to evaluate.
  - m_Datasets
```
protected PlaceholderFile[] m_Datasets
```
    the datasets to evaluate.
  - m_ClassAttribute
```
protected AbstractClassAttributeHeuristic m_ClassAttribute
```
    how to determine the class attribute.
  - m_ClassLabelIndex
```
protected Index m_ClassLabelIndex
```
    the class label index for per-class stats.
  - m_UseFilename
```
protected boolean m_UseFilename
```
    whether to use the filename (w/o path) instead of relationname.
  - m_PrefixDatasetsWithIndex
```
protected boolean m_PrefixDatasetsWithIndex
```
    whether to prefix the relation names with the index.
  - m_Runs
```
protected int m_Runs
```
    the number of runs.
  - m_ResultsHandler
```
protected AbstractResultsHandler m_ResultsHandler
```
    the handler for the results.
  - m_Notes
```
protected BaseText m_Notes
```
    the notes for the experiment.
  - m_StatusMessageHandler
```
protected transient StatusMessageHandler m_StatusMessageHandler
```
    for notifications.
  - m_Running
```
protected transient boolean m_Running
```
    whether the experiment is running.
  - m_Stopped
```
protected transient boolean m_Stopped
```
    whether the experiment was stopped.
  - m_CommandLineHandler
```
protected transient WekaCommandLineHandler m_CommandLineHandler
```
    for handling commandlines.
  - m_Results
```
protected SpreadSheet m_Results
```
    the generated results.
  - m_Generated
```
protected List<SpreadSheet> m_Generated
```
    the results generated by the evaluations.
  - m_JobRunner
```
protected JobRunner m_JobRunner
```
    the JobRunner template.
  - m_ActualJobRunner
```
protected transient JobRunner m_ActualJobRunner
```
    JobRunner for the classifier/dataset combinations.
  - m_JobCounter
```
protected int m_JobCounter
```
    the counter for finished jobs.
  - m_JobTotal
```
protected int m_JobTotal
```
    the total number of jobs.
- Constructor Detail
  - AbstractExperiment
```
public AbstractExperiment()
```
- Method Detail
  - defineOptions
```
public void defineOptions()
```
    Adds options to the internal list of options.
    
    Specified by:
    
    defineOptions in interface OptionHandler
    
    Overrides:
    
    defineOptions in class AbstractOptionHandler
  - setResetResults
```
public void setResetResults(boolean value)
```
    Sets whether to clear the results before starting the experiment.
    
    Specified by:
    
    setResetResults in interface ResettableExperiment
    
    Parameters:
    
    value - true if to clear results
  - getResetResults
```
public boolean getResetResults()
```
    Returns whether to clear the results before starting the experiment.
    
    Specified by:
    
    getResetResults in interface ResettableExperiment
    
    Returns:
    
    true if to clear results
  - resetResultsTipText
```
public String resetResultsTipText()
```
    Returns the tip text for this property.
    
    Returns:
    
    tip text for this property suitable for displaying in the GUI or for listing the options.
  - setResultsHandler
```
public void setResultsHandler(AbstractResultsHandler value)
```
    Sets the results handler to use.
    
    Parameters:
    
    value - the handler
  - getResultsHandler
```
public AbstractResultsHandler getResultsHandler()
```
    Returns the results handler to use.
    
    Returns:
    
    the handler
  - resultWriterTipText
```
public String resultWriterTipText()
```
    Returns the tip text for this property.
    
    Returns:
    
    tip text for this property suitable for displaying in the GUI or for listing the options.
  - setClassifiers
```
public void setClassifiers(weka.classifiers.Classifier[] value)
```
    Sets the classifiers to use.
    
    Parameters:
    
    value - the classifiers
  - getClassifiers
```
public weka.classifiers.Classifier[] getClassifiers()
```
    Returns the classifiers.
    
    Returns:
    
    the classifiers
  - classifiersTipText
```
public String classifiersTipText()
```
    Returns the tip text for this property.
    
    Returns:
    
    tip text for this property suitable for displaying in the GUI or for listing the options.
  - addClassifier
```
public void addClassifier(weka.classifiers.Classifier cls)
```
    Adds a classifier.
    
    Parameters:
    
    cls - the classifier to add
  - setDatasets
```
public void setDatasets(PlaceholderFile[] value)
```
    Sets the datasets to use.
    
    Parameters:
    
    value - the datasets
  - getDatasets
```
public PlaceholderFile[] getDatasets()
```
    The datasets in use.
    
    Returns:
    
    the datasets
  - datasetsTipText
```
public String datasetsTipText()
```
    Returns the tip text for this property.
    
    Returns:
    
    tip text for this property suitable for displaying in the GUI or for listing the options.
  - addDataset
```
public void addDataset(PlaceholderFile file)
```
    Adds a dataset.
    
    Parameters:
    
    file - the dataset to add
  - setClassAttribute
```
public void setClassAttribute(AbstractClassAttributeHeuristic value)
```
    Sets the heuristic for determining the class attribute (if not explicitly set).
    
    Parameters:
    
    value - the heuristic
  - getClassAttribute
```
public AbstractClassAttributeHeuristic getClassAttribute()
```
    Returns the heuristic for determining the class attribute (if not explicitly set).
    
    Returns:
    
    the heuristic
  - classAttributeTipText
```
public String classAttributeTipText()
```
    Returns the tip text for this property.
    
    Returns:
    
    tip text for this property suitable for displaying in the GUI or for listing the options.
  - setClassLabelIndex
```
public void setClassLabelIndex(Index value)
```
    Sets the index of the class label to use when generating per-class statistics.
    
    Parameters:
    
    value - the index
  - getClassLabelIndex
```
public Index getClassLabelIndex()
```
    Returns the index of the class label to use when generating per-class statistics.
    
    Returns:
    
    the index
  - classLabelIndexTipText
```
public String classLabelIndexTipText()
```
    Returns the tip text for this property.
    
    Returns:
    
    tip text for this property suitable for displaying in the GUI or for listing the options.
  - setUseFilename
```
public void setUseFilename(boolean value)
```
    Sets whether to use the filename (w/o path) instead of the relationname.
    
    Specified by:
    
    setUseFilename in interface ExperimentWithCustomizableRelationNames
    
    Parameters:
    
    value - true if to use filename
  - getUseFilename
```
public boolean getUseFilename()
```
    Returns whether to use the filename (w/o path) instead of the relationname.
    
    Specified by:
    
    getUseFilename in interface ExperimentWithCustomizableRelationNames
    
    Returns:
    
    true if to use the filename
  - useFilenameTipText
```
public String useFilenameTipText()
```
    Returns the tip text for this property.
    
    Returns:
    
    tip text for this property suitable for displaying in the GUI or for listing the options.
  - setPrefixDatasetsWithIndex
```
public void setPrefixDatasetsWithIndex(boolean value)
```
    Sets whether to prefix the datasets with the index.
    
    Specified by:
    
    setPrefixDatasetsWithIndex in interface ExperimentWithCustomizableRelationNames
    
    Parameters:
    
    value - true if to prefix
  - getPrefixDatasetsWithIndex
```
public boolean getPrefixDatasetsWithIndex()
```
    Returns whether to prefix the datasets with the index.
    
    Specified by:
    
    getPrefixDatasetsWithIndex in interface ExperimentWithCustomizableRelationNames
    
    Returns:
    
    true if to prefix
  - prefixDatasetsWithIndexTipText
```
public String prefixDatasetsWithIndexTipText()
```
    Returns the tip text for this property.
    
    Returns:
    
    tip text for this property suitable for displaying in the GUI or for listing the options.
  - setRuns
```
public void setRuns(int value)
```
    Sets the number of runs.
    
    Parameters:
    
    value - the runs
  - getRuns
```
public int getRuns()
```
    Returns the number of runs.
    
    Returns:
    
    the runs
  - runsTipText
```
public String runsTipText()
```
    Returns the tip text for this property.
    
    Returns:
    
    tip text for this property suitable for displaying in the GUI or for listing the options.
  - setNotes
```
public void setNotes(BaseText value)
```
    Sets the notes for the experiment.
    
    Parameters:
    
    value - the notes
  - getNotes
```
public BaseText getNotes()
```
    Returns the notes for the experiment.
    
    Returns:
    
    the notes
  - notesTipText
```
public String notesTipText()
```
    Returns the tip text for this property.
    
    Returns:
    
    tip text for this property suitable for displaying in the GUI or for listing the options.
  - setJobRunner
```
public void setJobRunner(JobRunner value)
```
    Sets the jobrunner for the experiment.
    
    Parameters:
    
    value - the jobrunner
  - getJobRunner
```
public JobRunner getJobRunner()
```
    Returns the jobrunner for the experiment.
    
    Returns:
    
    the jobrunner
  - jobRunnerTipText
```
public String jobRunnerTipText()
```
    Returns the tip text for this property.
    
    Returns:
    
    tip text for this property suitable for displaying in the GUI or for listing the options.
  - setStatusMessageHandler
```
public void setStatusMessageHandler(StatusMessageHandler value)
```
    Sets the status message handler to use.
    
    Parameters:
    
    value - the handler, null to turn off
  - getStatusMessageHandler
```
public StatusMessageHandler getStatusMessageHandler()
```
    Returns the current status message handler in use.
    
    Returns:
    
    the handler, null if none set
  - preExecute
```
protected String preExecute()
```
    Hook method just before the experiment is run (after initialization).
    
    Returns:
    
    null if successful, otherwise error message
  - toSpreadSheet
```
public SpreadSheet toSpreadSheet()
```
    Returns the collected results.
    
    Specified by:
    
    toSpreadSheet in interface SpreadSheetSupporter
    
    Returns:
    
    the results
  - toInstances
```
public weka.core.Instances toInstances()
```
    Returns the collected results.
    
    Returns:
    
    the results as Instances
  - log
```
protected void log(String msg)
```
    Displays the message.
    
    Parameters:
    
    msg - the message to display
  - log
```
protected void log(String msg,
                   Throwable t)
```
    Displays the error.
    
    Parameters:
    
    msg - the message to display
    
    t - the exception
  - initResults
```
protected SpreadSheet initResults()
```
    Initializes the results.
    
    Returns:
    
    the results
  - initExecute
```
protected String initExecute()
```
    Initializes the experiment.
  - loadDataset
```
protected weka.core.Instances loadDataset(int index)
```
    Loads the dataset.
    
    Parameters:
    
    index - the index of the dataset to load
    
    Returns:
    
    the dataset
  - configureRowFinder
```
protected MultiRowFinder configureRowFinder(int currentRun,
                                            weka.classifiers.Classifier cls,
                                            weka.core.Instances data)
```
    Configures the row finder that determines whether the classifier/dataset combination is still required.
    
    Parameters:
    
    currentRun - the current run
    
    cls - the classifier to check
    
    data - the dataset to check
    
    Returns:
    
    the row finder setup
  - isComplete
```
protected boolean isComplete(int[] rows)
```
    Checks whether the number of rows located in the current results are complete.
    
    Parameters:
    
    rows - the located results
    
    Returns:
    
    true if complete
  - isRequired
```
protected boolean isRequired(int currentRun,
                             weka.classifiers.Classifier cls,
                             weka.core.Instances data)
```
    Checks whether the classifier/dataset combination is required.
    
    Parameters:
    
    currentRun - the current run
    
    cls - the classifier to check
    
    data - the dataset to check
    
    Returns:
    
    true if required
  - removeIncomplete
```
protected void removeIncomplete(int currentRun,
                                weka.classifiers.Classifier cls,
                                weka.core.Instances data)
```
    Removes the incomplete rows of the classifier/dataset combination.
    
    Parameters:
    
    currentRun - the current run
    
    cls - the classifier to check
    
    data - the dataset to check
  - showProgress
```
protected void showProgress()
```
    Updates the progress of the experiment.
  - clearProgress
```
protected void clearProgress()
```
    Clears the progress of the experiment.
  - initProgress
```
public void initProgress()
```
    Initializes progress.
  - incProgress
```
public void incProgress()
```
    Increments and updates the progress.
  - appendResults
```
public void appendResults(SpreadSheet results)
```
    Adds the results to the existing ones.
    
    Parameters:
    
    results - the results to add
  - evaluate
```
protected abstract AbstractExperiment.AbstractExperimentJob<? extends AbstractExperiment> evaluate(int currentRun,
                                                                                                   weka.classifiers.Classifier cls,
                                                                                                   weka.core.Instances data)
```
    Creates a runnabel to evaluate the classifier on the dataset.
    
    Parameters:
    
    currentRun - the current run
    
    cls - the classifier to evaluate
    
    data - the dataset to evaluate on
    
    Returns:
    
    the runnable
  - doExecute
```
protected String doExecute()
```
    Runs the actual experiment.
    
    Returns:
    
    null if successful, otherwise error message
  - postExecute
```
protected void postExecute(boolean success)
```
    Hook method just after the experiment was run.
    
    Parameters:
    
    success - true if successfully run
  - execute
```
public String execute()
```
    Executes the experiment.
    
    Returns:
    
    null if successful, otherwise error message
  - stopExecution
```
public void stopExecution()
```
    Stops the execution.
    
    Specified by:
    
    stopExecution in interface Stoppable
  - isStopped
```
public boolean isStopped()
```
    Returns whether the experiment has been stopped.
    
    Specified by:
    
    isStopped in interface StoppableWithFeedback
    
    Returns:
    
    true if stopped
  - shortenCommandLine
```
public static String shortenCommandLine(weka.classifiers.Classifier cls)
```
    Returns a shortened commandline string for the classifier.
    
    Parameters:
    
    cls - the classifier to get the shortened commandline for
    
    Returns:
    
    the shortened commandline

Class AbstractExperiment

Nested Class Summary

Field Summary

Fields inherited from class adams.core.option.AbstractOptionHandler

Fields inherited from class adams.core.logging.LoggingObject

Constructor Summary

Method Summary

Methods inherited from class adams.core.option.AbstractOptionHandler

Methods inherited from class adams.core.logging.LoggingObject

Methods inherited from class java.lang.Object

Methods inherited from interface adams.core.logging.LoggingLevelHandler

Field Detail

m_ResetResults

m_Classifiers

m_Datasets

m_ClassAttribute

m_ClassLabelIndex

m_UseFilename

m_PrefixDatasetsWithIndex

m_Runs

m_ResultsHandler

m_Notes

m_StatusMessageHandler

m_Running

m_Stopped

m_CommandLineHandler

m_Results

m_Generated

m_JobRunner

m_ActualJobRunner

m_JobCounter

m_JobTotal

Constructor Detail

AbstractExperiment

Method Detail

defineOptions

setResetResults

getResetResults

resetResultsTipText

setResultsHandler

getResultsHandler

resultWriterTipText

setClassifiers

getClassifiers

classifiersTipText

addClassifier

setDatasets

getDatasets

datasetsTipText

addDataset

setClassAttribute

getClassAttribute

classAttributeTipText

setClassLabelIndex

getClassLabelIndex

classLabelIndexTipText

setUseFilename

getUseFilename

useFilenameTipText

setPrefixDatasetsWithIndex

getPrefixDatasetsWithIndex

prefixDatasetsWithIndexTipText

setRuns

getRuns

runsTipText

setNotes

getNotes

notesTipText

setJobRunner

getJobRunner

jobRunnerTipText

setStatusMessageHandler

getStatusMessageHandler

preExecute

toSpreadSheet

toInstances

log

log

initResults

initExecute