Class AbstractExperiment
- java.lang.Object
-
- adams.core.logging.LoggingObject
-
- adams.core.logging.CustomLoggingLevelObject
-
- adams.core.option.AbstractOptionHandler
-
- adams.gui.tools.wekamultiexperimenter.experiment.AbstractExperiment
-
- All Implemented Interfaces:
Destroyable
,GlobalInfoSupporter
,LoggingLevelHandler
,LoggingSupporter
,OptionHandler
,SizeOfHandler
,Stoppable
,StoppableWithFeedback
,SpreadSheetSupporter
,ExperimentWithCustomizableRelationNames
,ResettableExperiment
,Serializable
- Direct Known Subclasses:
CrossValidationExperiment
,TrainTestSplitExperiment
public abstract class AbstractExperiment extends AbstractOptionHandler implements StoppableWithFeedback, ExperimentWithCustomizableRelationNames, ResettableExperiment, SpreadSheetSupporter
Ancestor for simple experiments.- Author:
- FracPete (fracpete at waikato dot ac dot nz)
- See Also:
- Serialized Form
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
AbstractExperiment.AbstractExperimentJob<T extends AbstractExperiment>
For evaluating a single classifier/dataset combination.
-
Field Summary
Fields Modifier and Type Field Description protected JobRunner
m_ActualJobRunner
JobRunner for the classifier/dataset combinations.protected AbstractClassAttributeHeuristic
m_ClassAttribute
how to determine the class attribute.protected weka.classifiers.Classifier[]
m_Classifiers
the classifiers to evaluate.protected Index
m_ClassLabelIndex
the class label index for per-class stats.protected WekaCommandLineHandler
m_CommandLineHandler
for handling commandlines.protected PlaceholderFile[]
m_Datasets
the datasets to evaluate.protected List<SpreadSheet>
m_Generated
the results generated by the evaluations.protected int
m_JobCounter
the counter for finished jobs.protected JobRunner
m_JobRunner
the JobRunner template.protected int
m_JobTotal
the total number of jobs.protected BaseText
m_Notes
the notes for the experiment.protected boolean
m_PrefixDatasetsWithIndex
whether to prefix the relation names with the index.protected boolean
m_ResetResults
whether to reset the results before starting the experiment.protected SpreadSheet
m_Results
the generated results.protected AbstractResultsHandler
m_ResultsHandler
the handler for the results.protected boolean
m_Running
whether the experiment is running.protected int
m_Runs
the number of runs.protected StatusMessageHandler
m_StatusMessageHandler
for notifications.protected boolean
m_Stopped
whether the experiment was stopped.protected boolean
m_UseFilename
whether to use the filename (w/o path) instead of relationname.-
Fields inherited from class adams.core.option.AbstractOptionHandler
m_OptionManager
-
Fields inherited from class adams.core.logging.LoggingObject
m_Logger, m_LoggingIsEnabled, m_LoggingLevel
-
-
Constructor Summary
Constructors Constructor Description AbstractExperiment()
-
Method Summary
All Methods Static Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description void
addClassifier(weka.classifiers.Classifier cls)
Adds a classifier.void
addDataset(PlaceholderFile file)
Adds a dataset.void
appendResults(SpreadSheet results)
Adds the results to the existing ones.String
classAttributeTipText()
Returns the tip text for this property.String
classifiersTipText()
Returns the tip text for this property.String
classLabelIndexTipText()
Returns the tip text for this property.protected void
clearProgress()
Clears the progress of the experiment.protected MultiRowFinder
configureRowFinder(int currentRun, weka.classifiers.Classifier cls, weka.core.Instances data)
Configures the row finder that determines whether the classifier/dataset combination is still required.String
datasetsTipText()
Returns the tip text for this property.void
defineOptions()
Adds options to the internal list of options.protected String
doExecute()
Runs the actual experiment.protected abstract AbstractExperiment.AbstractExperimentJob<? extends AbstractExperiment>
evaluate(int currentRun, weka.classifiers.Classifier cls, weka.core.Instances data)
Creates a runnabel to evaluate the classifier on the dataset.String
execute()
Executes the experiment.AbstractClassAttributeHeuristic
getClassAttribute()
Returns the heuristic for determining the class attribute (if not explicitly set).weka.classifiers.Classifier[]
getClassifiers()
Returns the classifiers.Index
getClassLabelIndex()
Returns the index of the class label to use when generating per-class statistics.PlaceholderFile[]
getDatasets()
The datasets in use.JobRunner
getJobRunner()
Returns the jobrunner for the experiment.BaseText
getNotes()
Returns the notes for the experiment.boolean
getPrefixDatasetsWithIndex()
Returns whether to prefix the datasets with the index.boolean
getResetResults()
Returns whether to clear the results before starting the experiment.AbstractResultsHandler
getResultsHandler()
Returns the results handler to use.int
getRuns()
Returns the number of runs.StatusMessageHandler
getStatusMessageHandler()
Returns the current status message handler in use.boolean
getUseFilename()
Returns whether to use the filename (w/o path) instead of the relationname.void
incProgress()
Increments and updates the progress.protected String
initExecute()
Initializes the experiment.void
initProgress()
Initializes progress.protected SpreadSheet
initResults()
Initializes the results.protected boolean
isComplete(int[] rows)
Checks whether the number of rows located in the current results are complete.protected boolean
isRequired(int currentRun, weka.classifiers.Classifier cls, weka.core.Instances data)
Checks whether the classifier/dataset combination is required.boolean
isStopped()
Returns whether the experiment has been stopped.String
jobRunnerTipText()
Returns the tip text for this property.protected weka.core.Instances
loadDataset(int index)
Loads the dataset.protected int[]
locateRows(int run, weka.classifiers.Classifier cls, weka.core.Instances data)
Returns the rows for the classifier/dataset combination.protected void
log(String msg)
Displays the message.protected void
log(String msg, Throwable t)
Displays the error.String
notesTipText()
Returns the tip text for this property.protected void
postExecute(boolean success)
Hook method just after the experiment was run.protected String
preExecute()
Hook method just before the experiment is run (after initialization).String
prefixDatasetsWithIndexTipText()
Returns the tip text for this property.protected void
removeIncomplete(int currentRun, weka.classifiers.Classifier cls, weka.core.Instances data)
Removes the incomplete rows of the classifier/dataset combination.String
resetResultsTipText()
Returns the tip text for this property.String
resultsHandlerTipText()
Returns the tip text for this property.String
runsTipText()
Returns the tip text for this property.void
setClassAttribute(AbstractClassAttributeHeuristic value)
Sets the heuristic for determining the class attribute (if not explicitly set).void
setClassifiers(weka.classifiers.Classifier[] value)
Sets the classifiers to use.void
setClassLabelIndex(Index value)
Sets the index of the class label to use when generating per-class statistics.void
setDatasets(PlaceholderFile[] value)
Sets the datasets to use.void
setJobRunner(JobRunner value)
Sets the jobrunner for the experiment.void
setNotes(BaseText value)
Sets the notes for the experiment.void
setPrefixDatasetsWithIndex(boolean value)
Sets whether to prefix the datasets with the index.void
setResetResults(boolean value)
Sets whether to clear the results before starting the experiment.void
setResultsHandler(AbstractResultsHandler value)
Sets the results handler to use.void
setRuns(int value)
Sets the number of runs.void
setStatusMessageHandler(StatusMessageHandler value)
Sets the status message handler to use.void
setUseFilename(boolean value)
Sets whether to use the filename (w/o path) instead of the relationname.static String
shortenCommandLine(weka.classifiers.Classifier cls)
Returns a shortened commandline string for the classifier.protected void
showProgress()
Updates the progress of the experiment.protected void
sortResults()
void
stopExecution()
Stops the execution.weka.core.Instances
toInstances()
Returns the collected results.SpreadSheet
toSpreadSheet()
Returns the collected results.String
useFilenameTipText()
Returns the tip text for this property.-
Methods inherited from class adams.core.option.AbstractOptionHandler
cleanUpOptions, destroy, finishInit, getDefaultLoggingLevel, getOptionManager, globalInfo, initialize, loggingLevelTipText, newOptionManager, reset, setLoggingLevel, toCommandLine, toString
-
Methods inherited from class adams.core.logging.LoggingObject
configureLogger, getLogger, getLoggingLevel, initializeLogging, isLoggingEnabled, sizeOf
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
-
Methods inherited from interface adams.core.logging.LoggingLevelHandler
getLoggingLevel
-
-
-
-
Field Detail
-
m_ResetResults
protected boolean m_ResetResults
whether to reset the results before starting the experiment.
-
m_Classifiers
protected weka.classifiers.Classifier[] m_Classifiers
the classifiers to evaluate.
-
m_Datasets
protected PlaceholderFile[] m_Datasets
the datasets to evaluate.
-
m_ClassAttribute
protected AbstractClassAttributeHeuristic m_ClassAttribute
how to determine the class attribute.
-
m_ClassLabelIndex
protected Index m_ClassLabelIndex
the class label index for per-class stats.
-
m_UseFilename
protected boolean m_UseFilename
whether to use the filename (w/o path) instead of relationname.
-
m_PrefixDatasetsWithIndex
protected boolean m_PrefixDatasetsWithIndex
whether to prefix the relation names with the index.
-
m_Runs
protected int m_Runs
the number of runs.
-
m_ResultsHandler
protected AbstractResultsHandler m_ResultsHandler
the handler for the results.
-
m_Notes
protected BaseText m_Notes
the notes for the experiment.
-
m_StatusMessageHandler
protected transient StatusMessageHandler m_StatusMessageHandler
for notifications.
-
m_Running
protected transient boolean m_Running
whether the experiment is running.
-
m_Stopped
protected transient boolean m_Stopped
whether the experiment was stopped.
-
m_CommandLineHandler
protected transient WekaCommandLineHandler m_CommandLineHandler
for handling commandlines.
-
m_Results
protected SpreadSheet m_Results
the generated results.
-
m_Generated
protected List<SpreadSheet> m_Generated
the results generated by the evaluations.
-
m_JobRunner
protected JobRunner m_JobRunner
the JobRunner template.
-
m_ActualJobRunner
protected transient JobRunner m_ActualJobRunner
JobRunner for the classifier/dataset combinations.
-
m_JobCounter
protected int m_JobCounter
the counter for finished jobs.
-
m_JobTotal
protected int m_JobTotal
the total number of jobs.
-
-
Method Detail
-
defineOptions
public void defineOptions()
Adds options to the internal list of options.- Specified by:
defineOptions
in interfaceOptionHandler
- Overrides:
defineOptions
in classAbstractOptionHandler
-
setResetResults
public void setResetResults(boolean value)
Sets whether to clear the results before starting the experiment.- Specified by:
setResetResults
in interfaceResettableExperiment
- Parameters:
value
- true if to clear results
-
getResetResults
public boolean getResetResults()
Returns whether to clear the results before starting the experiment.- Specified by:
getResetResults
in interfaceResettableExperiment
- Returns:
- true if to clear results
-
resetResultsTipText
public String resetResultsTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setResultsHandler
public void setResultsHandler(AbstractResultsHandler value)
Sets the results handler to use.- Parameters:
value
- the handler
-
getResultsHandler
public AbstractResultsHandler getResultsHandler()
Returns the results handler to use.- Returns:
- the handler
-
resultsHandlerTipText
public String resultsHandlerTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setClassifiers
public void setClassifiers(weka.classifiers.Classifier[] value)
Sets the classifiers to use.- Parameters:
value
- the classifiers
-
getClassifiers
public weka.classifiers.Classifier[] getClassifiers()
Returns the classifiers.- Returns:
- the classifiers
-
classifiersTipText
public String classifiersTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
addClassifier
public void addClassifier(weka.classifiers.Classifier cls)
Adds a classifier.- Parameters:
cls
- the classifier to add
-
setDatasets
public void setDatasets(PlaceholderFile[] value)
Sets the datasets to use.- Parameters:
value
- the datasets
-
getDatasets
public PlaceholderFile[] getDatasets()
The datasets in use.- Returns:
- the datasets
-
datasetsTipText
public String datasetsTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
addDataset
public void addDataset(PlaceholderFile file)
Adds a dataset.- Parameters:
file
- the dataset to add
-
setClassAttribute
public void setClassAttribute(AbstractClassAttributeHeuristic value)
Sets the heuristic for determining the class attribute (if not explicitly set).- Parameters:
value
- the heuristic
-
getClassAttribute
public AbstractClassAttributeHeuristic getClassAttribute()
Returns the heuristic for determining the class attribute (if not explicitly set).- Returns:
- the heuristic
-
classAttributeTipText
public String classAttributeTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setClassLabelIndex
public void setClassLabelIndex(Index value)
Sets the index of the class label to use when generating per-class statistics.- Parameters:
value
- the index
-
getClassLabelIndex
public Index getClassLabelIndex()
Returns the index of the class label to use when generating per-class statistics.- Returns:
- the index
-
classLabelIndexTipText
public String classLabelIndexTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setUseFilename
public void setUseFilename(boolean value)
Sets whether to use the filename (w/o path) instead of the relationname.- Specified by:
setUseFilename
in interfaceExperimentWithCustomizableRelationNames
- Parameters:
value
- true if to use filename
-
getUseFilename
public boolean getUseFilename()
Returns whether to use the filename (w/o path) instead of the relationname.- Specified by:
getUseFilename
in interfaceExperimentWithCustomizableRelationNames
- Returns:
- true if to use the filename
-
useFilenameTipText
public String useFilenameTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setPrefixDatasetsWithIndex
public void setPrefixDatasetsWithIndex(boolean value)
Sets whether to prefix the datasets with the index.- Specified by:
setPrefixDatasetsWithIndex
in interfaceExperimentWithCustomizableRelationNames
- Parameters:
value
- true if to prefix
-
getPrefixDatasetsWithIndex
public boolean getPrefixDatasetsWithIndex()
Returns whether to prefix the datasets with the index.- Specified by:
getPrefixDatasetsWithIndex
in interfaceExperimentWithCustomizableRelationNames
- Returns:
- true if to prefix
-
prefixDatasetsWithIndexTipText
public String prefixDatasetsWithIndexTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setRuns
public void setRuns(int value)
Sets the number of runs.- Parameters:
value
- the runs
-
getRuns
public int getRuns()
Returns the number of runs.- Returns:
- the runs
-
runsTipText
public String runsTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setNotes
public void setNotes(BaseText value)
Sets the notes for the experiment.- Parameters:
value
- the notes
-
getNotes
public BaseText getNotes()
Returns the notes for the experiment.- Returns:
- the notes
-
notesTipText
public String notesTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setJobRunner
public void setJobRunner(JobRunner value)
Sets the jobrunner for the experiment.- Parameters:
value
- the jobrunner
-
getJobRunner
public JobRunner getJobRunner()
Returns the jobrunner for the experiment.- Returns:
- the jobrunner
-
jobRunnerTipText
public String jobRunnerTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setStatusMessageHandler
public void setStatusMessageHandler(StatusMessageHandler value)
Sets the status message handler to use.- Parameters:
value
- the handler, null to turn off
-
getStatusMessageHandler
public StatusMessageHandler getStatusMessageHandler()
Returns the current status message handler in use.- Returns:
- the handler, null if none set
-
preExecute
protected String preExecute()
Hook method just before the experiment is run (after initialization).- Returns:
- null if successful, otherwise error message
-
toSpreadSheet
public SpreadSheet toSpreadSheet()
Returns the collected results.- Specified by:
toSpreadSheet
in interfaceSpreadSheetSupporter
- Returns:
- the results
-
toInstances
public weka.core.Instances toInstances()
Returns the collected results.- Returns:
- the results as Instances
-
log
protected void log(String msg)
Displays the message.- Parameters:
msg
- the message to display
-
log
protected void log(String msg, Throwable t)
Displays the error.- Parameters:
msg
- the message to displayt
- the exception
-
initResults
protected SpreadSheet initResults()
Initializes the results.- Returns:
- the results
-
initExecute
protected String initExecute()
Initializes the experiment.
-
loadDataset
protected weka.core.Instances loadDataset(int index)
Loads the dataset.- Parameters:
index
- the index of the dataset to load- Returns:
- the dataset
-
configureRowFinder
protected MultiRowFinder configureRowFinder(int currentRun, weka.classifiers.Classifier cls, weka.core.Instances data)
Configures the row finder that determines whether the classifier/dataset combination is still required.- Parameters:
currentRun
- the current runcls
- the classifier to checkdata
- the dataset to check- Returns:
- the row finder setup
-
isComplete
protected boolean isComplete(int[] rows)
Checks whether the number of rows located in the current results are complete.- Parameters:
rows
- the located results- Returns:
- true if complete
-
locateRows
protected int[] locateRows(int run, weka.classifiers.Classifier cls, weka.core.Instances data)
Returns the rows for the classifier/dataset combination.- Parameters:
run
- the runcls
- the classifier to checkdata
- the dataset to check- Returns:
- the rows
-
isRequired
protected boolean isRequired(int currentRun, weka.classifiers.Classifier cls, weka.core.Instances data)
Checks whether the classifier/dataset combination is required.- Parameters:
currentRun
- the current runcls
- the classifier to checkdata
- the dataset to check- Returns:
- true if required
-
removeIncomplete
protected void removeIncomplete(int currentRun, weka.classifiers.Classifier cls, weka.core.Instances data)
Removes the incomplete rows of the classifier/dataset combination.- Parameters:
currentRun
- the current runcls
- the classifier to checkdata
- the dataset to check
-
showProgress
protected void showProgress()
Updates the progress of the experiment.
-
clearProgress
protected void clearProgress()
Clears the progress of the experiment.
-
initProgress
public void initProgress()
Initializes progress.
-
incProgress
public void incProgress()
Increments and updates the progress.
-
appendResults
public void appendResults(SpreadSheet results)
Adds the results to the existing ones.- Parameters:
results
- the results to add
-
evaluate
protected abstract AbstractExperiment.AbstractExperimentJob<? extends AbstractExperiment> evaluate(int currentRun, weka.classifiers.Classifier cls, weka.core.Instances data)
Creates a runnabel to evaluate the classifier on the dataset.- Parameters:
currentRun
- the current runcls
- the classifier to evaluatedata
- the dataset to evaluate on- Returns:
- the runnable
-
doExecute
protected String doExecute()
Runs the actual experiment.- Returns:
- null if successful, otherwise error message
-
sortResults
protected void sortResults()
-
postExecute
protected void postExecute(boolean success)
Hook method just after the experiment was run.- Parameters:
success
- true if successfully run
-
execute
public String execute()
Executes the experiment.- Returns:
- null if successful, otherwise error message
-
stopExecution
public void stopExecution()
Stops the execution.- Specified by:
stopExecution
in interfaceStoppable
-
isStopped
public boolean isStopped()
Returns whether the experiment has been stopped.- Specified by:
isStopped
in interfaceStoppableWithFeedback
- Returns:
- true if stopped
-
shortenCommandLine
public static String shortenCommandLine(weka.classifiers.Classifier cls)
Returns a shortened commandline string for the classifier.- Parameters:
cls
- the classifier to get the shortened commandline for- Returns:
- the shortened commandline
-
-