Class AbstractExperiment
- java.lang.Object
-
- adams.core.logging.LoggingObject
-
- adams.core.logging.CustomLoggingLevelObject
-
- adams.core.option.AbstractOptionHandler
-
- adams.gui.tools.wekamultiexperimenter.experiment.AbstractExperiment
-
- All Implemented Interfaces:
adams.core.Destroyable,adams.core.GlobalInfoSupporter,adams.core.logging.LoggingLevelHandler,adams.core.logging.LoggingSupporter,adams.core.option.OptionHandler,adams.core.SizeOfHandler,adams.core.Stoppable,adams.core.StoppableWithFeedback,adams.data.spreadsheet.SpreadSheetSupporter,ExperimentWithCustomizableRelationNames,ResettableExperiment,Serializable
- Direct Known Subclasses:
CrossValidationExperiment,TrainTestSplitExperiment
public abstract class AbstractExperiment extends adams.core.option.AbstractOptionHandler implements adams.core.StoppableWithFeedback, ExperimentWithCustomizableRelationNames, ResettableExperiment, adams.data.spreadsheet.SpreadSheetSupporter
Ancestor for simple experiments.- Author:
- FracPete (fracpete at waikato dot ac dot nz)
- See Also:
- Serialized Form
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static classAbstractExperiment.AbstractExperimentJob<T extends AbstractExperiment>For evaluating a single classifier/dataset combination.
-
Field Summary
Fields Modifier and Type Field Description protected adams.multiprocess.JobRunnerm_ActualJobRunnerJobRunner for the classifier/dataset combinations.protected AbstractClassAttributeHeuristicm_ClassAttributehow to determine the class attribute.protected weka.classifiers.Classifier[]m_Classifiersthe classifiers to evaluate.protected adams.core.Indexm_ClassLabelIndexthe class label index for per-class stats.protected WekaCommandLineHandlerm_CommandLineHandlerfor handling commandlines.protected adams.core.io.PlaceholderFile[]m_Datasetsthe datasets to evaluate.protected List<adams.data.spreadsheet.SpreadSheet>m_Generatedthe results generated by the evaluations.protected intm_JobCounterthe counter for finished jobs.protected adams.multiprocess.JobRunnerm_JobRunnerthe JobRunner template.protected intm_JobTotalthe total number of jobs.protected adams.core.base.BaseTextm_Notesthe notes for the experiment.protected booleanm_PrefixDatasetsWithIndexwhether to prefix the relation names with the index.protected booleanm_ResetResultswhether to reset the results before starting the experiment.protected adams.data.spreadsheet.SpreadSheetm_Resultsthe generated results.protected AbstractResultsHandlerm_ResultsHandlerthe handler for the results.protected booleanm_Runningwhether the experiment is running.protected intm_Runsthe number of runs.protected adams.core.StatusMessageHandlerm_StatusMessageHandlerfor notifications.protected booleanm_Stoppedwhether the experiment was stopped.protected booleanm_UseFilenamewhether to use the filename (w/o path) instead of relationname.
-
Constructor Summary
Constructors Constructor Description AbstractExperiment()
-
Method Summary
All Methods Static Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description voidaddClassifier(weka.classifiers.Classifier cls)Adds a classifier.voidaddDataset(adams.core.io.PlaceholderFile file)Adds a dataset.voidappendResults(adams.data.spreadsheet.SpreadSheet results)Adds the results to the existing ones.StringclassAttributeTipText()Returns the tip text for this property.StringclassifiersTipText()Returns the tip text for this property.StringclassLabelIndexTipText()Returns the tip text for this property.protected voidclearProgress()Clears the progress of the experiment.protected adams.data.spreadsheet.rowfinder.MultiRowFinderconfigureRowFinder(int currentRun, weka.classifiers.Classifier cls, weka.core.Instances data)Configures the row finder that determines whether the classifier/dataset combination is still required.StringdatasetsTipText()Returns the tip text for this property.voiddefineOptions()Adds options to the internal list of options.protected StringdoExecute()Runs the actual experiment.protected abstract AbstractExperiment.AbstractExperimentJob<? extends AbstractExperiment>evaluate(int currentRun, weka.classifiers.Classifier cls, weka.core.Instances data)Creates a runnabel to evaluate the classifier on the dataset.Stringexecute()Executes the experiment.AbstractClassAttributeHeuristicgetClassAttribute()Returns the heuristic for determining the class attribute (if not explicitly set).weka.classifiers.Classifier[]getClassifiers()Returns the classifiers.adams.core.IndexgetClassLabelIndex()Returns the index of the class label to use when generating per-class statistics.adams.core.io.PlaceholderFile[]getDatasets()The datasets in use.adams.multiprocess.JobRunnergetJobRunner()Returns the jobrunner for the experiment.adams.core.base.BaseTextgetNotes()Returns the notes for the experiment.booleangetPrefixDatasetsWithIndex()Returns whether to prefix the datasets with the index.booleangetResetResults()Returns whether to clear the results before starting the experiment.AbstractResultsHandlergetResultsHandler()Returns the results handler to use.intgetRuns()Returns the number of runs.adams.core.StatusMessageHandlergetStatusMessageHandler()Returns the current status message handler in use.booleangetUseFilename()Returns whether to use the filename (w/o path) instead of the relationname.voidincProgress()Increments and updates the progress.protected StringinitExecute()Initializes the experiment.voidinitProgress()Initializes progress.protected adams.data.spreadsheet.SpreadSheetinitResults()Initializes the results.protected booleanisComplete(int[] rows)Checks whether the number of rows located in the current results are complete.protected booleanisRequired(int currentRun, weka.classifiers.Classifier cls, weka.core.Instances data)Checks whether the classifier/dataset combination is required.booleanisStopped()Returns whether the experiment has been stopped.StringjobRunnerTipText()Returns the tip text for this property.protected weka.core.InstancesloadDataset(int index)Loads the dataset.protected int[]locateRows(int run, weka.classifiers.Classifier cls, weka.core.Instances data)Returns the rows for the classifier/dataset combination.protected voidlog(String msg)Displays the message.protected voidlog(String msg, Throwable t)Displays the error.StringnotesTipText()Returns the tip text for this property.protected voidpostExecute(boolean success)Hook method just after the experiment was run.protected StringpreExecute()Hook method just before the experiment is run (after initialization).StringprefixDatasetsWithIndexTipText()Returns the tip text for this property.protected voidremoveIncomplete(int currentRun, weka.classifiers.Classifier cls, weka.core.Instances data)Removes the incomplete rows of the classifier/dataset combination.StringresetResultsTipText()Returns the tip text for this property.StringresultsHandlerTipText()Returns the tip text for this property.StringrunsTipText()Returns the tip text for this property.voidsetClassAttribute(AbstractClassAttributeHeuristic value)Sets the heuristic for determining the class attribute (if not explicitly set).voidsetClassifiers(weka.classifiers.Classifier[] value)Sets the classifiers to use.voidsetClassLabelIndex(adams.core.Index value)Sets the index of the class label to use when generating per-class statistics.voidsetDatasets(adams.core.io.PlaceholderFile[] value)Sets the datasets to use.voidsetJobRunner(adams.multiprocess.JobRunner value)Sets the jobrunner for the experiment.voidsetNotes(adams.core.base.BaseText value)Sets the notes for the experiment.voidsetPrefixDatasetsWithIndex(boolean value)Sets whether to prefix the datasets with the index.voidsetResetResults(boolean value)Sets whether to clear the results before starting the experiment.voidsetResultsHandler(AbstractResultsHandler value)Sets the results handler to use.voidsetRuns(int value)Sets the number of runs.voidsetStatusMessageHandler(adams.core.StatusMessageHandler value)Sets the status message handler to use.voidsetUseFilename(boolean value)Sets whether to use the filename (w/o path) instead of the relationname.static StringshortenCommandLine(weka.classifiers.Classifier cls)Returns a shortened commandline string for the classifier.protected voidshowProgress()Updates the progress of the experiment.protected voidsortResults()voidstopExecution()Stops the execution.weka.core.InstancestoInstances()Returns the collected results.adams.data.spreadsheet.SpreadSheettoSpreadSheet()Returns the collected results.StringuseFilenameTipText()Returns the tip text for this property.-
Methods inherited from class adams.core.option.AbstractOptionHandler
cleanUpOptions, destroy, finishInit, getDefaultLoggingLevel, getOptionManager, globalInfo, initialize, loggingLevelTipText, newOptionManager, reset, setLoggingLevel, toCommandLine, toString
-
Methods inherited from class adams.core.logging.LoggingObject
configureLogger, getLogger, getLoggingLevel, initializeLogging, isLoggingEnabled, sizeOf
-
-
-
-
Field Detail
-
m_ResetResults
protected boolean m_ResetResults
whether to reset the results before starting the experiment.
-
m_Classifiers
protected weka.classifiers.Classifier[] m_Classifiers
the classifiers to evaluate.
-
m_Datasets
protected adams.core.io.PlaceholderFile[] m_Datasets
the datasets to evaluate.
-
m_ClassAttribute
protected AbstractClassAttributeHeuristic m_ClassAttribute
how to determine the class attribute.
-
m_ClassLabelIndex
protected adams.core.Index m_ClassLabelIndex
the class label index for per-class stats.
-
m_UseFilename
protected boolean m_UseFilename
whether to use the filename (w/o path) instead of relationname.
-
m_PrefixDatasetsWithIndex
protected boolean m_PrefixDatasetsWithIndex
whether to prefix the relation names with the index.
-
m_Runs
protected int m_Runs
the number of runs.
-
m_ResultsHandler
protected AbstractResultsHandler m_ResultsHandler
the handler for the results.
-
m_Notes
protected adams.core.base.BaseText m_Notes
the notes for the experiment.
-
m_StatusMessageHandler
protected transient adams.core.StatusMessageHandler m_StatusMessageHandler
for notifications.
-
m_Running
protected transient boolean m_Running
whether the experiment is running.
-
m_Stopped
protected transient boolean m_Stopped
whether the experiment was stopped.
-
m_CommandLineHandler
protected transient WekaCommandLineHandler m_CommandLineHandler
for handling commandlines.
-
m_Results
protected adams.data.spreadsheet.SpreadSheet m_Results
the generated results.
-
m_Generated
protected List<adams.data.spreadsheet.SpreadSheet> m_Generated
the results generated by the evaluations.
-
m_JobRunner
protected adams.multiprocess.JobRunner m_JobRunner
the JobRunner template.
-
m_ActualJobRunner
protected transient adams.multiprocess.JobRunner m_ActualJobRunner
JobRunner for the classifier/dataset combinations.
-
m_JobCounter
protected int m_JobCounter
the counter for finished jobs.
-
m_JobTotal
protected int m_JobTotal
the total number of jobs.
-
-
Method Detail
-
defineOptions
public void defineOptions()
Adds options to the internal list of options.- Specified by:
defineOptionsin interfaceadams.core.option.OptionHandler- Overrides:
defineOptionsin classadams.core.option.AbstractOptionHandler
-
setResetResults
public void setResetResults(boolean value)
Sets whether to clear the results before starting the experiment.- Specified by:
setResetResultsin interfaceResettableExperiment- Parameters:
value- true if to clear results
-
getResetResults
public boolean getResetResults()
Returns whether to clear the results before starting the experiment.- Specified by:
getResetResultsin interfaceResettableExperiment- Returns:
- true if to clear results
-
resetResultsTipText
public String resetResultsTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setResultsHandler
public void setResultsHandler(AbstractResultsHandler value)
Sets the results handler to use.- Parameters:
value- the handler
-
getResultsHandler
public AbstractResultsHandler getResultsHandler()
Returns the results handler to use.- Returns:
- the handler
-
resultsHandlerTipText
public String resultsHandlerTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setClassifiers
public void setClassifiers(weka.classifiers.Classifier[] value)
Sets the classifiers to use.- Parameters:
value- the classifiers
-
getClassifiers
public weka.classifiers.Classifier[] getClassifiers()
Returns the classifiers.- Returns:
- the classifiers
-
classifiersTipText
public String classifiersTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
addClassifier
public void addClassifier(weka.classifiers.Classifier cls)
Adds a classifier.- Parameters:
cls- the classifier to add
-
setDatasets
public void setDatasets(adams.core.io.PlaceholderFile[] value)
Sets the datasets to use.- Parameters:
value- the datasets
-
getDatasets
public adams.core.io.PlaceholderFile[] getDatasets()
The datasets in use.- Returns:
- the datasets
-
datasetsTipText
public String datasetsTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
addDataset
public void addDataset(adams.core.io.PlaceholderFile file)
Adds a dataset.- Parameters:
file- the dataset to add
-
setClassAttribute
public void setClassAttribute(AbstractClassAttributeHeuristic value)
Sets the heuristic for determining the class attribute (if not explicitly set).- Parameters:
value- the heuristic
-
getClassAttribute
public AbstractClassAttributeHeuristic getClassAttribute()
Returns the heuristic for determining the class attribute (if not explicitly set).- Returns:
- the heuristic
-
classAttributeTipText
public String classAttributeTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setClassLabelIndex
public void setClassLabelIndex(adams.core.Index value)
Sets the index of the class label to use when generating per-class statistics.- Parameters:
value- the index
-
getClassLabelIndex
public adams.core.Index getClassLabelIndex()
Returns the index of the class label to use when generating per-class statistics.- Returns:
- the index
-
classLabelIndexTipText
public String classLabelIndexTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setUseFilename
public void setUseFilename(boolean value)
Sets whether to use the filename (w/o path) instead of the relationname.- Specified by:
setUseFilenamein interfaceExperimentWithCustomizableRelationNames- Parameters:
value- true if to use filename
-
getUseFilename
public boolean getUseFilename()
Returns whether to use the filename (w/o path) instead of the relationname.- Specified by:
getUseFilenamein interfaceExperimentWithCustomizableRelationNames- Returns:
- true if to use the filename
-
useFilenameTipText
public String useFilenameTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setPrefixDatasetsWithIndex
public void setPrefixDatasetsWithIndex(boolean value)
Sets whether to prefix the datasets with the index.- Specified by:
setPrefixDatasetsWithIndexin interfaceExperimentWithCustomizableRelationNames- Parameters:
value- true if to prefix
-
getPrefixDatasetsWithIndex
public boolean getPrefixDatasetsWithIndex()
Returns whether to prefix the datasets with the index.- Specified by:
getPrefixDatasetsWithIndexin interfaceExperimentWithCustomizableRelationNames- Returns:
- true if to prefix
-
prefixDatasetsWithIndexTipText
public String prefixDatasetsWithIndexTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setRuns
public void setRuns(int value)
Sets the number of runs.- Parameters:
value- the runs
-
getRuns
public int getRuns()
Returns the number of runs.- Returns:
- the runs
-
runsTipText
public String runsTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setNotes
public void setNotes(adams.core.base.BaseText value)
Sets the notes for the experiment.- Parameters:
value- the notes
-
getNotes
public adams.core.base.BaseText getNotes()
Returns the notes for the experiment.- Returns:
- the notes
-
notesTipText
public String notesTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setJobRunner
public void setJobRunner(adams.multiprocess.JobRunner value)
Sets the jobrunner for the experiment.- Parameters:
value- the jobrunner
-
getJobRunner
public adams.multiprocess.JobRunner getJobRunner()
Returns the jobrunner for the experiment.- Returns:
- the jobrunner
-
jobRunnerTipText
public String jobRunnerTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setStatusMessageHandler
public void setStatusMessageHandler(adams.core.StatusMessageHandler value)
Sets the status message handler to use.- Parameters:
value- the handler, null to turn off
-
getStatusMessageHandler
public adams.core.StatusMessageHandler getStatusMessageHandler()
Returns the current status message handler in use.- Returns:
- the handler, null if none set
-
preExecute
protected String preExecute()
Hook method just before the experiment is run (after initialization).- Returns:
- null if successful, otherwise error message
-
toSpreadSheet
public adams.data.spreadsheet.SpreadSheet toSpreadSheet()
Returns the collected results.- Specified by:
toSpreadSheetin interfaceadams.data.spreadsheet.SpreadSheetSupporter- Returns:
- the results
-
toInstances
public weka.core.Instances toInstances()
Returns the collected results.- Returns:
- the results as Instances
-
log
protected void log(String msg)
Displays the message.- Parameters:
msg- the message to display
-
log
protected void log(String msg, Throwable t)
Displays the error.- Parameters:
msg- the message to displayt- the exception
-
initResults
protected adams.data.spreadsheet.SpreadSheet initResults()
Initializes the results.- Returns:
- the results
-
initExecute
protected String initExecute()
Initializes the experiment.
-
loadDataset
protected weka.core.Instances loadDataset(int index)
Loads the dataset.- Parameters:
index- the index of the dataset to load- Returns:
- the dataset
-
configureRowFinder
protected adams.data.spreadsheet.rowfinder.MultiRowFinder configureRowFinder(int currentRun, weka.classifiers.Classifier cls, weka.core.Instances data)Configures the row finder that determines whether the classifier/dataset combination is still required.- Parameters:
currentRun- the current runcls- the classifier to checkdata- the dataset to check- Returns:
- the row finder setup
-
isComplete
protected boolean isComplete(int[] rows)
Checks whether the number of rows located in the current results are complete.- Parameters:
rows- the located results- Returns:
- true if complete
-
locateRows
protected int[] locateRows(int run, weka.classifiers.Classifier cls, weka.core.Instances data)Returns the rows for the classifier/dataset combination.- Parameters:
run- the runcls- the classifier to checkdata- the dataset to check- Returns:
- the rows
-
isRequired
protected boolean isRequired(int currentRun, weka.classifiers.Classifier cls, weka.core.Instances data)Checks whether the classifier/dataset combination is required.- Parameters:
currentRun- the current runcls- the classifier to checkdata- the dataset to check- Returns:
- true if required
-
removeIncomplete
protected void removeIncomplete(int currentRun, weka.classifiers.Classifier cls, weka.core.Instances data)Removes the incomplete rows of the classifier/dataset combination.- Parameters:
currentRun- the current runcls- the classifier to checkdata- the dataset to check
-
showProgress
protected void showProgress()
Updates the progress of the experiment.
-
clearProgress
protected void clearProgress()
Clears the progress of the experiment.
-
initProgress
public void initProgress()
Initializes progress.
-
incProgress
public void incProgress()
Increments and updates the progress.
-
appendResults
public void appendResults(adams.data.spreadsheet.SpreadSheet results)
Adds the results to the existing ones.- Parameters:
results- the results to add
-
evaluate
protected abstract AbstractExperiment.AbstractExperimentJob<? extends AbstractExperiment> evaluate(int currentRun, weka.classifiers.Classifier cls, weka.core.Instances data)
Creates a runnabel to evaluate the classifier on the dataset.- Parameters:
currentRun- the current runcls- the classifier to evaluatedata- the dataset to evaluate on- Returns:
- the runnable
-
doExecute
protected String doExecute()
Runs the actual experiment.- Returns:
- null if successful, otherwise error message
-
sortResults
protected void sortResults()
-
postExecute
protected void postExecute(boolean success)
Hook method just after the experiment was run.- Parameters:
success- true if successfully run
-
execute
public String execute()
Executes the experiment.- Returns:
- null if successful, otherwise error message
-
stopExecution
public void stopExecution()
Stops the execution.- Specified by:
stopExecutionin interfaceadams.core.Stoppable
-
isStopped
public boolean isStopped()
Returns whether the experiment has been stopped.- Specified by:
isStoppedin interfaceadams.core.StoppableWithFeedback- Returns:
- true if stopped
-
shortenCommandLine
public static String shortenCommandLine(weka.classifiers.Classifier cls)
Returns a shortened commandline string for the classifier.- Parameters:
cls- the classifier to get the shortened commandline for- Returns:
- the shortened commandline
-
-