Class AbstractExperiment

    • Field Summary

      Fields 
      Modifier and Type Field Description
      protected adams.multiprocess.JobRunner m_ActualJobRunner
      JobRunner for the classifier/dataset combinations.
      protected AbstractClassAttributeHeuristic m_ClassAttribute
      how to determine the class attribute.
      protected weka.classifiers.Classifier[] m_Classifiers
      the classifiers to evaluate.
      protected adams.core.Index m_ClassLabelIndex
      the class label index for per-class stats.
      protected WekaCommandLineHandler m_CommandLineHandler
      for handling commandlines.
      protected adams.core.io.PlaceholderFile[] m_Datasets
      the datasets to evaluate.
      protected List<adams.data.spreadsheet.SpreadSheet> m_Generated
      the results generated by the evaluations.
      protected int m_JobCounter
      the counter for finished jobs.
      protected adams.multiprocess.JobRunner m_JobRunner
      the JobRunner template.
      protected int m_JobTotal
      the total number of jobs.
      protected adams.core.base.BaseText m_Notes
      the notes for the experiment.
      protected boolean m_PrefixDatasetsWithIndex
      whether to prefix the relation names with the index.
      protected boolean m_ResetResults
      whether to reset the results before starting the experiment.
      protected adams.data.spreadsheet.SpreadSheet m_Results
      the generated results.
      protected AbstractResultsHandler m_ResultsHandler
      the handler for the results.
      protected boolean m_Running
      whether the experiment is running.
      protected int m_Runs
      the number of runs.
      protected adams.core.StatusMessageHandler m_StatusMessageHandler
      for notifications.
      protected boolean m_Stopped
      whether the experiment was stopped.
      protected boolean m_UseFilename
      whether to use the filename (w/o path) instead of relationname.
      • Fields inherited from class adams.core.option.AbstractOptionHandler

        m_OptionManager
      • Fields inherited from class adams.core.logging.LoggingObject

        m_Logger, m_LoggingIsEnabled, m_LoggingLevel
    • Method Summary

      All Methods Static Methods Instance Methods Abstract Methods Concrete Methods 
      Modifier and Type Method Description
      void addClassifier​(weka.classifiers.Classifier cls)
      Adds a classifier.
      void addDataset​(adams.core.io.PlaceholderFile file)
      Adds a dataset.
      void appendResults​(adams.data.spreadsheet.SpreadSheet results)
      Adds the results to the existing ones.
      String classAttributeTipText()
      Returns the tip text for this property.
      String classifiersTipText()
      Returns the tip text for this property.
      String classLabelIndexTipText()
      Returns the tip text for this property.
      protected void clearProgress()
      Clears the progress of the experiment.
      protected adams.data.spreadsheet.rowfinder.MultiRowFinder configureRowFinder​(int currentRun, weka.classifiers.Classifier cls, weka.core.Instances data)
      Configures the row finder that determines whether the classifier/dataset combination is still required.
      String datasetsTipText()
      Returns the tip text for this property.
      void defineOptions()
      Adds options to the internal list of options.
      protected String doExecute()
      Runs the actual experiment.
      protected abstract AbstractExperiment.AbstractExperimentJob<? extends AbstractExperiment> evaluate​(int currentRun, weka.classifiers.Classifier cls, weka.core.Instances data)
      Creates a runnabel to evaluate the classifier on the dataset.
      String execute()
      Executes the experiment.
      AbstractClassAttributeHeuristic getClassAttribute()
      Returns the heuristic for determining the class attribute (if not explicitly set).
      weka.classifiers.Classifier[] getClassifiers()
      Returns the classifiers.
      adams.core.Index getClassLabelIndex()
      Returns the index of the class label to use when generating per-class statistics.
      adams.core.io.PlaceholderFile[] getDatasets()
      The datasets in use.
      adams.multiprocess.JobRunner getJobRunner()
      Returns the jobrunner for the experiment.
      adams.core.base.BaseText getNotes()
      Returns the notes for the experiment.
      boolean getPrefixDatasetsWithIndex()
      Returns whether to prefix the datasets with the index.
      boolean getResetResults()
      Returns whether to clear the results before starting the experiment.
      AbstractResultsHandler getResultsHandler()
      Returns the results handler to use.
      int getRuns()
      Returns the number of runs.
      adams.core.StatusMessageHandler getStatusMessageHandler()
      Returns the current status message handler in use.
      boolean getUseFilename()
      Returns whether to use the filename (w/o path) instead of the relationname.
      void incProgress()
      Increments and updates the progress.
      protected String initExecute()
      Initializes the experiment.
      void initProgress()
      Initializes progress.
      protected adams.data.spreadsheet.SpreadSheet initResults()
      Initializes the results.
      protected boolean isComplete​(int[] rows)
      Checks whether the number of rows located in the current results are complete.
      protected boolean isRequired​(int currentRun, weka.classifiers.Classifier cls, weka.core.Instances data)
      Checks whether the classifier/dataset combination is required.
      boolean isStopped()
      Returns whether the experiment has been stopped.
      String jobRunnerTipText()
      Returns the tip text for this property.
      protected weka.core.Instances loadDataset​(int index)
      Loads the dataset.
      protected int[] locateRows​(int run, weka.classifiers.Classifier cls, weka.core.Instances data)
      Returns the rows for the classifier/dataset combination.
      protected void log​(String msg)
      Displays the message.
      protected void log​(String msg, Throwable t)
      Displays the error.
      String notesTipText()
      Returns the tip text for this property.
      protected void postExecute​(boolean success)
      Hook method just after the experiment was run.
      protected String preExecute()
      Hook method just before the experiment is run (after initialization).
      String prefixDatasetsWithIndexTipText()
      Returns the tip text for this property.
      protected void removeIncomplete​(int currentRun, weka.classifiers.Classifier cls, weka.core.Instances data)
      Removes the incomplete rows of the classifier/dataset combination.
      String resetResultsTipText()
      Returns the tip text for this property.
      String resultsHandlerTipText()
      Returns the tip text for this property.
      String runsTipText()
      Returns the tip text for this property.
      void setClassAttribute​(AbstractClassAttributeHeuristic value)
      Sets the heuristic for determining the class attribute (if not explicitly set).
      void setClassifiers​(weka.classifiers.Classifier[] value)
      Sets the classifiers to use.
      void setClassLabelIndex​(adams.core.Index value)
      Sets the index of the class label to use when generating per-class statistics.
      void setDatasets​(adams.core.io.PlaceholderFile[] value)
      Sets the datasets to use.
      void setJobRunner​(adams.multiprocess.JobRunner value)
      Sets the jobrunner for the experiment.
      void setNotes​(adams.core.base.BaseText value)
      Sets the notes for the experiment.
      void setPrefixDatasetsWithIndex​(boolean value)
      Sets whether to prefix the datasets with the index.
      void setResetResults​(boolean value)
      Sets whether to clear the results before starting the experiment.
      void setResultsHandler​(AbstractResultsHandler value)
      Sets the results handler to use.
      void setRuns​(int value)
      Sets the number of runs.
      void setStatusMessageHandler​(adams.core.StatusMessageHandler value)
      Sets the status message handler to use.
      void setUseFilename​(boolean value)
      Sets whether to use the filename (w/o path) instead of the relationname.
      static String shortenCommandLine​(weka.classifiers.Classifier cls)
      Returns a shortened commandline string for the classifier.
      protected void showProgress()
      Updates the progress of the experiment.
      protected void sortResults()  
      void stopExecution()
      Stops the execution.
      weka.core.Instances toInstances()
      Returns the collected results.
      adams.data.spreadsheet.SpreadSheet toSpreadSheet()
      Returns the collected results.
      String useFilenameTipText()
      Returns the tip text for this property.
      • Methods inherited from class adams.core.option.AbstractOptionHandler

        cleanUpOptions, destroy, finishInit, getDefaultLoggingLevel, getOptionManager, globalInfo, initialize, loggingLevelTipText, newOptionManager, reset, setLoggingLevel, toCommandLine, toString
      • Methods inherited from class adams.core.logging.LoggingObject

        configureLogger, getLogger, getLoggingLevel, initializeLogging, isLoggingEnabled, sizeOf
      • Methods inherited from interface adams.core.logging.LoggingLevelHandler

        getLoggingLevel
    • Field Detail

      • m_ResetResults

        protected boolean m_ResetResults
        whether to reset the results before starting the experiment.
      • m_Classifiers

        protected weka.classifiers.Classifier[] m_Classifiers
        the classifiers to evaluate.
      • m_Datasets

        protected adams.core.io.PlaceholderFile[] m_Datasets
        the datasets to evaluate.
      • m_ClassLabelIndex

        protected adams.core.Index m_ClassLabelIndex
        the class label index for per-class stats.
      • m_UseFilename

        protected boolean m_UseFilename
        whether to use the filename (w/o path) instead of relationname.
      • m_PrefixDatasetsWithIndex

        protected boolean m_PrefixDatasetsWithIndex
        whether to prefix the relation names with the index.
      • m_Runs

        protected int m_Runs
        the number of runs.
      • m_Notes

        protected adams.core.base.BaseText m_Notes
        the notes for the experiment.
      • m_StatusMessageHandler

        protected transient adams.core.StatusMessageHandler m_StatusMessageHandler
        for notifications.
      • m_Running

        protected transient boolean m_Running
        whether the experiment is running.
      • m_Stopped

        protected transient boolean m_Stopped
        whether the experiment was stopped.
      • m_CommandLineHandler

        protected transient WekaCommandLineHandler m_CommandLineHandler
        for handling commandlines.
      • m_Results

        protected adams.data.spreadsheet.SpreadSheet m_Results
        the generated results.
      • m_Generated

        protected List<adams.data.spreadsheet.SpreadSheet> m_Generated
        the results generated by the evaluations.
      • m_JobRunner

        protected adams.multiprocess.JobRunner m_JobRunner
        the JobRunner template.
      • m_ActualJobRunner

        protected transient adams.multiprocess.JobRunner m_ActualJobRunner
        JobRunner for the classifier/dataset combinations.
      • m_JobCounter

        protected int m_JobCounter
        the counter for finished jobs.
      • m_JobTotal

        protected int m_JobTotal
        the total number of jobs.
    • Constructor Detail

      • AbstractExperiment

        public AbstractExperiment()
    • Method Detail

      • defineOptions

        public void defineOptions()
        Adds options to the internal list of options.
        Specified by:
        defineOptions in interface adams.core.option.OptionHandler
        Overrides:
        defineOptions in class adams.core.option.AbstractOptionHandler
      • setResetResults

        public void setResetResults​(boolean value)
        Sets whether to clear the results before starting the experiment.
        Specified by:
        setResetResults in interface ResettableExperiment
        Parameters:
        value - true if to clear results
      • getResetResults

        public boolean getResetResults()
        Returns whether to clear the results before starting the experiment.
        Specified by:
        getResetResults in interface ResettableExperiment
        Returns:
        true if to clear results
      • resetResultsTipText

        public String resetResultsTipText()
        Returns the tip text for this property.
        Returns:
        tip text for this property suitable for displaying in the GUI or for listing the options.
      • setResultsHandler

        public void setResultsHandler​(AbstractResultsHandler value)
        Sets the results handler to use.
        Parameters:
        value - the handler
      • getResultsHandler

        public AbstractResultsHandler getResultsHandler()
        Returns the results handler to use.
        Returns:
        the handler
      • resultsHandlerTipText

        public String resultsHandlerTipText()
        Returns the tip text for this property.
        Returns:
        tip text for this property suitable for displaying in the GUI or for listing the options.
      • setClassifiers

        public void setClassifiers​(weka.classifiers.Classifier[] value)
        Sets the classifiers to use.
        Parameters:
        value - the classifiers
      • getClassifiers

        public weka.classifiers.Classifier[] getClassifiers()
        Returns the classifiers.
        Returns:
        the classifiers
      • classifiersTipText

        public String classifiersTipText()
        Returns the tip text for this property.
        Returns:
        tip text for this property suitable for displaying in the GUI or for listing the options.
      • addClassifier

        public void addClassifier​(weka.classifiers.Classifier cls)
        Adds a classifier.
        Parameters:
        cls - the classifier to add
      • setDatasets

        public void setDatasets​(adams.core.io.PlaceholderFile[] value)
        Sets the datasets to use.
        Parameters:
        value - the datasets
      • getDatasets

        public adams.core.io.PlaceholderFile[] getDatasets()
        The datasets in use.
        Returns:
        the datasets
      • datasetsTipText

        public String datasetsTipText()
        Returns the tip text for this property.
        Returns:
        tip text for this property suitable for displaying in the GUI or for listing the options.
      • addDataset

        public void addDataset​(adams.core.io.PlaceholderFile file)
        Adds a dataset.
        Parameters:
        file - the dataset to add
      • setClassAttribute

        public void setClassAttribute​(AbstractClassAttributeHeuristic value)
        Sets the heuristic for determining the class attribute (if not explicitly set).
        Parameters:
        value - the heuristic
      • getClassAttribute

        public AbstractClassAttributeHeuristic getClassAttribute()
        Returns the heuristic for determining the class attribute (if not explicitly set).
        Returns:
        the heuristic
      • classAttributeTipText

        public String classAttributeTipText()
        Returns the tip text for this property.
        Returns:
        tip text for this property suitable for displaying in the GUI or for listing the options.
      • setClassLabelIndex

        public void setClassLabelIndex​(adams.core.Index value)
        Sets the index of the class label to use when generating per-class statistics.
        Parameters:
        value - the index
      • getClassLabelIndex

        public adams.core.Index getClassLabelIndex()
        Returns the index of the class label to use when generating per-class statistics.
        Returns:
        the index
      • classLabelIndexTipText

        public String classLabelIndexTipText()
        Returns the tip text for this property.
        Returns:
        tip text for this property suitable for displaying in the GUI or for listing the options.
      • setUseFilename

        public void setUseFilename​(boolean value)
        Sets whether to use the filename (w/o path) instead of the relationname.
        Specified by:
        setUseFilename in interface ExperimentWithCustomizableRelationNames
        Parameters:
        value - true if to use filename
      • useFilenameTipText

        public String useFilenameTipText()
        Returns the tip text for this property.
        Returns:
        tip text for this property suitable for displaying in the GUI or for listing the options.
      • prefixDatasetsWithIndexTipText

        public String prefixDatasetsWithIndexTipText()
        Returns the tip text for this property.
        Returns:
        tip text for this property suitable for displaying in the GUI or for listing the options.
      • setRuns

        public void setRuns​(int value)
        Sets the number of runs.
        Parameters:
        value - the runs
      • getRuns

        public int getRuns()
        Returns the number of runs.
        Returns:
        the runs
      • runsTipText

        public String runsTipText()
        Returns the tip text for this property.
        Returns:
        tip text for this property suitable for displaying in the GUI or for listing the options.
      • setNotes

        public void setNotes​(adams.core.base.BaseText value)
        Sets the notes for the experiment.
        Parameters:
        value - the notes
      • getNotes

        public adams.core.base.BaseText getNotes()
        Returns the notes for the experiment.
        Returns:
        the notes
      • notesTipText

        public String notesTipText()
        Returns the tip text for this property.
        Returns:
        tip text for this property suitable for displaying in the GUI or for listing the options.
      • setJobRunner

        public void setJobRunner​(adams.multiprocess.JobRunner value)
        Sets the jobrunner for the experiment.
        Parameters:
        value - the jobrunner
      • getJobRunner

        public adams.multiprocess.JobRunner getJobRunner()
        Returns the jobrunner for the experiment.
        Returns:
        the jobrunner
      • jobRunnerTipText

        public String jobRunnerTipText()
        Returns the tip text for this property.
        Returns:
        tip text for this property suitable for displaying in the GUI or for listing the options.
      • setStatusMessageHandler

        public void setStatusMessageHandler​(adams.core.StatusMessageHandler value)
        Sets the status message handler to use.
        Parameters:
        value - the handler, null to turn off
      • getStatusMessageHandler

        public adams.core.StatusMessageHandler getStatusMessageHandler()
        Returns the current status message handler in use.
        Returns:
        the handler, null if none set
      • preExecute

        protected String preExecute()
        Hook method just before the experiment is run (after initialization).
        Returns:
        null if successful, otherwise error message
      • toSpreadSheet

        public adams.data.spreadsheet.SpreadSheet toSpreadSheet()
        Returns the collected results.
        Specified by:
        toSpreadSheet in interface adams.data.spreadsheet.SpreadSheetSupporter
        Returns:
        the results
      • toInstances

        public weka.core.Instances toInstances()
        Returns the collected results.
        Returns:
        the results as Instances
      • log

        protected void log​(String msg)
        Displays the message.
        Parameters:
        msg - the message to display
      • log

        protected void log​(String msg,
                           Throwable t)
        Displays the error.
        Parameters:
        msg - the message to display
        t - the exception
      • initResults

        protected adams.data.spreadsheet.SpreadSheet initResults()
        Initializes the results.
        Returns:
        the results
      • initExecute

        protected String initExecute()
        Initializes the experiment.
      • loadDataset

        protected weka.core.Instances loadDataset​(int index)
        Loads the dataset.
        Parameters:
        index - the index of the dataset to load
        Returns:
        the dataset
      • configureRowFinder

        protected adams.data.spreadsheet.rowfinder.MultiRowFinder configureRowFinder​(int currentRun,
                                                                                     weka.classifiers.Classifier cls,
                                                                                     weka.core.Instances data)
        Configures the row finder that determines whether the classifier/dataset combination is still required.
        Parameters:
        currentRun - the current run
        cls - the classifier to check
        data - the dataset to check
        Returns:
        the row finder setup
      • isComplete

        protected boolean isComplete​(int[] rows)
        Checks whether the number of rows located in the current results are complete.
        Parameters:
        rows - the located results
        Returns:
        true if complete
      • locateRows

        protected int[] locateRows​(int run,
                                   weka.classifiers.Classifier cls,
                                   weka.core.Instances data)
        Returns the rows for the classifier/dataset combination.
        Parameters:
        run - the run
        cls - the classifier to check
        data - the dataset to check
        Returns:
        the rows
      • isRequired

        protected boolean isRequired​(int currentRun,
                                     weka.classifiers.Classifier cls,
                                     weka.core.Instances data)
        Checks whether the classifier/dataset combination is required.
        Parameters:
        currentRun - the current run
        cls - the classifier to check
        data - the dataset to check
        Returns:
        true if required
      • removeIncomplete

        protected void removeIncomplete​(int currentRun,
                                        weka.classifiers.Classifier cls,
                                        weka.core.Instances data)
        Removes the incomplete rows of the classifier/dataset combination.
        Parameters:
        currentRun - the current run
        cls - the classifier to check
        data - the dataset to check
      • showProgress

        protected void showProgress()
        Updates the progress of the experiment.
      • clearProgress

        protected void clearProgress()
        Clears the progress of the experiment.
      • initProgress

        public void initProgress()
        Initializes progress.
      • incProgress

        public void incProgress()
        Increments and updates the progress.
      • appendResults

        public void appendResults​(adams.data.spreadsheet.SpreadSheet results)
        Adds the results to the existing ones.
        Parameters:
        results - the results to add
      • evaluate

        protected abstract AbstractExperiment.AbstractExperimentJob<? extends AbstractExperiment> evaluate​(int currentRun,
                                                                                                           weka.classifiers.Classifier cls,
                                                                                                           weka.core.Instances data)
        Creates a runnabel to evaluate the classifier on the dataset.
        Parameters:
        currentRun - the current run
        cls - the classifier to evaluate
        data - the dataset to evaluate on
        Returns:
        the runnable
      • doExecute

        protected String doExecute()
        Runs the actual experiment.
        Returns:
        null if successful, otherwise error message
      • sortResults

        protected void sortResults()
      • postExecute

        protected void postExecute​(boolean success)
        Hook method just after the experiment was run.
        Parameters:
        success - true if successfully run
      • execute

        public String execute()
        Executes the experiment.
        Returns:
        null if successful, otherwise error message
      • stopExecution

        public void stopExecution()
        Stops the execution.
        Specified by:
        stopExecution in interface adams.core.Stoppable
      • isStopped

        public boolean isStopped()
        Returns whether the experiment has been stopped.
        Specified by:
        isStopped in interface adams.core.StoppableWithFeedback
        Returns:
        true if stopped
      • shortenCommandLine

        public static String shortenCommandLine​(weka.classifiers.Classifier cls)
        Returns a shortened commandline string for the classifier.
        Parameters:
        cls - the classifier to get the shortened commandline for
        Returns:
        the shortened commandline