Package adams.flow.transformer

Class AbstractDataProcessor

java.lang.Object

adams.core.logging.LoggingObject

adams.core.logging.CustomLoggingLevelObject

adams.core.option.AbstractOptionHandler

adams.flow.core.AbstractActor

adams.flow.transformer.AbstractTransformer

adams.flow.transformer.AbstractDataProcessor

All Implemented Interfaces:

AdditionalInformationHandler, AtomicMoveSupporter, CleanUpHandler, Destroyable, GlobalInfoSupporter, LoggingLevelHandler, LoggingSupporter, OptionHandler, QuickInfoSupporter, ShallowCopySupporter<Actor>, SizeOfHandler, Stoppable, StoppableWithFeedback, VariablesInspectionHandler, VariableChangeListener, Actor, ErrorHandler, InputConsumer, OutputProducer, Serializable, Comparable

Direct Known Subclasses:

AbstractDbDataProcessor, FileProcessor

public abstract class AbstractDataProcessor
extends AbstractTransformer
implements AtomicMoveSupporter

Abstract ancestor for classes that process data on disk.

Version:

$Revision$

Author:

fracpete (fracpete at waikato dot ac dot nz)

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field	Description
protected boolean	m_AtomicMove	whether to perform an atomic move.
protected File	m_DestinationFile	the final resting place of the processed file (failed or not).
protected PlaceholderDirectory	m_Failed	the directory containing the elements that couldn't be processed.
protected PlaceholderDirectory	m_Processed	the directory containing the elements that were processed successfully.
protected String	m_ProcessError	the last error that occurred during processing.
protected PlaceholderDirectory	m_Processing	the directory containing the elements being processed.
protected SimpleDateFormat	m_TimestampDirFormat	the format for the timestamp directories.
protected boolean	m_UseTimestampDirs	whether to create directories with a timestamp as name for the files that are being processed, in order to avoid name clashes.

Fields inherited from class adams.flow.transformer.AbstractTransformer

BACKUP_INPUT, BACKUP_OUTPUT, m_InputToken, m_OutputToken

Fields inherited from class adams.flow.core.AbstractActor

m_Annotations, m_BackupState, m_DetectedObjectVariables, m_DetectedVariables, m_ErrorHandler, m_Executed, m_Executing, m_ExecutionListeningSupporter, m_FullName, m_LoggingPrefix, m_Name, m_Parent, m_ScopeHandler, m_Self, m_Silent, m_Skip, m_StopFlowOnError, m_StopMessage, m_Stopped, m_StorageHandler, m_VariablesUpdated

Fields inherited from class adams.core.option.AbstractOptionHandler

m_OptionManager

Fields inherited from class adams.core.logging.LoggingObject

m_Logger, m_LoggingIsEnabled, m_LoggingLevel

Fields inherited from interface adams.flow.core.Actor

FILE_EXTENSION, FILE_EXTENSION_GZ

Constructor Summary

Constructors

Constructor	Description
AbstractDataProcessor()

Method Summary

Modifier and Type	Method	Description
Class[]	accepts()	Returns the class that the consumer accepts.
String	atomicMoveTipText()	Returns the tip text for this property.
void	defineOptions()	Adds options to the internal list of options.
protected String	doExecute()	Executes the flow item.
String	failedTipText()	Returns the tip text for this property.
boolean	getAtomicMove()	Returns whether to attempt atomic move operation.
PlaceholderDirectory	getFailed()	Returns the "failed" directory.
PlaceholderDirectory	getProcessed()	Returns the "processed" directory.
PlaceholderDirectory	getProcessing()	Returns the "processing" directory.
boolean	getUseTimestampDirs()	Returns whether files/dirs that are being processed get encapsulated in timestamp directories or not.
protected void	initialize()	Initializes the members.
protected File	nextTimestampDir(File parent)	Returns the next available timestamp directory in the given parent directory.
protected boolean	postProcessData(File file, boolean success)	Post-processes the data.
protected File	prepareData(File file)	Moves the data to the "processing" directory.
protected abstract boolean	processData(File file)	Processes the given data.
String	processedTipText()	Returns the tip text for this property.
String	processingTipText()	Returns the tip text for this property.
void	setAtomicMove(boolean value)	Sets whether to attempt atomic move operation.
void	setFailed(PlaceholderDirectory value)	Sets the "failed" directory.
void	setProcessed(PlaceholderDirectory value)	Sets the "processed" directory.
void	setProcessing(PlaceholderDirectory value)	Sets the "processing" directory.
void	setUseTimestampDirs(boolean value)	Sets whether to use timestamp directories to avoid name clashes of the files/dirs being processed.
String	useTimestampDirsTipText()	Returns the tip text for this property.

Methods inherited from class adams.flow.transformer.AbstractTransformer

backupState, currentInput, execute, hasInput, hasPendingOutput, input, output, postExecute, restoreState, wrapUp

Methods inherited from class adams.flow.core.AbstractActor

annotationsTipText, canInspectOptions, canPerformSetUpCheck, cleanUp, compareTo, configureLogger, destroy, equals, finalUpdateVariables, findVariables, findVariables, forceVariables, forCommandLine, forName, forName, getAdditionalInformation, getAnnotations, getDefaultName, getDetectedVariables, getErrorHandler, getFlowActors, getFlowExecutionListeningSupporter, getFullName, getName, getNextSibling, getParent, getParentComponent, getPreviousSibling, getQuickInfo, getRoot, getScopeHandler, getSilent, getSkip, getStopFlowOnError, getStopMessage, getStorageHandler, getVariables, handleError, handleException, hasErrorHandler, hasStopMessage, index, isBackedUp, isExecuted, isExecuting, isFinished, isHeadless, isStopped, nameTipText, performSetUpChecks, performVariableChecks, preExecute, pruneBackup, pruneBackup, reset, setAnnotations, setErrorHandler, setName, setParent, setSilent, setSkip, setStopFlowOnError, setUp, setVariables, shallowCopy, shallowCopy, silentTipText, sizeOf, skipTipText, stopExecution, stopExecution, stopFlowOnErrorTipText, updateDetectedVariables, updatePrefix, updateVariables, variableChanged

Methods inherited from class adams.core.option.AbstractOptionHandler

cleanUpOptions, finishInit, getDefaultLoggingLevel, getOptionManager, globalInfo, loggingLevelTipText, newOptionManager, setLoggingLevel, toCommandLine, toString

Methods inherited from class adams.core.logging.LoggingObject

getLogger, getLoggingLevel, initializeLogging, isLoggingEnabled

Methods inherited from class java.lang.Object

clone, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface adams.flow.core.Actor

cleanUp, compareTo, destroy, equals, findVariables, getAnnotations, getDefaultName, getDetectedVariables, getErrorHandler, getFlowExecutionListeningSupporter, getFullName, getName, getNextSibling, getParent, getParentComponent, getPreviousSibling, getQuickInfo, getRoot, getScopeHandler, getSilent, getSkip, getStopFlowOnError, getStopMessage, getStorageHandler, getVariables, handleError, hasErrorHandler, hasStopMessage, index, isExecuted, isFinished, isHeadless, isStopped, setAnnotations, setErrorHandler, setName, setParent, setSilent, setSkip, setStopFlowOnError, setUp, setVariables, shallowCopy, shallowCopy, sizeOf, stopExecution, stopExecution, toCommandLine, variableChanged

Methods inherited from interface adams.core.AdditionalInformationHandler

getAdditionalInformation

Methods inherited from interface adams.core.logging.LoggingLevelHandler

getLoggingLevel, setLoggingLevel

Methods inherited from interface adams.core.logging.LoggingSupporter

getLogger, isLoggingEnabled

Methods inherited from interface adams.core.option.OptionHandler

cleanUpOptions, getOptionManager

Methods inherited from interface adams.flow.core.OutputProducer

generates

Methods inherited from interface adams.core.VariablesInspectionHandler

canInspectOptions

Field Detail

m_Processing

protected PlaceholderDirectory m_Processing

the directory containing the elements being processed.

m_Processed

protected PlaceholderDirectory m_Processed

the directory containing the elements that were processed successfully.

m_Failed

protected PlaceholderDirectory m_Failed

the directory containing the elements that couldn't be processed.

m_ProcessError

protected String m_ProcessError

the last error that occurred during processing.

m_UseTimestampDirs

protected boolean m_UseTimestampDirs

whether to create directories with a timestamp as name for the files that are being processed, in order to avoid name clashes.

m_TimestampDirFormat

protected SimpleDateFormat m_TimestampDirFormat

the format for the timestamp directories.

m_DestinationFile

protected File m_DestinationFile

the final resting place of the processed file (failed or not).

m_AtomicMove

protected boolean m_AtomicMove

whether to perform an atomic move.

Constructor Detail

AbstractDataProcessor

public AbstractDataProcessor()

Method Detail

defineOptions

public void defineOptions()

Adds options to the internal list of options.

Specified by:

defineOptions in interface OptionHandler

Overrides:

defineOptions in class AbstractActor

initialize

protected void initialize()

Initializes the members.

Overrides:

initialize in class AbstractActor

setProcessing

public void setProcessing(PlaceholderDirectory value)

Sets the "processing" directory.

Parameters:

value - the directory

getProcessing

public PlaceholderDirectory getProcessing()

Returns the "processing" directory.

Returns:

the directory

processingTipText

public String processingTipText()

Returns the tip text for this property.

Returns:

tip text for this property suitable for displaying in the GUI or for listing the options.

setProcessed

public void setProcessed(PlaceholderDirectory value)

Sets the "processed" directory.

Parameters:

value - the directory

getProcessed

public PlaceholderDirectory getProcessed()

Returns the "processed" directory.

Returns:

the directory

processedTipText

public String processedTipText()

Returns the tip text for this property.

Returns:

tip text for this property suitable for displaying in the GUI or for listing the options.

setFailed

public void setFailed(PlaceholderDirectory value)

Sets the "failed" directory.

Parameters:

value - the directory

getFailed

public PlaceholderDirectory getFailed()

Returns the "failed" directory.

Returns:

the directory

failedTipText

public String failedTipText()

Returns the tip text for this property.

Returns:

tip text for this property suitable for displaying in the GUI or for listing the options.

setUseTimestampDirs

public void setUseTimestampDirs(boolean value)

Sets whether to use timestamp directories to avoid name clashes of the files/dirs being processed.

Parameters:

value - if truen then the files/dirs get encapsulated in timestamp dir

getUseTimestampDirs

public boolean getUseTimestampDirs()

Returns whether files/dirs that are being processed get encapsulated in timestamp directories or not.

Returns:

true if files/dirs get encapsulated

useTimestampDirsTipText

public String useTimestampDirsTipText()

Returns the tip text for this property.

Returns:

tip text for this property suitable for displaying in the GUI or for listing the options.

setAtomicMove

public void setAtomicMove(boolean value)

Sets whether to attempt atomic move operation.

Specified by:

setAtomicMove in interface AtomicMoveSupporter

Parameters:

value - if true then attempt atomic move operation

getAtomicMove

public boolean getAtomicMove()

Returns whether to attempt atomic move operation.

Specified by:

getAtomicMove in interface AtomicMoveSupporter

Returns:

true if to attempt atomic move operation

atomicMoveTipText

public String atomicMoveTipText()

Returns the tip text for this property.

Specified by:

atomicMoveTipText in interface AtomicMoveSupporter

Returns:

tip text for this property suitable for displaying in the GUI or for listing the options.

accepts

public Class[] accepts()

Returns the class that the consumer accepts.

Specified by:

accepts in interface InputConsumer

Returns:

String.class, File.class

nextTimestampDir

protected File nextTimestampDir(File parent)

Returns the next available timestamp directory in the given parent directory.

Parameters:

parent - the directory in which to create the timestamp dir

Returns:

the generated timestamp directory

prepareData

protected File prepareData(File file)

Moves the data to the "processing" directory.

Parameters:

file - the file/dir to move

Returns:

the new location

processData

protected abstract boolean processData(File file)

Processes the given data.

Parameters:

file - the file/dir to process

Returns:

true if everything went alright

See Also:

m_ProcessError

postProcessData

protected boolean postProcessData(File file,
                                  boolean success)

Post-processes the data.

Parameters:

file - the file to finish up

success - whether processing was successful

Returns:

true if successful

See Also:

m_ProcessError

doExecute

protected String doExecute()

Executes the flow item.

Specified by:

doExecute in class AbstractActor

Returns:

null if everything is fine, otherwise error message