Package adams.flow.control
Class AbstractDataContainerFileChecker<T extends DataContainer>
- java.lang.Object
-
- adams.core.logging.LoggingObject
-
- adams.core.logging.CustomLoggingLevelObject
-
- adams.core.option.AbstractOptionHandler
-
- adams.flow.core.AbstractActor
-
- adams.flow.control.AbstractControlActor
-
- adams.flow.control.AbstractTee
-
- adams.flow.control.Tee
-
- adams.flow.control.AbstractDataContainerFileChecker<T>
-
- Type Parameters:
T
- the type of container to read
- All Implemented Interfaces:
AdditionalInformationHandler
,CleanUpHandler
,Destroyable
,GlobalInfoSupporter
,LoggingLevelHandler
,LoggingSupporter
,OptionHandler
,Pausable
,QuickInfoSupporter
,ShallowCopySupporter<Actor>
,SizeOfHandler
,Stoppable
,StoppableWithFeedback
,VariablesInspectionHandler
,FlowPauseStateListener
,VariableChangeListener
,AtomicExecution
,Actor
,ActorHandler
,ActorWithConditionalEquivalent
,ActorWithTimedEquivalent
,ControlActor
,ErrorHandler
,Flushable
,InputConsumer
,MutableActorHandler
,OutputProducer
,SubFlowWrapUp
,Serializable
,Comparable
public abstract class AbstractDataContainerFileChecker<T extends DataContainer> extends Tee
Abstract ancestor for transformers that check data container files whether they are consistents before passing on the file/file arrays. These transformers keep an internal "temporary black list" of files that failed to load correctly. They are kept in this list until they either load correctly or they expire and get moved to the "permament black list". The content of the permanent black list can be written to a user-specified log file. Files that get added to the permanent black-list are also forwarded the tee actor.- Version:
- $Revision$
- Author:
- fracpete (fracpete at waikato dot ac dot nz)
- See Also:
- Serialized Form
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
AbstractDataContainerFileChecker.FileInfo
Container for storing additional information about a file, i.e., a timestamp when it first got black listed and a timestamp for when it should get checked again.-
Nested classes/interfaces inherited from class adams.flow.control.AbstractTee
AbstractTee.TeeSwingWorker
-
-
Field Summary
Fields Modifier and Type Field Description static String
BACKUP_FINALLIST
the key for storing the final list in the backup.static String
BACKUP_TEMPLIST
the key for storing the final list in the backup.protected AbstractDataContainerReader<T>
m_ActualReader
the actual reader to use for loading.protected Hashtable<File,AbstractDataContainerFileChecker.FileInfo>
m_BlackListFinal
the final blacklist.protected Hashtable<File,AbstractDataContainerFileChecker.FileInfo>
m_BlackListTemp
the temporary blacklist.protected BaseTime
m_CheckInterval
the check interval for temporary blacklisted items, before checking them again.protected BaseDateTime
m_ExpiryInterval
the expiry interval for temporary blacklisted items, before moving them into the final list.protected DateFormat
m_Formatter
the formatter for logging the dates.protected PlaceholderFile
m_Log
the (optional) log file to write the expired items to.protected AbstractDataContainerReader<T>
m_Reader
the reader to use for loading the data containers.-
Fields inherited from class adams.flow.control.AbstractTee
BACKUP_INPUT, m_Actors, m_Asynchronous, m_AsynchronousWorker, m_InputToken, m_MinimumActiveActors, m_OutputToken
-
Fields inherited from class adams.flow.control.AbstractControlActor
m_Compatibility, m_PauseStateManager, m_SetUpSubActors
-
Fields inherited from class adams.flow.core.AbstractActor
m_Annotations, m_BackupState, m_DetectedObjectVariables, m_DetectedVariables, m_ErrorHandler, m_Executed, m_Executing, m_ExecutionListeningSupporter, m_FullName, m_LoggingPrefix, m_Name, m_Parent, m_ScopeHandler, m_Self, m_Silent, m_Skip, m_StopFlowOnError, m_StopMessage, m_Stopped, m_StorageHandler, m_VariablesUpdated
-
Fields inherited from class adams.core.option.AbstractOptionHandler
m_OptionManager
-
Fields inherited from class adams.core.logging.LoggingObject
m_Logger, m_LoggingIsEnabled, m_LoggingLevel
-
Fields inherited from interface adams.flow.core.Actor
FILE_EXTENSION, FILE_EXTENSION_GZ
-
-
Constructor Summary
Constructors Constructor Description AbstractDataContainerFileChecker()
-
Method Summary
All Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description Class[]
accepts()
Returns the class that the consumer accepts.protected Hashtable<String,Object>
backupState()
Backs up the current state of the actor before update the variables.protected boolean
canProcessInput(Token token)
Tokens, i.e., files are not automatically forwarded to the tee actor, since only final black-listed ones are output there.String
checkIntervalTipText()
Returns the tip text for this property.void
cleanUp()
Cleans up after the execution has finished.void
defineOptions()
Adds options to the internal list of options.protected String
doExecute()
Executes the flow item.String
expiryIntervalTipText()
Returns the tip text for this property.Class[]
generates()
Returns the class of objects that it generates.BaseTime
getCheckInterval()
Returns the check interval after which blacklisted files get checked again whether they can finally be loaded correctly.protected abstract AbstractDataContainerReader<T>
getDefaultReader()
Returns the default reader to use.BaseDateTime
getExpiryInterval()
Returns the expiry interval after which blacklisted files get moved from the temporary list to the final list.protected DateFormat
getFormatter()
Returns the formatter for dates (used in the log).PlaceholderFile
getLog()
Returns the log file to write the files to that expired from the final blacklist.AbstractDataContainerReader<T>
getReader()
Returns the reader in use.protected void
initialize()
Initializes the members.protected boolean
isValid(File file)
Checks whether the file can be loaded correctly.protected void
logExpired(File file, AbstractDataContainerFileChecker.FileInfo info, Date expired)
Logs the expired file.String
logTipText()
Returns the tip text for this property.protected void
pruneBackup()
Removes entries from the backup.String
readerTipText()
Returns the tip text for this property.protected void
reset()
Resets the scheme.protected void
restoreState(Hashtable<String,Object> state)
Restores the state of the actor before the variables got updated.void
setCheckInterval(BaseTime value)
Sets the check interval after which blacklisted files get checked again whether they can finally be loaded correctly.void
setExpiryInterval(BaseDateTime value)
Sets the expiry interval after which blacklisted files get moved from the temporary list to the final list.void
setLog(PlaceholderFile value)
Sets the log file to write the files to that expired from the final blacklist.void
setReader(AbstractDataContainerReader<T> value)
Sets the reader to use.protected String
setUpTeeActors()
Gets called in the setUp() method.-
Methods inherited from class adams.flow.control.Tee
checkTeeActor, checkTeeActors, getActorHandlerInfo, getConditionalEquivalent, getTimedEquivalent, globalInfo
-
Methods inherited from class adams.flow.control.AbstractTee
actorsTipText, add, add, asynchronousTipText, check, createTeeToken, currentInput, execute, finishBeforeStoppingTipText, finishedAsynchronousExecution, flushExecution, forceVariables, get, getActors, getAsynchronous, getFinishBeforeStopping, getInternalActors, getMinimumActiveActors, getQuickInfo, getStoppingTimeout, hasInput, hasPendingOutput, indexOf, input, output, processInput, remove, removeAll, set, setActors, setAsynchronous, setFinishBeforeStopping, setLoggingLevel, setMinimumActiveActors, setSkip, setStoppingTimeout, setUpSubActors, size, stopExecution, stoppingTimeoutTipText, updateParent, waitForAsynchronousExecution, wrapUp
-
Methods inherited from class adams.flow.control.AbstractControlActor
active, canInspectOptions, destroy, firstActive, firstInputConsumer, flowPauseStateChanged, isPaused, isSubFlowWrappedUp, lastActive, pauseExecution, preExecute, resumeExecution, setParent, setUp, wrapUpSubFlow
-
Methods inherited from class adams.flow.core.AbstractActor
annotationsTipText, canPerformSetUpCheck, compareTo, configureLogger, equals, finalUpdateVariables, findVariables, findVariables, forCommandLine, forName, forName, getAdditionalInformation, getAnnotations, getDefaultName, getDetectedVariables, getErrorHandler, getFlowActors, getFlowExecutionListeningSupporter, getFullName, getName, getNextSibling, getParent, getParentComponent, getPreviousSibling, getRoot, getScopeHandler, getSilent, getSkip, getStopFlowOnError, getStopMessage, getStorageHandler, getVariables, handleError, handleException, hasErrorHandler, hasStopMessage, index, isBackedUp, isExecuted, isExecuting, isFinished, isHeadless, isStopped, nameTipText, performSetUpChecks, performVariableChecks, postExecute, pruneBackup, setAnnotations, setErrorHandler, setName, setSilent, setStopFlowOnError, setVariables, shallowCopy, shallowCopy, silentTipText, sizeOf, skipTipText, stopExecution, stopFlowOnErrorTipText, updateDetectedVariables, updatePrefix, updateVariables, variableChanged
-
Methods inherited from class adams.core.option.AbstractOptionHandler
cleanUpOptions, finishInit, getDefaultLoggingLevel, getOptionManager, loggingLevelTipText, newOptionManager, toCommandLine, toString
-
Methods inherited from class adams.core.logging.LoggingObject
getLogger, getLoggingLevel, initializeLogging, isLoggingEnabled
-
Methods inherited from class java.lang.Object
clone, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
-
Methods inherited from interface adams.flow.core.Actor
compareTo, destroy, equals, execute, findVariables, getAnnotations, getDefaultName, getDetectedVariables, getErrorHandler, getFlowExecutionListeningSupporter, getFullName, getName, getNextSibling, getParent, getParentComponent, getPreviousSibling, getQuickInfo, getRoot, getScopeHandler, getSilent, getSkip, getStopFlowOnError, getStopMessage, getStorageHandler, getVariables, handleError, hasErrorHandler, hasStopMessage, index, isExecuted, isFinished, isHeadless, isStopped, setAnnotations, setErrorHandler, setName, setParent, setSilent, setSkip, setStopFlowOnError, setUp, setVariables, shallowCopy, shallowCopy, sizeOf, stopExecution, stopExecution, toCommandLine, variableChanged, wrapUp
-
Methods inherited from interface adams.flow.core.ActorHandler
firstActive, lastActive
-
Methods inherited from interface adams.core.AdditionalInformationHandler
getAdditionalInformation
-
Methods inherited from interface adams.core.logging.LoggingLevelHandler
getLoggingLevel, setLoggingLevel
-
Methods inherited from interface adams.core.logging.LoggingSupporter
getLogger, isLoggingEnabled
-
Methods inherited from interface adams.core.option.OptionHandler
cleanUpOptions, getOptionManager
-
Methods inherited from interface adams.core.VariablesInspectionHandler
canInspectOptions
-
-
-
-
Field Detail
-
BACKUP_TEMPLIST
public static final String BACKUP_TEMPLIST
the key for storing the final list in the backup.- See Also:
- Constant Field Values
-
BACKUP_FINALLIST
public static final String BACKUP_FINALLIST
the key for storing the final list in the backup.- See Also:
- Constant Field Values
-
m_BlackListTemp
protected Hashtable<File,AbstractDataContainerFileChecker.FileInfo> m_BlackListTemp
the temporary blacklist.
-
m_BlackListFinal
protected Hashtable<File,AbstractDataContainerFileChecker.FileInfo> m_BlackListFinal
the final blacklist.
-
m_Reader
protected AbstractDataContainerReader<T extends DataContainer> m_Reader
the reader to use for loading the data containers.
-
m_ActualReader
protected AbstractDataContainerReader<T extends DataContainer> m_ActualReader
the actual reader to use for loading.
-
m_ExpiryInterval
protected BaseDateTime m_ExpiryInterval
the expiry interval for temporary blacklisted items, before moving them into the final list.
-
m_CheckInterval
protected BaseTime m_CheckInterval
the check interval for temporary blacklisted items, before checking them again.
-
m_Log
protected PlaceholderFile m_Log
the (optional) log file to write the expired items to.
-
m_Formatter
protected transient DateFormat m_Formatter
the formatter for logging the dates.
-
-
Method Detail
-
defineOptions
public void defineOptions()
Adds options to the internal list of options.- Specified by:
defineOptions
in interfaceOptionHandler
- Overrides:
defineOptions
in classAbstractTee
-
initialize
protected void initialize()
Initializes the members.- Overrides:
initialize
in classAbstractTee
-
reset
protected void reset()
Resets the scheme.- Overrides:
reset
in classAbstractControlActor
-
getFormatter
protected DateFormat getFormatter()
Returns the formatter for dates (used in the log).- Returns:
- the formatter
-
pruneBackup
protected void pruneBackup()
Removes entries from the backup.- Overrides:
pruneBackup
in classAbstractActor
- See Also:
AbstractActor.reset()
-
backupState
protected Hashtable<String,Object> backupState()
Backs up the current state of the actor before update the variables.- Overrides:
backupState
in classAbstractTee
- Returns:
- the backup
- See Also:
AbstractActor.updateVariables()
,AbstractActor.restoreState(Hashtable)
-
restoreState
protected void restoreState(Hashtable<String,Object> state)
Restores the state of the actor before the variables got updated.- Overrides:
restoreState
in classAbstractTee
- Parameters:
state
- the backup of the state to restore from- See Also:
AbstractActor.updateVariables()
,AbstractActor.backupState()
-
getDefaultReader
protected abstract AbstractDataContainerReader<T> getDefaultReader()
Returns the default reader to use.- Returns:
- the reader
-
setReader
public void setReader(AbstractDataContainerReader<T> value)
Sets the reader to use.- Parameters:
value
- the reader
-
getReader
public AbstractDataContainerReader<T> getReader()
Returns the reader in use.- Returns:
- the reader
-
readerTipText
public String readerTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setExpiryInterval
public void setExpiryInterval(BaseDateTime value)
Sets the expiry interval after which blacklisted files get moved from the temporary list to the final list.- Parameters:
value
- the time interval specification, e.g., "START +1 HOUR"
-
getExpiryInterval
public BaseDateTime getExpiryInterval()
Returns the expiry interval after which blacklisted files get moved from the temporary list to the final list.- Returns:
- the time interval specification, e.g., "START +1 HOUR"
-
expiryIntervalTipText
public String expiryIntervalTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setCheckInterval
public void setCheckInterval(BaseTime value)
Sets the check interval after which blacklisted files get checked again whether they can finally be loaded correctly.- Parameters:
value
- the time interval specification, e.g., "START +1 HOUR"
-
getCheckInterval
public BaseTime getCheckInterval()
Returns the check interval after which blacklisted files get checked again whether they can finally be loaded correctly.- Returns:
- the time interval specification, e.g., "START +1 HOUR"
-
checkIntervalTipText
public String checkIntervalTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
setLog
public void setLog(PlaceholderFile value)
Sets the log file to write the files to that expired from the final blacklist. If pointing to a directory, nothing gets written.- Parameters:
value
- the log file
-
getLog
public PlaceholderFile getLog()
Returns the log file to write the files to that expired from the final blacklist. If pointing to a directory, nothing gets written.- Returns:
- the log file
-
logTipText
public String logTipText()
Returns the tip text for this property.- Returns:
- tip text for this property suitable for displaying in the GUI or for listing the options.
-
accepts
public Class[] accepts()
Returns the class that the consumer accepts.- Specified by:
accepts
in interfaceInputConsumer
- Overrides:
accepts
in classAbstractTee
- Returns:
- File, File[]
-
generates
public Class[] generates()
Returns the class of objects that it generates.- Specified by:
generates
in interfaceOutputProducer
- Overrides:
generates
in classAbstractTee
- Returns:
- File, File[]
-
setUpTeeActors
protected String setUpTeeActors()
Gets called in the setUp() method. Returns null if tee-actor is fine, otherwise error message.- Overrides:
setUpTeeActors
in classTee
- Returns:
- null if everything OK, otherwise error message
-
isValid
protected boolean isValid(File file)
Checks whether the file can be loaded correctly.- Parameters:
file
- the file to check- Returns:
- true if the file can be loaded correctly
-
logExpired
protected void logExpired(File file, AbstractDataContainerFileChecker.FileInfo info, Date expired)
Logs the expired file. Does nothing if log is pointing to a directory.- Parameters:
file
- the file to loginfo
- the information about the fileexpired
- the timestamp when the file expired
-
canProcessInput
protected boolean canProcessInput(Token token)
Tokens, i.e., files are not automatically forwarded to the tee actor, since only final black-listed ones are output there.- Overrides:
canProcessInput
in classAbstractTee
- Parameters:
token
- the token to process- Returns:
- always false
-
doExecute
protected String doExecute()
Executes the flow item.- Overrides:
doExecute
in classAbstractTee
- Returns:
- null if everything is fine, otherwise error message
-
cleanUp
public void cleanUp()
Cleans up after the execution has finished. Also removes graphical components.- Specified by:
cleanUp
in interfaceActor
- Specified by:
cleanUp
in interfaceCleanUpHandler
- Overrides:
cleanUp
in classAbstractTee
-
-