adams.flow.transformer
Class TwitterConverter

java.lang.Object
  adams.core.ConsoleObject
      adams.core.option.AbstractOptionHandler
          adams.flow.core.AbstractActor
              adams.flow.transformer.AbstractTransformer
                  adams.flow.transformer.TwitterConverter

All Implemented Interfaces:

AdditionalInformationHandler, CleanUpHandler, Debuggable, DebugOutputHandler, Destroyable, OptionHandler, QuickInfoSupporter, ShallowCopySupporter<AbstractActor>, SizeOfHandler, Stoppable, TechnicalInformationHandler, VariableChangeListener, Actor, ErrorHandler, InputConsumer, OutputProducer, Serializable, Comparable

public class TwitterConverter
extends AbstractTransformer
implements TechnicalInformationHandler

Turns a Twitter tweet or status into different representation.
Please note that tweet and status objects differ in what fields are available:
- tweet-specific: LANGUAGE_CODE
- status-specific: PLACE, COUNTRY, COUNTRY_CODE
Also, GEO location data might not be available.
For more information on the date format, see:
Javadoc. java.text.SimpleDateFormat.

BibTeX:

 @misc{missing_id,
    author = {Javadoc},
    title = {java.text.SimpleDateFormat},
    HTTP = {http://download.oracle.com/javase/1,5.0/docs/api/java/text/SimpleDateFormat.html}
 }
 

Input/output:
- accepts:
twitter4j.Tweet
twitter4j.Status
- generates:
java.lang.String

Valid options are:

-D <int> (property: debugLevel)
    The greater the number the more additional info the scheme may output to
    the console (0 = off).
    default: 0
    minimum: 0
 

-name <java.lang.String> (property: name)
    The name of the actor.
    default: TwitterConverter
 

-annotation <adams.core.base.BaseText> (property: annotations)
    The annotations to attach to this actor.
    default:
 

-skip (property: skip)
    If set to true, transformation is skipped and the input token is just forwarded
    as it is.
 

-stop-flow-on-error (property: stopFlowOnError)
    If set to true, the flow gets stopped in case this actor encounters an error;
     useful for critical actors.
 

-output-type <STRING|INSTANCE> (property: outputType)
    The type of output to generate.
    default: STRING
 

-field <ID|USER_ID|USER_NAME|SOURCE|TEXT|CREATED|GEO_LATITUDE|GEO_LONGITUDE|LANGUAGE_CODE|PLACE|COUNTRY|COUNTRY_CODE> [-field ...] (property: fields)
    The fields to use for generating the output.
    default: TEXT
 

-separator <java.lang.String> (property: separator)
    The separator to use when generating strings as output; tab, new line, carriage
    return and backslash need to be escaped, ie, '   ', '
    ', '
', '\'.
    default: \t
 

-quote (property: quote)
    If enabled all sub-strings are quoted if necessary when generating string
    output.
 

-date-format <java.lang.String> (property: dateFormat)
    The format for the dates.
    default: yyyy-MM-dd HH:mm:ss
 

Version:

$Revision: 5867 $

Author:

fracpete (fracpete at waikato dot ac dot nz)

See Also:

Serialized Form

Nested Class Summary

static class	TwitterConverter.OutputType The output to generate.
static class	TwitterConverter.TwitterField The available fields for generating the output.

Field Summary

static String	BACKUP_HEADER the key for storing the current header in the backup.
protected String	m_DateFormat for format for dates.
protected DateFormat	m_DateFormatter for formatting/parsing dates.
protected TwitterConverter.TwitterField[]	m_Fields the fields to generate the output from.
protected Instances	m_Header the dataset header when generating Instance objects.
protected TwitterConverter.OutputType	m_OutputType the type of output to generate.
protected boolean	m_Quote whether to quote strings.
protected String	m_Separator the separator, when generating a string.

Fields inherited from class adams.flow.transformer.AbstractTransformer

BACKUP_INPUT, BACKUP_OUTPUT, m_InputToken, m_OutputToken

Fields inherited from class adams.flow.core.AbstractActor

m_Annotations, m_BackupState, m_DetectedObjectVariables, m_DetectedVariables, m_ErrorHandler, m_Executed, m_Executing, m_FullName, m_Headless, m_Name, m_Parent, m_Root, m_Self, m_Skip, m_StopFlowOnError, m_StopMessage, m_Stopped, m_StorageHandler, m_VariablesUpdated

Fields inherited from class adams.core.option.AbstractOptionHandler

m_DebugLevel, m_OptionManager

Fields inherited from interface adams.flow.core.Actor

FILE_EXTENSION, FILE_EXTENSION_GZ

Constructor Summary

TwitterConverter()

Method Summary

Class[]	accepts() Returns the class that the consumer accepts.
protected Hashtable<String,Object>	backupState() Backs up the current state of the actor before update the variables.
String	dateFormatTipText() Returns the tip text for this property.
void	defineOptions() Adds options to the internal list of options.
protected String	doExecute() Executes the flow item.
String	fieldsTipText() Returns the tip text for this property.
protected void	generateHeader() Generates the dataset header, if necessary.
Class[]	generates() Returns the class of objects that it generates.
String	getDateFormat() Returns the current date format.
protected DateFormat	getDateFormatter() Returns the date formatter to use.
TwitterConverter.TwitterField[]	getFields() Returns the fields to generate the output from.
TwitterConverter.OutputType	getOutputType() Returns the type of output to generate.
String	getQuickInfo() Returns a quick info about the actor, which will be displayed in the GUI.
boolean	getQuote() Returns whether to quote the sub-strings when generating string output.
String	getSeparator() Returns the separator in use.
TechnicalInformation	getTechnicalInformation() Returns an instance of a TechnicalInformation object, containing detailed information about the technical background of this class, e.g., paper reference or book this class is based on.
String	globalInfo() Returns a string describing the object.
protected void	initialize() Initializes the members.
String	outputTypeTipText() Returns the tip text for this property.
protected Token	processFields(Hashtable<TwitterConverter.TwitterField,Object> fields) Creates an output token from the given fields.
protected Hashtable<TwitterConverter.TwitterField,Object>	processStatus(twitter4j.Status status) Processes the specified status.
protected Hashtable<TwitterConverter.TwitterField,Object>	processTweet(twitter4j.Tweet tweet) Processes the specified tweet.
protected void	pruneBackup() Removes entries from the backup.
String	quoteTipText() Returns the tip text for this property.
protected void	restoreState(Hashtable<String,Object> state) Restores the state of the actor before the variables got updated.
String	separatorTipText() Returns the tip text for this property.
void	setDateFormat(String value) Sets the date format.
void	setFields(TwitterConverter.TwitterField[] value) Sets fields to generate the output from.
void	setOutputType(TwitterConverter.OutputType value) Sets the type of output to generate.
void	setQuote(boolean value) Sets whether to quote the sub-strings when generating string output.
void	setSeparator(String value) Sets the separator to use.

Methods inherited from class adams.flow.transformer.AbstractTransformer

execute, hasPendingOutput, input, output, postExecute, reset, wrapUp

Methods inherited from class adams.flow.core.AbstractActor

annotationsTipText, canInspectOptions, canPerformSetUpCheck, cleanUp, compareTo, debug, destroy, equals, findVariables, findVariables, findVariables, forceVariables, forCommandLine, forName, getAdditionalInformation, getAnnotations, getDefaultName, getDetectedVariables, getErrorHandler, getFlowActors, getFullName, getName, getNextSibling, getParent, getPreviousSibling, getRoot, getSkip, getStopFlowOnError, getStopMessage, getStorageHandler, getVariables, handleError, handleException, hasErrorHandler, hasStopMessage, index, isBackedUp, isExecuted, isExecuting, isFinished, isHeadless, isStopped, nameTipText, performSetUpChecks, preExecute, pruneBackup, setAnnotations, setErrorHandler, setHeadless, setName, setParent, setSkip, setStopFlowOnError, setUp, setVariables, shallowCopy, shallowCopy, sizeOf, skipTipText, stopExecution, stopExecution, stopFlowOnErrorTipText, updateDetectedVariables, updatePrefix, updateVariables, variableChanged

Methods inherited from class adams.core.option.AbstractOptionHandler

cleanUpOptions, debug, debugLevelTipText, finishInit, getDebugLevel, getOptionManager, isDebugOn, newOptionManager, setDebugLevel, toCommandLine, toString

Methods inherited from class adams.core.ConsoleObject

getDebugging, getSystemErr, getSystemOut

Methods inherited from class java.lang.Object

clone, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface adams.flow.core.Actor

cleanUp, compareTo, debug, destroy, equals, findVariables, getAnnotations, getDefaultName, getDetectedVariables, getErrorHandler, getFullName, getName, getNextSibling, getParent, getPreviousSibling, getRoot, getSkip, getStopFlowOnError, getStopMessage, getStorageHandler, getVariables, handleError, hasErrorHandler, hasStopMessage, index, isExecuted, isFinished, isHeadless, isStopped, setAnnotations, setErrorHandler, setHeadless, setName, setParent, setSkip, setStopFlowOnError, setUp, setVariables, sizeOf, stopExecution, stopExecution, variableChanged

Methods inherited from interface adams.core.AdditionalInformationHandler

getAdditionalInformation

Methods inherited from interface adams.core.option.OptionHandler

cleanUpOptions, getOptionManager

Field Detail

BACKUP_HEADER

public static final String BACKUP_HEADER

the key for storing the current header in the backup.

See Also:

Constant Field Values

m_OutputType

protected TwitterConverter.OutputType m_OutputType

the type of output to generate.

m_Fields

protected TwitterConverter.TwitterField[] m_Fields

the fields to generate the output from.

m_Separator

protected String m_Separator

the separator, when generating a string.

m_Quote

protected boolean m_Quote

whether to quote strings.

m_Header

protected Instances m_Header

the dataset header when generating Instance objects.

m_DateFormat

protected String m_DateFormat

for format for dates.

m_DateFormatter

protected transient DateFormat m_DateFormatter

for formatting/parsing dates.

Constructor Detail

TwitterConverter

public TwitterConverter()

Method Detail

globalInfo

public String globalInfo()

Returns a string describing the object.

Specified by:

globalInfo in class AbstractOptionHandler

Returns:

a description suitable for displaying in the gui

getTechnicalInformation

public TechnicalInformation getTechnicalInformation()

Returns an instance of a TechnicalInformation object, containing detailed information about the technical background of this class, e.g., paper reference or book this class is based on.

Specified by:

getTechnicalInformation in interface TechnicalInformationHandler

Returns:

the technical information about this class

defineOptions

public void defineOptions()

Adds options to the internal list of options.

Specified by:

defineOptions in interface OptionHandler

Overrides:

defineOptions in class AbstractActor

initialize

protected void initialize()

Initializes the members.

Overrides:

initialize in class AbstractActor

setOutputType

public void setOutputType(TwitterConverter.OutputType value)

Sets the type of output to generate.

Parameters:

value - the type

getOutputType

public TwitterConverter.OutputType getOutputType()

Returns the type of output to generate.

Returns:

the type

outputTypeTipText

public String outputTypeTipText()

Returns the tip text for this property.

Returns:

tip text for this property suitable for displaying in the GUI or for listing the options.

setFields

public void setFields(TwitterConverter.TwitterField[] value)

Sets fields to generate the output from.

Parameters:

value - the fields

getFields

public TwitterConverter.TwitterField[] getFields()

Returns the fields to generate the output from.

Returns:

the fields

fieldsTipText

public String fieldsTipText()

Returns the tip text for this property.

Returns:

tip text for this property suitable for displaying in the GUI or for listing the options.

setSeparator

public void setSeparator(String value)

Sets the separator to use. \t, \n, \r, \\ must be quoted.

Parameters:

value - the separator

getSeparator

public String getSeparator()

Returns the separator in use. \t, \r, \n, \\ get returned quoted.

Returns:

the separator

separatorTipText

public String separatorTipText()

Returns the tip text for this property.

Returns:

tip text for this property suitable for displaying in the GUI or for listing the options.

setQuote

public void setQuote(boolean value)

Sets whether to quote the sub-strings when generating string output.

Parameters:

value - if true then the sub-strings get quoted

getQuote

public boolean getQuote()

Returns whether to quote the sub-strings when generating string output.

Returns:

true if quoting enabled

quoteTipText

public String quoteTipText()

Returns the tip text for this property.

Returns:

tip text for this property suitable for displaying in the GUI or for listing the options.

setDateFormat

public void setDateFormat(String value)

Sets the date format. See SimpleDateFormat for more information.

Parameters:

value - the separator

getDateFormat

public String getDateFormat()

Returns the current date format.

Returns:

the date format

dateFormatTipText

public String dateFormatTipText()

Returns the tip text for this property.

Returns:

tip text for this property suitable for displaying in the GUI or for listing the options.

getQuickInfo

public String getQuickInfo()

Returns a quick info about the actor, which will be displayed in the GUI.

Specified by:

getQuickInfo in interface QuickInfoSupporter

Specified by:

getQuickInfo in interface Actor

Overrides:

getQuickInfo in class AbstractActor

Returns:

null if no info available, otherwise short string

accepts

public Class[] accepts()

Returns the class that the consumer accepts.

Specified by:

accepts in interface InputConsumer

Returns:

twitter4j.Tweet.class, twitter4j.Status.class

generates

public Class[] generates()

Returns the class of objects that it generates.

Specified by:

generates in interface OutputProducer

Returns:

java.lang.String.class

pruneBackup

protected void pruneBackup()

Removes entries from the backup.

Overrides:

pruneBackup in class AbstractActor

backupState

protected Hashtable<String,Object> backupState()

Backs up the current state of the actor before update the variables.

Overrides:

backupState in class AbstractTransformer

Returns:

the backup

restoreState

protected void restoreState(Hashtable<String,Object> state)

Restores the state of the actor before the variables got updated.

Overrides:

restoreState in class AbstractTransformer

Parameters:

state - the backup of the state to restore from

getDateFormatter

protected DateFormat getDateFormatter()

Returns the date formatter to use.

Returns:

the formatter

generateHeader

protected void generateHeader()

Generates the dataset header, if necessary.

processTweet

protected Hashtable<TwitterConverter.TwitterField,Object> processTweet(twitter4j.Tweet tweet)

Processes the specified tweet.

Parameters:

tweet - the tweet to process

Returns:

the association between fields and tweet values

processStatus

protected Hashtable<TwitterConverter.TwitterField,Object> processStatus(twitter4j.Status status)

Processes the specified status.

Parameters:

status - the status to process

Returns:

the association between fields and status values

processFields

protected Token processFields(Hashtable<TwitterConverter.TwitterField,Object> fields)

Creates an output token from the given fields.

Parameters:

fields - the data to use

Returns:

the generated token

doExecute

protected String doExecute()

Executes the flow item.

Specified by:

doExecute in class AbstractActor

Returns:

null if everything is fine, otherwise error message