org.canova.api.records.reader.impl.regex

Class RegexLineRecordReader

java.lang.Object

org.canova.api.records.reader.impl.LineRecordReader

org.canova.api.records.reader.impl.regex.RegexLineRecordReader

All Implemented Interfaces:

Closeable, Serializable, AutoCloseable, Configurable, RecordReader

public class RegexLineRecordReader
extends LineRecordReader

RegexLineRecordReader: Read a file, one line at a time, and split it into fields using a regex. Specifically, we are using Pattern and Matcher.
To load an entire file using a Example: Data in format "2016-01-01 23:59:59.001 1 DEBUG First entry message!"
using regex String "(\\d{4}-\\d{2}-\\d{2} \\d{2}:\\d{2}:\\d{2}\\.\\d{3}) (\\d+) ([A-Z]+) (.*)"
would be split into 4 Text writables: ["2016-01-01 23:59:59.001", "1", "DEBUG", "First entry message!"]

Author:

Alex Black

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
static String	SKIP_NUM_LINES

Fields inherited from class org.canova.api.records.reader.impl.LineRecordReader

conf, inputSplit

Fields inherited from interface org.canova.api.records.reader.RecordReader

APPEND_LABEL, LABELS, NAME_SPACE

Constructor Summary

Constructors

Constructor and Description
RegexLineRecordReader(String regex, int skipNumLines)

Method Summary

Modifier and Type	Method and Description
void	initialize(Configuration conf, InputSplit split) Called once at initialization.
Collection<Writable>	next() Get the next record
Collection<Writable>	record(URI uri, DataInputStream dataInputStream) Load the record from the given DataInputStream Unlike RecordReader.next() the internal state of the RecordReader is not modified Implementations of this method should not close the DataInputStream
void	reset() Reset record reader iterator

Methods inherited from class org.canova.api.records.reader.impl.LineRecordReader

close, getConf, getLabels, hasNext, initialize, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

SKIP_NUM_LINES

public static final String SKIP_NUM_LINES

Constructor Detail

RegexLineRecordReader

public RegexLineRecordReader(String regex,
                             int skipNumLines)

Method Detail

initialize

public void initialize(Configuration conf,
                       InputSplit split)
                throws IOException,
                       InterruptedException

Description copied from interface: RecordReader

Called once at initialization.

Specified by:

initialize in interface RecordReader

Overrides:

initialize in class LineRecordReader

Parameters:

conf - a configuration for initialization

split - the split that defines the range of records to read

Throws:

IOException

InterruptedException

next

public Collection<Writable> next()

Description copied from interface: RecordReader

Get the next record

Specified by:

next in interface RecordReader

Overrides:

next in class LineRecordReader

Returns:

record

public Collection<Writable> record(URI uri,
                                   DataInputStream dataInputStream)
                            throws IOException

Description copied from interface: RecordReader

Load the record from the given DataInputStream Unlike RecordReader.next() the internal state of the RecordReader is not modified Implementations of this method should not close the DataInputStream

Specified by:

record in interface RecordReader

Overrides:

record in class LineRecordReader

Throws:

IOException - if error occurs during reading from the input stream

reset

public void reset()

Description copied from interface: RecordReader

Reset record reader iterator

Specified by:

reset in interface RecordReader

Overrides:

reset in class LineRecordReader