org.canova.api.split

Class LimitFileSplit

java.lang.Object

org.canova.api.split.BaseInputSplit

org.canova.api.split.FileSplit

org.canova.api.split.LimitFileSplit

All Implemented Interfaces:

Serializable, InputSplit, Writable

public class LimitFileSplit
extends FileSplit

File input split that allows limits on number of files loaded. Define total number of examples to pull from the dataset. Use numCategories to split the numExamples and pull equal number of examples per category. Use pattern and patternPosition to segment the filename that represents the category. Created by nyghtowl on 11/9/15.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected static org.slf4j.Logger	log
protected int	numCategories
protected String	pattern
protected int	patternPosition
protected int	totalNumExamples

Fields inherited from class org.canova.api.split.FileSplit

allowFormat, random, randomize, recursive, rootDir

Fields inherited from class org.canova.api.split.BaseInputSplit

length, locations

Constructor Summary

Constructors

Constructor and Description
LimitFileSplit(File rootDir, int numExamples)
LimitFileSplit(File rootDir, int numExamples, int numCategories, String pattern)
LimitFileSplit(File rootDir, int numExamples, Random random)
LimitFileSplit(File rootDir, int numExamples, String pattern)
LimitFileSplit(File rootDir, int numExamples, String pattern, Random random)
LimitFileSplit(File rootDir, String[] allowFormat, boolean recursive, int numExamples)
LimitFileSplit(File rootDir, String[] allowFormat, boolean recursive, int numExamples, int numCategories, String pattern, int patternPosition, Random random)
LimitFileSplit(File rootDir, String[] allowFormat, int numExamples, int numCategories, String pattern)
LimitFileSplit(File rootDir, String[] allowFormat, int numExamples, int numCategories, String pattern, int patternPosition, Random random)
LimitFileSplit(File rootDir, String[] allowFormat, int numExamples, int numCategories, String pattern, Random random)
LimitFileSplit(File rootDir, String[] allowFormat, int numExamples, String pattern, int patternPosition, Random random)
LimitFileSplit(File rootDir, String[] allowFormat, int numExamples, String pattern, Random random)

Method Summary

Modifier and Type	Method and Description
protected void	initialize()

Methods inherited from class org.canova.api.split.FileSplit

getRootDir, length, readFields, write

Methods inherited from class org.canova.api.split.BaseInputSplit

locations, sample, toDouble, toFloat, toInt, toLong

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

log

protected static org.slf4j.Logger log

totalNumExamples

protected int totalNumExamples

numCategories

protected int numCategories

pattern

protected String pattern

patternPosition

protected int patternPosition

Constructor Detail

LimitFileSplit

public LimitFileSplit(File rootDir,
                      String[] allowFormat,
                      boolean recursive,
                      int numExamples,
                      int numCategories,
                      String pattern,
                      int patternPosition,
                      Random random)

LimitFileSplit

public LimitFileSplit(File rootDir,
                      int numExamples)

LimitFileSplit

public LimitFileSplit(File rootDir,
                      String[] allowFormat,
                      boolean recursive,
                      int numExamples)

LimitFileSplit

public LimitFileSplit(File rootDir,
                      int numExamples,
                      String pattern)

LimitFileSplit

public LimitFileSplit(File rootDir,
                      int numExamples,
                      int numCategories,
                      String pattern)

LimitFileSplit

public LimitFileSplit(File rootDir,
                      String[] allowFormat,
                      int numExamples,
                      int numCategories,
                      String pattern)

LimitFileSplit

public LimitFileSplit(File rootDir,
                      String[] allowFormat,
                      int numExamples,
                      int numCategories,
                      String pattern,
                      int patternPosition,
                      Random random)

LimitFileSplit

public LimitFileSplit(File rootDir,
                      int numExamples,
                      Random random)

LimitFileSplit

public LimitFileSplit(File rootDir,
                      int numExamples,
                      String pattern,
                      Random random)

LimitFileSplit

public LimitFileSplit(File rootDir,
                      String[] allowFormat,
                      int numExamples,
                      int numCategories,
                      String pattern,
                      Random random)

LimitFileSplit

public LimitFileSplit(File rootDir,
                      String[] allowFormat,
                      int numExamples,
                      String pattern,
                      Random random)

LimitFileSplit

public LimitFileSplit(File rootDir,
                      String[] allowFormat,
                      int numExamples,
                      String pattern,
                      int patternPosition,
                      Random random)

Method Detail

initialize

protected void initialize()

Overrides:

initialize in class FileSplit