- MnistDbFile - Class in org.canova.image.mnist
-
MNIST database file containing entries that can represent image or label
data.
- MnistDbFile(String, String) - Constructor for class org.canova.image.mnist.MnistDbFile
-
Creates new instance and reads the header information.
- MnistFetcher - Class in org.canova.image.mnist
-
- MnistFetcher() - Constructor for class org.canova.image.mnist.MnistFetcher
-
- MnistImageFile - Class in org.canova.image.mnist
-
MNIST database image file.
- MnistImageFile(String, String) - Constructor for class org.canova.image.mnist.MnistImageFile
-
Creates new MNIST database image file ready for reading.
- MNISTInputFormat - Class in org.canova.image.format
-
- MNISTInputFormat() - Constructor for class org.canova.image.format.MNISTInputFormat
-
- MnistLabelFile - Class in org.canova.image.mnist
-
MNIST database label file.
- MnistLabelFile(String, String) - Constructor for class org.canova.image.mnist.MnistLabelFile
-
Creates new MNIST database label file ready for reading.
- MnistManager - Class in org.canova.image.mnist
-
Utility class for working with the MNIST database.
- MnistManager(String, String) - Constructor for class org.canova.image.mnist.MnistManager
-
Constructs an instance managing the two given data files.
- MNISTRecordReader - Class in org.canova.image.recordreader
-
Record reader that understands the MNIST file format as described here:
http://yann.lecun.com/exdb/mnist/
Not built to handle splits of the file, for now forces a single worker to process file
Why?
- the MNIST training file is 47MB unzipped
- right now (June 2015) Canova only runs in local/serial mode
- when we add MapReduce as a runtime engine, the training file size (47MB) is still smaller
than the lowest production block size in HDFS these days (64MB, 128MB), so normally MapReduce's
scheduling system would not split the file (unless you manually set the file's block size lower)
- This input format's main purpose is to read MNIST raw data into Canova to be written out
as another format (SVMLight most likely) for DL4J's input format's to read
Assumes that file exists locally and has been unzipped
Why?
- When we do port this input format to be HDFS-aware, these mechanics will be incompatible
(we dont want N workers all trying to download files or coordinate who is downloading the file)
- MNISTRecordReader() - Constructor for class org.canova.image.recordreader.MNISTRecordReader
-