co.cask.format.avro.input

Class PathTrackingAvroInputFormat

java.lang.Object

org.apache.hadoop.mapreduce.InputFormat<K,V>

org.apache.hadoop.mapreduce.lib.input.FileInputFormat<org.apache.hadoop.io.NullWritable,co.cask.cdap.api.data.format.StructuredRecord>

co.cask.hydrator.format.input.PathTrackingInputFormat

co.cask.format.avro.input.PathTrackingAvroInputFormat

public class PathTrackingAvroInputFormat
extends PathTrackingInputFormat

Avro format that tracks which file each record was read from.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

org.apache.hadoop.mapreduce.lib.input.FileInputFormat.Counter

Field Summary

Fields inherited from class co.cask.hydrator.format.input.PathTrackingInputFormat

COPY_HEADER

Fields inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

INPUT_DIR, INPUT_DIR_RECURSIVE, NUM_INPUT_FILES, PATHFILTER_CLASS, SPLIT_MAXSIZE, SPLIT_MINSIZE

Constructor Summary

Constructors

Constructor and Description
PathTrackingAvroInputFormat()

Method Summary

Modifier and Type	Method and Description
protected org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.NullWritable,co.cask.cdap.api.data.format.StructuredRecord.Builder>	createRecordReader(org.apache.hadoop.mapreduce.lib.input.FileSplit split, org.apache.hadoop.mapreduce.TaskAttemptContext context, String pathField, co.cask.cdap.api.data.schema.Schema schema)

Methods inherited from class co.cask.hydrator.format.input.PathTrackingInputFormat

createRecordReader

Methods inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

addInputPath, addInputPathRecursively, addInputPaths, computeSplitSize, getBlockIndex, getFormatMinSplitSize, getInputDirRecursive, getInputPathFilter, getInputPaths, getMaxSplitSize, getMinSplitSize, getSplits, isSplitable, listStatus, makeSplit, setInputDirRecursive, setInputPathFilter, setInputPaths, setInputPaths, setMaxInputSplitSize, setMinInputSplitSize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PathTrackingAvroInputFormat

public PathTrackingAvroInputFormat()

Method Detail

createRecordReader

protected org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.NullWritable,co.cask.cdap.api.data.format.StructuredRecord.Builder> createRecordReader(org.apache.hadoop.mapreduce.lib.input.FileSplit split,
                                                                                                                                                               org.apache.hadoop.mapreduce.TaskAttemptContext context,
                                                                                                                                                               @Nullable
                                                                                                                                                               String pathField,
                                                                                                                                                               @Nullable
                                                                                                                                                               co.cask.cdap.api.data.schema.Schema schema)
                                                                                                                                                        throws IOException,
                                                                                                                                                               InterruptedException

Specified by:

createRecordReader in class PathTrackingInputFormat

Throws:

IOException

InterruptedException