co.cask.hydrator.plugin.spark

Class ReferenceStreamingSource<T>

java.lang.Object

co.cask.cdap.etl.api.streaming.StreamingSource<T>

co.cask.hydrator.plugin.spark.ReferenceStreamingSource<T>

Type Parameters:

T - type of object read by the source.

All Implemented Interfaces:

PipelineConfigurable, Serializable

Direct Known Subclasses:

FileStreamingSource, TwitterStreamingSource

public abstract class ReferenceStreamingSource<T>
extends StreamingSource<T>

Base streaming source that adds an External Dataset for a reference name, and performs a single getDataset() call to make sure CDAP records that it was accessed.

See Also:

Serialized Form

Field Summary

Fields inherited from class co.cask.cdap.etl.api.streaming.StreamingSource

PLUGIN_TYPE

Constructor Summary

Constructors

Constructor and Description
ReferenceStreamingSource(ReferencePluginConfig conf)

Method Summary

Modifier and Type	Method and Description
void	configurePipeline(PipelineConfigurer pipelineConfigurer)

Methods inherited from class co.cask.cdap.etl.api.streaming.StreamingSource

getRequiredExecutors, getStream

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ReferenceStreamingSource

public ReferenceStreamingSource(ReferencePluginConfig conf)

Method Detail

configurePipeline

public void configurePipeline(PipelineConfigurer pipelineConfigurer)
                       throws IllegalArgumentException

Specified by:

configurePipeline in interface PipelineConfigurable

Overrides:

configurePipeline in class StreamingSource<T>

Throws:

IllegalArgumentException