C F G I P R S T W

C

co.cask.cdap.etl.api.batch - package co.cask.cdap.etl.api.batch

Interfaces for Spark plugins.

co.cask.cdap.etl.api.streaming - package co.cask.cdap.etl.api.streaming

configurePipeline(PipelineConfigurer) - Method in class co.cask.cdap.etl.api.batch.SparkCompute

Configure a pipeline.

configurePipeline(PipelineConfigurer) - Method in class co.cask.cdap.etl.api.streaming.StreamingSource

configurePipeline(PipelineConfigurer) - Method in class co.cask.cdap.etl.api.streaming.Windower

createSparkInterpreter() - Method in interface co.cask.cdap.etl.api.batch.SparkExecutionPluginContext

Creates a new instance of SparkInterpreter for Scala code compilation and interpretation.

F

fromDataset(String) - Method in interface co.cask.cdap.etl.api.batch.SparkExecutionPluginContext

Creates a JavaPairRDD from the given Dataset.

fromDataset(String, Map<String, String>) - Method in interface co.cask.cdap.etl.api.batch.SparkExecutionPluginContext

Creates a JavaPairRDD from the given Dataset with the given set of Dataset arguments.

fromDataset(String, Map<String, String>, Iterable<? extends Split>) - Method in interface co.cask.cdap.etl.api.batch.SparkExecutionPluginContext

Creates a JavaPairRDD from the given Dataset with the given set of Dataset arguments and custom list of Splits.

fromStream(String) - Method in interface co.cask.cdap.etl.api.batch.SparkExecutionPluginContext

Creates a JavaRDD that represents all events from the given stream.

fromStream(String, long, long) - Method in interface co.cask.cdap.etl.api.batch.SparkExecutionPluginContext

Creates a JavaRDD that represents events from the given stream in the given time range.

fromStream(String, Class<V>) - Method in interface co.cask.cdap.etl.api.batch.SparkExecutionPluginContext

Creates a JavaPairRDD that represents all events from the given stream.

fromStream(String, long, long, Class<V>) - Method in interface co.cask.cdap.etl.api.batch.SparkExecutionPluginContext

Creates a JavaPairRDD that represents events from the given stream in the given time range.

fromStream(String, long, long, Class<? extends StreamEventDecoder<K, V>>, Class<K>, Class<V>) - Method in interface co.cask.cdap.etl.api.batch.SparkExecutionPluginContext

Creates a JavaPairRDD that represents events from the given stream in the given time range.

G

getLogicalStartTime() - Method in interface co.cask.cdap.etl.api.batch.SparkExecutionPluginContext

Returns the logical start time of the Batch Job.

getPluginContext() - Method in interface co.cask.cdap.etl.api.batch.SparkExecutionPluginContext

Returns a Serializable PluginContext which can be used to request for plugins instances.

getRequiredExecutors() - Method in class co.cask.cdap.etl.api.streaming.StreamingSource

Get number of required executors for the streaming source.

getRuntimeArguments() - Method in interface co.cask.cdap.etl.api.batch.SparkExecutionPluginContext

Returns runtime arguments of the Batch Job.

getSlideInterval() - Method in class co.cask.cdap.etl.api.streaming.Windower

getSparkContext() - Method in interface co.cask.cdap.etl.api.batch.SparkExecutionPluginContext

Returns the JavaSparkContext used during the execution.

getSparkExecutionContext() - Method in interface co.cask.cdap.etl.api.streaming.StreamingContext

getSparkStreamingContext() - Method in interface co.cask.cdap.etl.api.streaming.StreamingContext

getStream(StreamingContext) - Method in class co.cask.cdap.etl.api.streaming.StreamingSource

Get the stream to read from.

getWidth() - Method in class co.cask.cdap.etl.api.streaming.Windower

I

initialize(SparkExecutionPluginContext) - Method in class co.cask.cdap.etl.api.batch.SparkCompute

Initialize the plugin.

P

PLUGIN_TYPE - Static variable in class co.cask.cdap.etl.api.batch.SparkCompute

PLUGIN_TYPE - Static variable in class co.cask.cdap.etl.api.batch.SparkSink

PLUGIN_TYPE - Static variable in class co.cask.cdap.etl.api.streaming.StreamingSource

PLUGIN_TYPE - Static variable in class co.cask.cdap.etl.api.streaming.Windower

R

registerLineage(String) - Method in interface co.cask.cdap.etl.api.streaming.StreamingContext

Register lineage for this Spark program using the given reference name

run(SparkExecutionPluginContext, JavaRDD<IN>) - Method in class co.cask.cdap.etl.api.batch.SparkSink

User Spark job which will be executed and is responsible for persisting any data.

S

saveAsDataset(JavaPairRDD<K, V>, String) - Method in interface co.cask.cdap.etl.api.batch.SparkExecutionPluginContext

Saves the given JavaPairRDD to the given Dataset.

saveAsDataset(JavaPairRDD<K, V>, String, Map<String, String>) - Method in interface co.cask.cdap.etl.api.batch.SparkExecutionPluginContext

Saves the given JavaPairRDD to the given Dataset with the given set of Dataset arguments.

setSparkConf(SparkConf) - Method in interface co.cask.cdap.etl.api.batch.SparkPluginContext

Sets a SparkConf to be used for the Spark execution.

SparkCompute<IN,OUT> - Class in co.cask.cdap.etl.api.batch

Spark Compute stage.

SparkCompute() - Constructor for class co.cask.cdap.etl.api.batch.SparkCompute

SparkExecutionPluginContext - Interface in co.cask.cdap.etl.api.batch

Context passed to spark plugin types.

SparkPluginContext - Interface in co.cask.cdap.etl.api.batch

Context passed to spark plugin types during prepare run phase.

SparkSink<IN> - Class in co.cask.cdap.etl.api.batch

SparkSink composes a final, optional stage of a Batch ETL Pipeline.

SparkSink() - Constructor for class co.cask.cdap.etl.api.batch.SparkSink

StreamingContext - Interface in co.cask.cdap.etl.api.streaming

Context for streaming plugin stages.

StreamingSource<T> - Class in co.cask.cdap.etl.api.streaming

Source for Spark Streaming pipelines.

StreamingSource() - Constructor for class co.cask.cdap.etl.api.streaming.StreamingSource

T

transform(SparkExecutionPluginContext, JavaRDD<IN>) - Method in class co.cask.cdap.etl.api.batch.SparkCompute

Transform the input and return the output to be sent to the next stage in the pipeline.

W

Windower - Class in co.cask.cdap.etl.api.streaming

Windowing plugin.

Windower() - Constructor for class co.cask.cdap.etl.api.streaming.Windower

C F G I P R S T W