co.cask.cdap.etl.api.streaming

Interface StreamingContext

All Superinterfaces:

ServiceDiscoverer, StageContext, Transactional

@Beta
public interface StreamingContext
extends StageContext, Transactional

Context for streaming plugin stages.

Method Summary

Modifier and Type	Method and Description
JavaSparkExecutionContext	getSparkExecutionContext()
org.apache.spark.streaming.api.java.JavaStreamingContext	getSparkStreamingContext()
void	registerLineage(String referenceName) Register lineage for this Spark program using the given reference name

Methods inherited from interface co.cask.cdap.etl.api.StageContext

getArguments, getInputSchema, getInputSchemas, getLogicalStartTime, getMetrics, getNamespace, getOutputPortSchemas, getOutputSchema, getPipelineName, getPluginProperties, getPluginProperties, getStageName, loadPluginClass, newPluginInstance

Methods inherited from interface co.cask.cdap.api.ServiceDiscoverer

getServiceURL, getServiceURL

Methods inherited from interface co.cask.cdap.api.Transactional

execute, execute

Method Detail

getSparkStreamingContext

org.apache.spark.streaming.api.java.JavaStreamingContext getSparkStreamingContext()

Returns:

Spark JavaStreamingContext for the pipeline.

getSparkExecutionContext

JavaSparkExecutionContext getSparkExecutionContext()

Returns:

CDAP JavaSparkExecutionContext for the pipeline.

registerLineage

void registerLineage(String referenceName)
              throws DatasetManagementException,
                     org.apache.tephra.TransactionFailureException

Register lineage for this Spark program using the given reference name

Parameters:

referenceName - reference name used for source

Throws:

DatasetManagementException - thrown if there was an error in creating reference dataset

org.apache.tephra.TransactionFailureException - thrown if there was an error while fetching the dataset to register usage