A B C D E F G I J O R S T

A

AbstractExtendedSpark - Class in co.cask.cdap.api.spark

This abstract class provides extra features at configure time to Spark program through the ExtendedSparkConfigurer provided by the AbstractExtendedSpark.getConfigurer() method.

AbstractExtendedSpark() - Constructor for class co.cask.cdap.api.spark.AbstractExtendedSpark

AbstractSparkHttpServiceHandler - Class in co.cask.cdap.api.spark.service

An abstract implementation of SparkHttpServiceHandler.

AbstractSparkHttpServiceHandler() - Constructor for class co.cask.cdap.api.spark.service.AbstractSparkHttpServiceHandler

addHandlers(SparkHttpServiceHandler...) - Method in class co.cask.cdap.api.spark.AbstractExtendedSpark

Adds a list of SparkHttpServiceHandlers to runs in the Spark driver.

addHandlers(Iterable<? extends SparkHttpServiceHandler>) - Method in class co.cask.cdap.api.spark.AbstractExtendedSpark

Adds a list of SparkHttpServiceHandlers to runs in the Spark driver.

addHandlers(SparkHttpServiceHandler...) - Method in interface co.cask.cdap.api.spark.ExtendedSparkConfigurer

Adds a list of SparkHttpServiceHandlers to runs in the Spark driver.

addHandlers(Iterable<? extends SparkHttpServiceHandler>) - Method in interface co.cask.cdap.api.spark.ExtendedSparkConfigurer

Adds a list of SparkHttpServiceHandlers to runs in the Spark driver.

B

BindingException - Exception in co.cask.cdap.api.spark.dynamic

Exception to indicate there is an error in binding variable.

BindingException(String, String, Object, String...) - Constructor for exception co.cask.cdap.api.spark.dynamic.BindingException

C

close() - Method in interface co.cask.cdap.api.spark.service.SparkHttpServicePluginContext

Close and release resources owned by this object.

co.cask.cdap.api.spark - package co.cask.cdap.api.spark

Interfaces for Spark interactions.

co.cask.cdap.api.spark.dynamic - package co.cask.cdap.api.spark.dynamic

co.cask.cdap.api.spark.service - package co.cask.cdap.api.spark.service

co.cask.cdap.api.spark.sql - package co.cask.cdap.api.spark.sql

CompilationFailureException - Exception in co.cask.cdap.api.spark.dynamic

Exception to indicate there is a code compilation error.

CompilationFailureException(String) - Constructor for exception co.cask.cdap.api.spark.dynamic.CompilationFailureException

createInterpreter() - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a new instance of SparkInterpreter for Scala code compilation and interpretation.

createSparkCompiler() - Method in interface co.cask.cdap.api.spark.ExtendedSparkConfigurer

Creates a new instance of SparkCompiler for Scala code compilation.

D

DataFrames - Class in co.cask.cdap.api.spark.sql

Utility class for conversions between DataType and Schema.

destroy() - Method in class co.cask.cdap.api.spark.service.AbstractSparkHttpServiceHandler

destroy() - Method in interface co.cask.cdap.api.spark.service.SparkHttpServiceHandler

This method is called when the http service is about to shutdown.

E

execute(int, TxRunnable) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Transactions with a custom timeout are not supported in Spark.

ExtendedSparkConfigurer - Interface in co.cask.cdap.api.spark

Extends the functionality of SparkConfigurer to provide more spark specific feature at configuration time.

F

fromDataset(String) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD from the given Dataset.

fromDataset(String, String) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD from the given Dataset.

fromDataset(String, Map<String, String>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD from the given Dataset with the given set of dataset arguments.

fromDataset(String, String, Map<String, String>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD from the given Dataset with the given set of dataset arguments.

fromDataset(String, Map<String, String>, Iterable<? extends Split>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD from the given Dataset with the given set of dataset arguments and custom list of Splits.

fromDataset(String, String, Map<String, String>, Iterable<? extends Split>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD from the given Dataset with the given set of dataset arguments and custom list of Splits.

fromRow(Row, Schema) - Static method in class co.cask.cdap.api.spark.sql.DataFrames

Creates a StructuredRecord from the data in the given Row.

fromStream(String) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaRDD that represents all events from the given stream.

fromStream(String, String) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaRDD that represents all events from the given stream.

fromStream(String, long, long) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaRDD that represents events from the given stream in the given time range.

fromStream(String, String, long, long) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaRDD that represents events from the given stream in the given time range.

fromStream(String, Class<V>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD that represents all events from the given stream.

fromStream(String, String, Class<V>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD that represents all events from the given stream.

fromStream(String, long, long, Class<V>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD that represents events from the given stream in the given time range.

fromStream(String, String, long, long, Class<V>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD that represents events from the given stream in the given time range.

fromStream(String, long, long, Class<? extends StreamEventDecoder<K, V>>, Class<K>, Class<V>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD that represents events from the given stream in the given time range.

fromStream(String, String, long, long, Class<? extends StreamEventDecoder<K, V>>, Class<K>, Class<V>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD that represents events from the given stream in the given time range.

fromStream(String, FormatSpecification, Class<T>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD that represents all events from the given stream.

fromStream(String, String, FormatSpecification, Class<T>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD that represents all events from the given stream.

fromStream(String, FormatSpecification, long, long, Class<T>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD that represents data from the given stream for events in the given time range.

fromStream(String, String, FormatSpecification, long, long, Class<T>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD that represents data from the given stream for events in the given time range.

G

getConfigurer() - Method in class co.cask.cdap.api.spark.AbstractExtendedSpark

Returns the ExtendedSparkConfigurer for extended features, only available at configuration time.

getContext() - Method in class co.cask.cdap.api.spark.service.AbstractSparkHttpServiceHandler

Returns the SparkHttpServiceContext object that was passed to the AbstractSparkHttpServiceHandler.initialize(SparkHttpServiceContext) method.

getJavaSparkContext() - Method in interface co.cask.cdap.api.spark.service.SparkHttpServiceContextBase

Returns the JavaSparkContext wrapper for the SparkContext object created in the Spark driver.

getLocalizationContext() - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Returns a Serializable TaskLocalizationContext which can be used to retrieve files localized to task containers.

getLogicalStartTime() - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Returns the logical start time of this Spark job.

getMessagingContext() - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Returns a MessagingContext which can be used to interact with the transactional messaging system.

getMetrics() - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Returns a Serializable Metrics which can be used to emit custom metrics from user's Spark program.

getPluginContext() - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Returns a Serializable PluginContext which can be used to request for plugins instances.

getPluginContext() - Method in interface co.cask.cdap.api.spark.service.SparkHttpServiceContextBase

Returns a SparkHttpServicePluginContext for adding and using plugins.

getSecureStore() - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Returns a Serializable SecureStore which can be used to request for plugins instances.

getServiceDiscoverer() - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Returns a Serializable ServiceDiscoverer for Service Discovery in Spark Program which can be passed in Spark program's closures.

getSparkContext() - Method in interface co.cask.cdap.api.spark.service.SparkHttpServiceContextBase

Returns the SparkContext object created in the Spark driver.

getSparkExecutionContext() - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Returns the underlying SparkExecutionContext used by this object.

getSpecification() - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

I

initialize(SparkHttpServiceContext) - Method in class co.cask.cdap.api.spark.service.AbstractSparkHttpServiceHandler

initialize(SparkHttpServiceContext) - Method in interface co.cask.cdap.api.spark.service.SparkHttpServiceHandler

Initializes the service handler.

InterpretFailureException - Exception in co.cask.cdap.api.spark.dynamic

Exception to indicate there is a code interpretation error.

InterpretFailureException(String) - Constructor for exception co.cask.cdap.api.spark.dynamic.InterpretFailureException

J

JavaSparkExecutionContext - Class in co.cask.cdap.api.spark

Spark program execution context.

JavaSparkExecutionContext() - Constructor for class co.cask.cdap.api.spark.JavaSparkExecutionContext

JavaSparkExecutionContextBase - Class in co.cask.cdap.api.spark

Spark program execution context.

JavaSparkExecutionContextBase() - Constructor for class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

JavaSparkMain - Interface in co.cask.cdap.api.spark

A Java interface for Spark program to implement.

O

onError(HttpServiceResponder, Throwable) - Method in class co.cask.cdap.api.spark.service.SparkHttpContentConsumer

onError(Throwable) - Method in class co.cask.cdap.api.spark.service.SparkHttpContentProducer

onFinish(HttpServiceResponder) - Method in class co.cask.cdap.api.spark.service.SparkHttpContentConsumer

onFinish() - Method in class co.cask.cdap.api.spark.service.SparkHttpContentProducer

R

run(JavaSparkExecutionContext) - Method in interface co.cask.cdap.api.spark.JavaSparkMain

This method will be called when the Spark program starts.

S

saveAsDataset(JavaPairRDD<K, V>, String) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Saves the given JavaPairRDD to the given Dataset.

saveAsDataset(JavaPairRDD<K, V>, String, String) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Saves the given JavaPairRDD to the given Dataset.

saveAsDataset(JavaPairRDD<K, V>, String, Map<String, String>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Saves the given JavaPairRDD to the given Dataset with the given set of Dataset arguments.

saveAsDataset(JavaPairRDD<K, V>, String, String, Map<String, String>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Saves the given JavaPairRDD to the given Dataset with the given set of dataset arguments.

SparkHttpContentConsumer - Class in co.cask.cdap.api.spark.service

A HttpContentConsumer for SparkHttpServiceHandler for large request body consumption.

SparkHttpContentConsumer() - Constructor for class co.cask.cdap.api.spark.service.SparkHttpContentConsumer

SparkHttpContentProducer - Class in co.cask.cdap.api.spark.service

A HttpContentProducer for SparkHttpServiceHandler to stream large response body.

SparkHttpContentProducer() - Constructor for class co.cask.cdap.api.spark.service.SparkHttpContentProducer

SparkHttpServiceContext - Interface in co.cask.cdap.api.spark.service

Context interface for providing access to SparkContext and CDAP functionality.

SparkHttpServiceContextBase - Interface in co.cask.cdap.api.spark.service

A base context interface for providing access to SparkContext and CDAP functionality that are common to Spark 1 and Spark 2.

SparkHttpServiceHandler - Interface in co.cask.cdap.api.spark.service

Interface to be implemented to handle HTTP requests to the Spark driver process.

SparkHttpServicePluginContext - Interface in co.cask.cdap.api.spark.service

A PluginContext for Spark HTTP service.

T

toDataType(Schema) - Static method in class co.cask.cdap.api.spark.sql.DataFrames

Converts a Schema to Spark DataType.

toDataType(Schema, Function1<Schema, DataType>) - Static method in class co.cask.cdap.api.spark.sql.DataFrames

Converts a Schema to Spark DataType.

toRow(StructuredRecord, StructType) - Static method in class co.cask.cdap.api.spark.sql.DataFrames

Creates a Row object that represents data in the given StructuredRecord.

toSchema(DataType) - Static method in class co.cask.cdap.api.spark.sql.DataFrames

Converts a Spark DataType to a Schema object.

A B C D E F G I J O R S T