A B C D E F G I J R S T

A

AbstractExtendedSpark - Class in co.cask.cdap.api.spark

This abstract class provides extra features at configure time to Spark program through the ExtendedSparkConfigurer provided by the AbstractExtendedSpark.getConfigurer() method.

AbstractExtendedSpark() - Constructor for class co.cask.cdap.api.spark.AbstractExtendedSpark

B

BindingException - Exception in co.cask.cdap.api.spark.dynamic

Exception to indicate there is an error in binding variable.

BindingException(String, String, Object, String...) - Constructor for exception co.cask.cdap.api.spark.dynamic.BindingException

C

co.cask.cdap.api.spark - package co.cask.cdap.api.spark

Interfaces for Spark interactions.

co.cask.cdap.api.spark.dynamic - package co.cask.cdap.api.spark.dynamic

co.cask.cdap.api.spark.sql - package co.cask.cdap.api.spark.sql

CompilationFailureException - Exception in co.cask.cdap.api.spark.dynamic

Exception to indicate there is a code compilation error.

CompilationFailureException(String) - Constructor for exception co.cask.cdap.api.spark.dynamic.CompilationFailureException

createInterpreter() - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a new instance of SparkInterpreter for Scala code compilation and interpretation.

createSparkCompiler() - Method in interface co.cask.cdap.api.spark.ExtendedSparkConfigurer

Creates a new instance of SparkCompiler for Scala code compilation.

D

DataFrames - Class in co.cask.cdap.api.spark.sql

Utility class for conversions between DataType and Schema.

E

execute(int, TxRunnable) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Transactions with a custom timeout are not supported in Spark.

ExtendedSparkConfigurer - Interface in co.cask.cdap.api.spark

Extends the functionality of SparkConfigurer to provide more spark specific feature at configuration time.

F

fromDataset(String) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD from the given Dataset.

fromDataset(String, String) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD from the given Dataset.

fromDataset(String, Map<String, String>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD from the given Dataset with the given set of dataset arguments.

fromDataset(String, String, Map<String, String>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD from the given Dataset with the given set of dataset arguments.

fromDataset(String, Map<String, String>, Iterable<? extends Split>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD from the given Dataset with the given set of dataset arguments and custom list of Splits.

fromDataset(String, String, Map<String, String>, Iterable<? extends Split>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD from the given Dataset with the given set of dataset arguments and custom list of Splits.

fromRow(Row, Schema) - Static method in class co.cask.cdap.api.spark.sql.DataFrames

Creates a StructuredRecord from the data in the given Row.

fromStream(String) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaRDD that represents all events from the given stream.

fromStream(String, String) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaRDD that represents all events from the given stream.

fromStream(String, long, long) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaRDD that represents events from the given stream in the given time range.

fromStream(String, String, long, long) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaRDD that represents events from the given stream in the given time range.

fromStream(String, Class<V>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD that represents all events from the given stream.

fromStream(String, String, Class<V>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD that represents all events from the given stream.

fromStream(String, long, long, Class<V>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD that represents events from the given stream in the given time range.

fromStream(String, String, long, long, Class<V>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD that represents events from the given stream in the given time range.

fromStream(String, long, long, Class<? extends StreamEventDecoder<K, V>>, Class<K>, Class<V>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD that represents events from the given stream in the given time range.

fromStream(String, String, long, long, Class<? extends StreamEventDecoder<K, V>>, Class<K>, Class<V>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD that represents events from the given stream in the given time range.

fromStream(String, FormatSpecification, Class<T>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD that represents all events from the given stream.

fromStream(String, String, FormatSpecification, Class<T>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD that represents all events from the given stream.

fromStream(String, FormatSpecification, long, long, Class<T>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD that represents data from the given stream for events in the given time range.

fromStream(String, String, FormatSpecification, long, long, Class<T>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Creates a JavaPairRDD that represents data from the given stream for events in the given time range.

G

getConfigurer() - Method in class co.cask.cdap.api.spark.AbstractExtendedSpark

Returns the ExtendedSparkConfigurer for extended features, only available at configuration time.

getLocalizationContext() - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Returns a Serializable TaskLocalizationContext which can be used to retrieve files localized to task containers.

getLogicalStartTime() - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Returns the logical start time of this Spark job.

getMessagingContext() - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Returns a MessagingContext which can be used to interact with the transactional messaging system.

getMetrics() - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Returns a Serializable Metrics which can be used to emit custom metrics from user's Spark program.

getPluginContext() - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Returns a Serializable PluginContext which can be used to request for plugins instances.

getSecureStore() - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Returns a Serializable SecureStore which can be used to request for plugins instances.

getServiceDiscoverer() - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Returns a Serializable ServiceDiscoverer for Service Discovery in Spark Program which can be passed in Spark program's closures.

getSparkExecutionContext() - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Returns the underlying SparkExecutionContext used by this object.

getSpecification() - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

I

InterpretFailureException - Exception in co.cask.cdap.api.spark.dynamic

Exception to indicate there is a code interpretation error.

InterpretFailureException(String) - Constructor for exception co.cask.cdap.api.spark.dynamic.InterpretFailureException

J

JavaSparkExecutionContext - Class in co.cask.cdap.api.spark

Spark program execution context.

JavaSparkExecutionContext() - Constructor for class co.cask.cdap.api.spark.JavaSparkExecutionContext

JavaSparkExecutionContextBase - Class in co.cask.cdap.api.spark

Spark program execution context.

JavaSparkExecutionContextBase() - Constructor for class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

JavaSparkMain - Interface in co.cask.cdap.api.spark

A Java interface for Spark program to implement.

R

run(JavaSparkExecutionContext) - Method in interface co.cask.cdap.api.spark.JavaSparkMain

This method will be called when the Spark program starts.

S

saveAsDataset(JavaPairRDD<K, V>, String) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Saves the given JavaPairRDD to the given Dataset.

saveAsDataset(JavaPairRDD<K, V>, String, String) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Saves the given JavaPairRDD to the given Dataset.

saveAsDataset(JavaPairRDD<K, V>, String, Map<String, String>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Saves the given JavaPairRDD to the given Dataset with the given set of Dataset arguments.

saveAsDataset(JavaPairRDD<K, V>, String, String, Map<String, String>) - Method in class co.cask.cdap.api.spark.JavaSparkExecutionContextBase

Saves the given JavaPairRDD to the given Dataset with the given set of dataset arguments.

T

toDataType(Schema) - Static method in class co.cask.cdap.api.spark.sql.DataFrames

Converts a Schema to Spark DataType.

toDataType(Schema, Function1<Schema, DataType>) - Static method in class co.cask.cdap.api.spark.sql.DataFrames

Converts a Schema to Spark DataType.

toRow(StructuredRecord, StructType) - Static method in class co.cask.cdap.api.spark.sql.DataFrames

Creates a Row object that represents data in the given StructuredRecord.

toSchema(DataType) - Static method in class co.cask.cdap.api.spark.sql.DataFrames

Converts a Spark DataType to a Schema object.

A B C D E F G I J R S T