co.cask.hydrator.plugin.spark

Class NGramTransform

java.lang.Object

co.cask.cdap.etl.api.batch.SparkCompute<co.cask.cdap.api.data.format.StructuredRecord,co.cask.cdap.api.data.format.StructuredRecord>

co.cask.hydrator.plugin.spark.NGramTransform

All Implemented Interfaces:

co.cask.cdap.etl.api.PipelineConfigurable, Serializable

public class NGramTransform
extends co.cask.cdap.etl.api.batch.SparkCompute<co.cask.cdap.api.data.format.StructuredRecord,co.cask.cdap.api.data.format.StructuredRecord>

NGramTransform - SparkCompute to transform input features into n-grams.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	NGramTransform.Config Configuration for the NGramTransform Plugin.

Field Summary

Fields

Modifier and Type	Field and Description
co.cask.cdap.api.data.schema.Schema	outputSchema
static String	PLUGIN_NAME

Fields inherited from class co.cask.cdap.etl.api.batch.SparkCompute

PLUGIN_TYPE

Constructor Summary

Constructors

Constructor and Description
NGramTransform(NGramTransform.Config config)

Method Summary

Methods

Modifier and Type	Method and Description
void	configurePipeline(co.cask.cdap.etl.api.PipelineConfigurer pipelineConfigurer)
org.apache.spark.api.java.JavaRDD<co.cask.cdap.api.data.format.StructuredRecord>	transform(co.cask.cdap.etl.api.batch.SparkExecutionPluginContext context, org.apache.spark.api.java.JavaRDD<co.cask.cdap.api.data.format.StructuredRecord> input)

Methods inherited from class co.cask.cdap.etl.api.batch.SparkCompute

initialize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PLUGIN_NAME

public static final String PLUGIN_NAME

See Also:

Constant Field Values

outputSchema

public co.cask.cdap.api.data.schema.Schema outputSchema

Constructor Detail

NGramTransform

public NGramTransform(NGramTransform.Config config)

Method Detail

configurePipeline

public void configurePipeline(co.cask.cdap.etl.api.PipelineConfigurer pipelineConfigurer)
                       throws IllegalArgumentException

Specified by:

configurePipeline in interface co.cask.cdap.etl.api.PipelineConfigurable

Overrides:

configurePipeline in class co.cask.cdap.etl.api.batch.SparkCompute<co.cask.cdap.api.data.format.StructuredRecord,co.cask.cdap.api.data.format.StructuredRecord>

Throws:

IllegalArgumentException

transform

public org.apache.spark.api.java.JavaRDD<co.cask.cdap.api.data.format.StructuredRecord> transform(co.cask.cdap.etl.api.batch.SparkExecutionPluginContext context,
                                                                                         org.apache.spark.api.java.JavaRDD<co.cask.cdap.api.data.format.StructuredRecord> input)
                                                                                           throws Exception

Specified by:

transform in class co.cask.cdap.etl.api.batch.SparkCompute<co.cask.cdap.api.data.format.StructuredRecord,co.cask.cdap.api.data.format.StructuredRecord>

Throws:

Exception