Serialized Form (Spark Hydrator Plugins 1.5.0 API)

Package co.cask.hydrator.plugin.spark
- Class co.cask.hydrator.plugin.spark.DecisionTreePredictor extends SparkMLPredictor implements Serializable
  - Serialized Fields
    - loadedModel
```
org.apache.spark.mllib.tree.model.DecisionTreeModel loadedModel
```
- Class co.cask.hydrator.plugin.spark.DecisionTreeTrainer extends SparkMLTrainer implements Serializable
  - Serialized Fields
    - config
```
DecisionTreeTrainer.DecisionTreeTrainerConfig config
```
- Class co.cask.hydrator.plugin.spark.DecisionTreeTrainer.DecisionTreeTrainerConfig extends SparkMLTrainer.MLTrainerConfig implements Serializable
  - Serialized Fields
    - cardinalityMapping
```
String cardinalityMapping
```
    - maxDepth
```
Integer maxDepth
```
    - maxBins
```
Integer maxBins
```
- Class co.cask.hydrator.plugin.spark.FileStreamingSource extends ReferenceStreamingSource<co.cask.cdap.api.data.format.StructuredRecord> implements Serializable
  - Serialized Fields
    - conf
```
FileStreamingSource.Conf conf
```
- Class co.cask.hydrator.plugin.spark.FileStreamingSource.Conf extends ReferencePluginConfig implements Serializable
  - Serialized Fields
    - format
```
String format
```
    - schema
```
String schema
```
    - path
```
String path
```
    - ignoreThreshold
```
Integer ignoreThreshold
```
    - extensions
```
String extensions
```
- Class co.cask.hydrator.plugin.spark.HashingTFFeatureGenerator extends co.cask.cdap.etl.api.batch.SparkCompute<co.cask.cdap.api.data.format.StructuredRecord,co.cask.cdap.api.data.format.StructuredRecord> implements Serializable
  - Serialized Fields
    - config
```
HashingTFFeatureGenerator.HashingTFConfig config
```
    - outputSchema
```
co.cask.cdap.api.data.schema.Schema outputSchema
```
    - hashingTF
```
org.apache.spark.mllib.feature.HashingTF hashingTF
```
    - splitter
```
com.google.common.base.Splitter splitter
```
    - pattern
```
Pattern pattern
```
- Class co.cask.hydrator.plugin.spark.HashingTFFeatureGenerator.HashingTFConfig extends co.cask.cdap.api.plugin.PluginConfig implements Serializable
  - Serialized Fields
    - pattern
```
String pattern
```
    - numFeatures
```
Integer numFeatures
```
    - outputColumnMapping
```
String outputColumnMapping
```
- Class co.cask.hydrator.plugin.spark.HTTPPollerSource extends co.cask.cdap.etl.api.streaming.StreamingSource<co.cask.cdap.api.data.format.StructuredRecord> implements Serializable
  - Serialized Fields
    - conf
```
HTTPPollConfig conf
```
- Class co.cask.hydrator.plugin.spark.KafkaConfig extends ReferencePluginConfig implements Serializable
  
  serialVersionUID:
  
  8069169417140954175L
  - Serialized Fields
    - brokers
```
String brokers
```
    - topic
```
String topic
```
    - partitions
```
String partitions
```
    - initialPartitionOffsets
```
String initialPartitionOffsets
```
    - defaultInitialOffset
```
Long defaultInitialOffset
```
    - schema
```
String schema
```
    - format
```
String format
```
    - timeField
```
String timeField
```
    - keyField
```
String keyField
```
    - partitionField
```
String partitionField
```
    - offsetField
```
String offsetField
```
- Class co.cask.hydrator.plugin.spark.KafkaStreamingSource extends ReferenceStreamingSource<co.cask.cdap.api.data.format.StructuredRecord> implements Serializable
  - Serialized Fields
    - conf
```
KafkaConfig conf
```
- Class co.cask.hydrator.plugin.spark.LogisticRegressionClassifier extends SparkMLPredictor implements Serializable
  - Serialized Fields
    - loadedModel
```
org.apache.spark.mllib.classification.LogisticRegressionModel loadedModel
```
- Class co.cask.hydrator.plugin.spark.LogisticRegressionTrainer extends SparkMLTrainer implements Serializable
  - Serialized Fields
    - config
```
LogisticRegressionTrainer.Config config
```
- Class co.cask.hydrator.plugin.spark.LogisticRegressionTrainer.Config extends SparkMLTrainer.MLTrainerConfig implements Serializable
  - Serialized Fields
    - numClasses
```
Integer numClasses
```
- Class co.cask.hydrator.plugin.spark.NaiveBayesClassifier extends co.cask.cdap.etl.api.batch.SparkCompute<co.cask.cdap.api.data.format.StructuredRecord,co.cask.cdap.api.data.format.StructuredRecord> implements Serializable
  - Serialized Fields
    - config
```
NaiveBayesClassifier.Config config
```
    - outputSchema
```
co.cask.cdap.api.data.schema.Schema outputSchema
```
    - loadedModel
```
org.apache.spark.mllib.classification.NaiveBayesModel loadedModel
```
    - tf
```
org.apache.spark.mllib.feature.HashingTF tf
```
- Class co.cask.hydrator.plugin.spark.NaiveBayesClassifier.Config extends co.cask.cdap.api.plugin.PluginConfig implements Serializable
  - Serialized Fields
    - fileSetName
```
String fileSetName
```
    - path
```
String path
```
    - fieldToClassify
```
String fieldToClassify
```
    - predictionField
```
String predictionField
```
    - numFeatures
```
Integer numFeatures
```
- Class co.cask.hydrator.plugin.spark.NaiveBayesTrainer extends co.cask.cdap.etl.api.batch.SparkSink<co.cask.cdap.api.data.format.StructuredRecord> implements Serializable
  - Serialized Fields
    - config
```
NaiveBayesTrainer.Config config
```
- Class co.cask.hydrator.plugin.spark.NaiveBayesTrainer.Config extends co.cask.cdap.api.plugin.PluginConfig implements Serializable
  - Serialized Fields
    - fileSetName
```
String fileSetName
```
    - path
```
String path
```
    - fieldToClassify
```
String fieldToClassify
```
    - predictionField
```
String predictionField
```
    - numFeatures
```
Integer numFeatures
```
- Class co.cask.hydrator.plugin.spark.NGramTransform extends co.cask.cdap.etl.api.batch.SparkCompute<co.cask.cdap.api.data.format.StructuredRecord,co.cask.cdap.api.data.format.StructuredRecord> implements Serializable
  - Serialized Fields
    - config
```
NGramTransform.Config config
```
    - outputSchema
```
co.cask.cdap.api.data.schema.Schema outputSchema
```
- Class co.cask.hydrator.plugin.spark.NGramTransform.Config extends co.cask.cdap.api.plugin.PluginConfig implements Serializable
  - Serialized Fields
    - fieldToBeTransformed
```
String fieldToBeTransformed
```
    - ngramSize
```
Integer ngramSize
```
    - outputField
```
String outputField
```
- Class co.cask.hydrator.plugin.spark.ReferenceStreamingSource extends co.cask.cdap.etl.api.streaming.StreamingSource<T> implements Serializable
  - Serialized Fields
    - conf
```
ReferencePluginConfig conf
```
- Class co.cask.hydrator.plugin.spark.SkipGramFeatureGenerator extends co.cask.cdap.etl.api.batch.SparkCompute<co.cask.cdap.api.data.format.StructuredRecord,co.cask.cdap.api.data.format.StructuredRecord> implements Serializable
  - Serialized Fields
    - config
```
SkipGramFeatureGenerator.FeatureGeneratorConfig config
```
    - outputSchema
```
co.cask.cdap.api.data.schema.Schema outputSchema
```
    - loadedModel
```
org.apache.spark.mllib.feature.Word2VecModel loadedModel
```
    - splitter
```
com.google.common.base.Splitter splitter
```
    - pattern
```
Pattern pattern
```
- Class co.cask.hydrator.plugin.spark.SkipGramFeatureGenerator.FeatureGeneratorConfig extends co.cask.cdap.api.plugin.PluginConfig implements Serializable
  - Serialized Fields
    - fileSetName
```
String fileSetName
```
    - path
```
String path
```
    - outputColumnMapping
```
String outputColumnMapping
```
    - pattern
```
String pattern
```
- Class co.cask.hydrator.plugin.spark.SkipGramTrainer extends co.cask.cdap.etl.api.batch.SparkSink<co.cask.cdap.api.data.format.StructuredRecord> implements Serializable
  - Serialized Fields
    - config
```
SkipGramTrainer.SkipGramTrainerConfig config
```
    - splitter
```
com.google.common.base.Splitter splitter
```
- Class co.cask.hydrator.plugin.spark.SkipGramTrainer.SkipGramTrainerConfig extends co.cask.cdap.api.plugin.PluginConfig implements Serializable
  - Serialized Fields
    - fileSetName
```
String fileSetName
```
    - path
```
String path
```
    - inputCol
```
String inputCol
```
    - pattern
```
String pattern
```
    - vectorSize
```
Integer vectorSize
```
    - minCount
```
Integer minCount
```
    - numPartitions
```
Integer numPartitions
```
    - numIterations
```
Integer numIterations
```
    - windowSize
```
Integer windowSize
```
- Class co.cask.hydrator.plugin.spark.SparkMLPredictor extends co.cask.cdap.etl.api.batch.SparkCompute<co.cask.cdap.api.data.format.StructuredRecord,co.cask.cdap.api.data.format.StructuredRecord> implements Serializable
  - Serialized Fields
    - config
```
SparkMLPredictor.MLPredictorConfig config
```
    - outputSchema
```
co.cask.cdap.api.data.schema.Schema outputSchema
```
- Class co.cask.hydrator.plugin.spark.SparkMLPredictor.MLPredictorConfig extends co.cask.cdap.api.plugin.PluginConfig implements Serializable
  - Serialized Fields
    - fileSetName
```
String fileSetName
```
    - path
```
String path
```
    - featureFieldsToInclude
```
String featureFieldsToInclude
```
    - featureFieldsToExclude
```
String featureFieldsToExclude
```
    - predictionField
```
String predictionField
```
- Class co.cask.hydrator.plugin.spark.SparkMLTrainer extends co.cask.cdap.etl.api.batch.SparkSink<co.cask.cdap.api.data.format.StructuredRecord> implements Serializable
  - Serialized Fields
    - config
```
SparkMLTrainer.MLTrainerConfig config
```
- Class co.cask.hydrator.plugin.spark.SparkMLTrainer.MLTrainerConfig extends co.cask.cdap.api.plugin.PluginConfig implements Serializable
  - Serialized Fields
    - fileSetName
```
String fileSetName
```
    - path
```
String path
```
    - featureFieldsToInclude
```
String featureFieldsToInclude
```
    - featureFieldsToExclude
```
String featureFieldsToExclude
```
    - labelField
```
String labelField
```
- Class co.cask.hydrator.plugin.spark.Tokenizer extends co.cask.cdap.etl.api.batch.SparkCompute<co.cask.cdap.api.data.format.StructuredRecord,co.cask.cdap.api.data.format.StructuredRecord> implements Serializable
  - Serialized Fields
    - config
```
Tokenizer.Config config
```
    - outputSchema
```
co.cask.cdap.api.data.schema.Schema outputSchema
```
- Class co.cask.hydrator.plugin.spark.Tokenizer.Config extends co.cask.cdap.api.plugin.PluginConfig implements Serializable
  - Serialized Fields
    - columnToBeTokenized
```
String columnToBeTokenized
```
    - patternSeparator
```
String patternSeparator
```
    - outputColumn
```
String outputColumn
```
- Class co.cask.hydrator.plugin.spark.TwitterStreamingSource extends ReferenceStreamingSource<co.cask.cdap.api.data.format.StructuredRecord> implements Serializable
  - Serialized Fields
    - config
```
TwitterStreamingSource.TwitterStreamingConfig config
```
- Class co.cask.hydrator.plugin.spark.TwitterStreamingSource.TwitterStreamingConfig extends ReferencePluginConfig implements Serializable
  
  serialVersionUID:
  
  4218063781909515444L
  - Serialized Fields
    - consumerKey
```
String consumerKey
```
    - consumerSecret
```
String consumerSecret
```
    - accessToken
```
String accessToken
```
    - accessTokenSecret
```
String accessTokenSecret
```

Serialized Form

Package co.cask.hydrator.plugin.spark

Class co.cask.hydrator.plugin.spark.DecisionTreePredictor extends SparkMLPredictor implements Serializable

Serialized Fields

loadedModel

Class co.cask.hydrator.plugin.spark.DecisionTreeTrainer extends SparkMLTrainer implements Serializable

Serialized Fields

config

Class co.cask.hydrator.plugin.spark.DecisionTreeTrainer.DecisionTreeTrainerConfig extends SparkMLTrainer.MLTrainerConfig implements Serializable

Serialized Fields

cardinalityMapping

maxDepth

maxBins

Class co.cask.hydrator.plugin.spark.FileStreamingSource extends ReferenceStreamingSource<co.cask.cdap.api.data.format.StructuredRecord> implements Serializable

Serialized Fields

conf

Class co.cask.hydrator.plugin.spark.FileStreamingSource.Conf extends ReferencePluginConfig implements Serializable

Serialized Fields

format

schema

path

ignoreThreshold

extensions

Class co.cask.hydrator.plugin.spark.HashingTFFeatureGenerator extends co.cask.cdap.etl.api.batch.SparkCompute<co.cask.cdap.api.data.format.StructuredRecord,co.cask.cdap.api.data.format.StructuredRecord> implements Serializable

Serialized Fields

config

outputSchema

hashingTF

splitter

pattern

Class co.cask.hydrator.plugin.spark.HashingTFFeatureGenerator.HashingTFConfig extends co.cask.cdap.api.plugin.PluginConfig implements Serializable

Serialized Fields

pattern

numFeatures

outputColumnMapping

Class co.cask.hydrator.plugin.spark.HTTPPollerSource extends co.cask.cdap.etl.api.streaming.StreamingSource<co.cask.cdap.api.data.format.StructuredRecord> implements Serializable

Serialized Fields

conf

Class co.cask.hydrator.plugin.spark.KafkaConfig extends ReferencePluginConfig implements Serializable

Serialized Fields

brokers

topic

partitions

initialPartitionOffsets

defaultInitialOffset

schema

format

timeField

keyField

partitionField

offsetField

Class co.cask.hydrator.plugin.spark.KafkaStreamingSource extends ReferenceStreamingSource<co.cask.cdap.api.data.format.StructuredRecord> implements Serializable

Serialized Fields

conf

Class co.cask.hydrator.plugin.spark.LogisticRegressionClassifier extends SparkMLPredictor implements Serializable

Serialized Fields

loadedModel

Class co.cask.hydrator.plugin.spark.LogisticRegressionTrainer extends SparkMLTrainer implements Serializable

Serialized Fields

config

Class co.cask.hydrator.plugin.spark.LogisticRegressionTrainer.Config extends SparkMLTrainer.MLTrainerConfig implements Serializable

Serialized Fields

numClasses

Class co.cask.hydrator.plugin.spark.NaiveBayesClassifier extends co.cask.cdap.etl.api.batch.SparkCompute<co.cask.cdap.api.data.format.StructuredRecord,co.cask.cdap.api.data.format.StructuredRecord> implements Serializable

Serialized Fields

config

outputSchema

loadedModel

tf

Class co.cask.hydrator.plugin.spark.NaiveBayesClassifier.Config extends co.cask.cdap.api.plugin.PluginConfig implements Serializable

Serialized Fields

fileSetName

path

fieldToClassify

predictionField

numFeatures

Class co.cask.hydrator.plugin.spark.NaiveBayesTrainer extends co.cask.cdap.etl.api.batch.SparkSink<co.cask.cdap.api.data.format.StructuredRecord> implements Serializable

Serialized Fields

config

Class co.cask.hydrator.plugin.spark.NaiveBayesTrainer.Config extends co.cask.cdap.api.plugin.PluginConfig implements Serializable