co.cask.cdap.dq

Class DataQualityApp

java.lang.Object

co.cask.cdap.internal.api.AbstractProgramDatasetConfigurable

co.cask.cdap.internal.api.AbstractPluginConfigurable<ApplicationConfigurer>

co.cask.cdap.api.app.AbstractApplication<DataQualityApp.DataQualityConfig>

co.cask.cdap.dq.DataQualityApp

All Implemented Interfaces:

Application<DataQualityApp.DataQualityConfig>

public class DataQualityApp
extends AbstractApplication<DataQualityApp.DataQualityConfig>

Data Quality Application

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	DataQualityApp.AggregationMapper Take in data and output a field type as a key and a DataQualityWritable containing the field value as an associated value.
static class	DataQualityApp.AggregationReducer Generate and write an aggregation with the data collected by the mapper
static class	DataQualityApp.DataQualityConfig Configuration Class for the application Sets following fields: aggregationName, fields (a comma separated list of the fields we want to aggregate over), workflowScheduleMinutes, source, datasetName, inputFormat, and schema.
static class	DataQualityApp.FieldAggregator Map Reduce job that ingests a stream of data and builds an aggregation of the ingested data that maps timestamp, sourceId, field type and field value to frequency.

Constructor Summary

Constructors

Constructor and Description
DataQualityApp()

Method Summary

Methods

Modifier and Type	Method and Description
void	configure()

Methods inherited from class co.cask.cdap.api.app.AbstractApplication

addFlow, addMapReduce, addService, addService, addSpark, addWorker, addWorkflow, buildSchedule, configure, getConfig, getConfigurer, getContext, getTriggerFactory, schedule, scheduleWorkflow, scheduleWorkflow, setDescription, setName

Methods inherited from class co.cask.cdap.internal.api.AbstractPluginConfigurable

usePlugin, usePlugin, usePluginClass, usePluginClass

Methods inherited from class co.cask.cdap.internal.api.AbstractProgramDatasetConfigurable

addDatasetModule, addDatasetType, addStream, addStream, createDataset, createDataset, createDataset, createDataset

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DataQualityApp

public DataQualityApp()

Method Detail

configure

public void configure()

Specified by:

configure in class AbstractApplication<DataQualityApp.DataQualityConfig>