org.apache.beam.runners.core

Class UnboundedReadFromBoundedSource<T>

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<PInput,PCollection<T>>

org.apache.beam.runners.core.UnboundedReadFromBoundedSource<T>

All Implemented Interfaces:

Serializable, HasDisplayData

public class UnboundedReadFromBoundedSource<T>
extends PTransform<PInput,PCollection<T>>

PTransform that converts a BoundedSource as an UnboundedSource.

BoundedSource is read directly without calling BoundedSource.splitIntoBundles(long, org.apache.beam.sdk.options.PipelineOptions), and element timestamps are propagated. While any elements remain, the watermark is the beginning of time BoundedWindow.TIMESTAMP_MIN_VALUE, and after all elements have been produced the watermark goes to the end of time BoundedWindow.TIMESTAMP_MAX_VALUE.

Checkpoints are created by calling BoundedSource.BoundedReader.splitAtFraction(double) on inner BoundedSource. Sources that cannot be split are read entirely into memory, so this transform does not work well with large, unsplittable sources.

This transform is intended to be used by a runner during pipeline translation to convert a Read.Bounded into a Read.Unbounded.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	UnboundedReadFromBoundedSource.BoundedToUnboundedSourceAdapter<T> A BoundedSource to UnboundedSource adapter.

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

name

Constructor Summary

Constructors

Constructor and Description
UnboundedReadFromBoundedSource(BoundedSource<T> source) Constructs a PTransform that performs an unbounded read from a BoundedSource.

Method Summary

Modifier and Type	Method and Description
PCollection<T>	apply(PInput input)
protected Coder<T>	getDefaultOutputCoder()
String	getKindString()
void	populateDisplayData(DisplayData.Builder builder)

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

getDefaultOutputCoder, getDefaultOutputCoder, getName, toString, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

UnboundedReadFromBoundedSource

public UnboundedReadFromBoundedSource(BoundedSource<T> source)

Constructs a PTransform that performs an unbounded read from a BoundedSource.

Method Detail

apply

public PCollection<T> apply(PInput input)

Specified by:

apply in class PTransform<PInput,PCollection<T>>

getDefaultOutputCoder

protected Coder<T> getDefaultOutputCoder()

Overrides:

getDefaultOutputCoder in class PTransform<PInput,PCollection<T>>

getKindString

public String getKindString()

Overrides:

getKindString in class PTransform<PInput,PCollection<T>>

populateDisplayData

public void populateDisplayData(DisplayData.Builder builder)

Specified by:

populateDisplayData in interface HasDisplayData

Overrides:

populateDisplayData in class PTransform<PInput,PCollection<T>>