org.apache.beam.examples.complete.game

Class GameStats

java.lang.Object

org.apache.beam.examples.complete.game.UserScore

org.apache.beam.examples.complete.game.HourlyTeamScore

org.apache.beam.examples.complete.game.LeaderBoard

org.apache.beam.examples.complete.game.GameStats

public class GameStats
extends LeaderBoard

This class is the fourth in a series of four pipelines that tell a story in a 'gaming' domain, following UserScore, HourlyTeamScore, and LeaderBoard. New concepts: session windows and finding session duration; use of both singleton and non-singleton side inputs.

This pipeline builds on the LeaderBoard functionality, and adds some "business intelligence" analysis: abuse detection and usage patterns. The pipeline derives the Mean user score sum for a window, and uses that information to identify likely spammers/robots. (The robots have a higher click rate than the human users). The 'robot' users are then filtered out when calculating the team scores.

Additionally, user sessions are tracked: that is, we find bursts of user activity using session windows. Then, the mean session duration information is recorded in the context of subsequent fixed windowing. (This could be used to tell us what games are giving us greater user retention).

Run org.apache.beam.examples.complete.game.injector.Injector to generate pubsub data for this pipeline. The Injector documentation provides more detail.

To execute this pipeline using the Dataflow service, specify the pipeline configuration like this:

   --project=YOUR_PROJECT_ID
   --tempLocation=gs://YOUR_TEMP_DIRECTORY
   --runner=BlockingDataflowRunner
   --dataset=YOUR-DATASET
   --topic=projects/YOUR-PROJECT/topics/YOUR-TOPIC
 
 

where the BigQuery dataset you specify must already exist. The PubSub topic you specify should be the same topic to which the Injector is publishing.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	GameStats.CalculateSpammyUsers Filter out all but those users with a high clickrate, which we will consider as 'spammy' uesrs.

Nested classes/interfaces inherited from class org.apache.beam.examples.complete.game.UserScore

UserScore.ExtractAndSumScore

Constructor Summary

Constructors

Constructor and Description
GameStats()

Method Summary

Modifier and Type	Method and Description
protected static Map<String,WriteToBigQuery.FieldInfo<Double>>	configureSessionWindowWrite() Create a map of information that describes how to write pipeline output to BigQuery.
protected static Map<String,WriteToBigQuery.FieldInfo<KV<String,Integer>>>	configureWindowedWrite() Create a map of information that describes how to write pipeline output to BigQuery.
static void	main(String[] args)

Methods inherited from class org.apache.beam.examples.complete.game.LeaderBoard

configureGlobalWindowBigQueryWrite, configureWindowedTableWrite

Methods inherited from class org.apache.beam.examples.complete.game.UserScore

configureBigQueryWrite

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GameStats

public GameStats()

Method Detail

configureWindowedWrite

protected static Map<String,WriteToBigQuery.FieldInfo<KV<String,Integer>>> configureWindowedWrite()

Create a map of information that describes how to write pipeline output to BigQuery. This map is used to write information about team score sums.

configureSessionWindowWrite

protected static Map<String,WriteToBigQuery.FieldInfo<Double>> configureSessionWindowWrite()

Create a map of information that describes how to write pipeline output to BigQuery. This map is used to write information about mean user session time.

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception