co.cask.cdap.api.spark.service

Interface SparkHttpServiceHandler

All Superinterfaces:

ProgramLifecycle<SparkHttpServiceContext>

All Known Implementing Classes:

AbstractSparkHttpServiceHandler

public interface SparkHttpServiceHandler
extends ProgramLifecycle<SparkHttpServiceContext>

Interface to be implemented to handle HTTP requests to the Spark driver process. This interface can be implemented and passed along to ExtendedSparkConfigurer.addHandlers(Iterable). If a Spark program contains one or more SparkHttpServiceHandler, then the Spark program would become a long running process until explicitly stopped. Classes that implement this interface can add methods with the @Path annotation to specify the endpoint which that method handles. It can also use the @GET, @POST, @PUT, @DELETE annotations to specify the type of HTTP requests that it handles.

Example:

      public class MySparkHttpHandler extends AbstractSparkHttpServiceHandler {

         @GET
         @Path("/ping")
        public void process(HttpServiceRequest request, HttpServiceResponder responder) {
          Integer result = getContext().getJavaSparkContext()
            .parallelize(Arrays.asList(1, 2, 3, 4, 5, 6, 7, 8, 9, 10))
            .reduce((v1, v2) -> v1 + v2)
          responder.sendString(result.toString());
        }
      }
   

To handle HTTP request with large body, it is better to have the handler method to return a SparkHttpContentConsumer to avoid running out of memory. Similarly, to return a response with a large body, it is preferable to return respond with SparkHttpContentProducer. The transaction behavior of the handler method is the same as in the Spark driver main method, that is, transaction will be automatically created and committed when performing RDD/DataFrame operations that involve Datasets, but the handler method itself won't be called with transaction. Multiple operations can be done in the same transaction explicitly by using the Transactional.execute(TxRunnable) method. See SparkMain for more detail.

See Also:

SparkMain, HttpContentConsumer, HttpContentProducer

Method Summary

Modifier and Type	Method and Description
void	destroy() This method is called when the http service is about to shutdown.
void	initialize(SparkHttpServiceContext context) Initializes the service handler.

Method Detail

initialize

void initialize(SparkHttpServiceContext context)
         throws Exception

Initializes the service handler. This method will be called only once per SparkHttpServiceHandler instance.

Specified by:

initialize in interface ProgramLifecycle<SparkHttpServiceContext>

Parameters:

context - An instance of SparkHttpServiceContext

Throws:

Exception - If there is any error during initialization.

destroy

void destroy()

This method is called when the http service is about to shutdown. Cleanup job can be done in this method.

Specified by:

destroy in interface ProgramLifecycle<SparkHttpServiceContext>