public interface JobContext
Modifier and Type | Method and Description |
---|---|
void |
createStreamingContext(long batchDuration)
Creates the SparkStreaming context.
|
File |
getLocalTmpDir() |
<E> E |
getSharedObject(String name)
Get shared object
|
org.apache.spark.sql.hive.HiveContext |
hivectx() |
<E> E |
removeSharedObject(String name)
Remove shared object from cache
|
org.apache.spark.api.java.JavaSparkContext |
sc() |
<E> void |
setSharedObject(String name,
E object)
Set shared object, it will replace the old one if already existed
|
<E> E |
sparkSession() |
org.apache.spark.sql.SQLContext |
sqlctx() |
void |
stopStreamingCtx()
Stops the SparkStreaming context.
|
org.apache.spark.streaming.api.java.JavaStreamingContext |
streamingctx() |
org.apache.spark.api.java.JavaSparkContext sc()
org.apache.spark.sql.SQLContext sqlctx()
org.apache.spark.sql.hive.HiveContext hivectx()
org.apache.spark.streaming.api.java.JavaStreamingContext streamingctx()
<E> E getSharedObject(String name) throws NoSuchElementException
E
- The type of the shared objectname
- Name of the shared object to returnNoSuchElementException
<E> void setSharedObject(String name, E object)
E
- The type of the shared objectname
- Name of the shared object to be setobject
- The object to be set<E> E removeSharedObject(String name)
E
- The type of the shared objectname
- Name of the shared object to be removedvoid createStreamingContext(long batchDuration)
batchDuration
- Time interval at which streaming data will be divided into batches,
in milliseconds.void stopStreamingCtx()
File getLocalTmpDir()
Copyright © 2020 The Apache Software Foundation. All rights reserved.