public interface LivyClient
Modifier and Type | Method and Description |
---|---|
Future<?> |
addFile(URI uri)
Adds a file to the running remote context.
|
Future<?> |
addJar(URI uri)
Adds a jar file to the running remote context.
|
<T> Future<T> |
run(Job<T> job)
Asks the remote context to run a job immediately.
|
void |
stop(boolean shutdownContext)
Stops the remote context.
|
<T> JobHandle<T> |
submit(Job<T> job)
Submits a job for asynchronous execution.
|
Future<?> |
uploadFile(File file)
Upload a file to be passed to the Spark application
|
Future<?> |
uploadJar(File jar)
Upload a jar to be added to the Spark application classpath
|
<T> JobHandle<T> submit(Job<T> job)
T
- The return type of the jobjob
- The job to execute.<T> Future<T> run(Job<T> job)
Normally, the remote context will queue jobs and execute them based on how many worker threads have been configured. This method will run the submitted job in the same thread processing the RPC message, so that queueing does not apply.
It's recommended that this method only be used to run code that finishes quickly. This avoids interfering with the normal operation of the context.
T
- The return type of the jobjob
- The job to execute.void stop(boolean shutdownContext)
shutdownContext
- Whether to shutdown the underlying Spark context. If false, the
context will keep running and it's still possible to send commands
to it, if the backend being used supports it.Future<?> uploadJar(File jar)
jar
- The local file to be uploadedFuture<?> addJar(URI uri)
Note that the URL should be reachable by the Spark driver process. If running the driver in cluster mode, it may reside on a different host, meaning "file:" URLs have to exist on that node (and not on the client machine).
If the provided URI has no scheme, it's considered to be relative to the default file system configured in the Livy server.
uri
- The location of the jar file.Future<?> uploadFile(File file)
file
- The local file to be uploadedFuture<?> addFile(URI uri)
Note that the URL should be reachable by the Spark driver process. If running the driver in cluster mode, it may reside on a different host, meaning "file:" URLs have to exist on that node (and not on the client machine).
If the provided URI has no scheme, it's considered to be relative to the default file system configured in the Livy server.
uri
- The location of the file.Copyright © 2020 The Apache Software Foundation. All rights reserved.