Class

org.apache.livy.scalaapi

LivyScalaClient

Related Doc: package scalaapi

Permalink

class LivyScalaClient extends AnyRef

A client for submitting Spark-based jobs to a Livy backend.

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. LivyScalaClient
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new LivyScalaClient(livyJavaClient: LivyClient)

    Permalink

    Creates a Scala client.

    Creates a Scala client.

    livyJavaClient

    the Java client of Livy.

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. def addFile(uri: URI): Future[_]

    Permalink

    Adds a file to the running remote context.

    Adds a file to the running remote context.

    Note that the URL should be reachable by the Spark driver process. If running the driver in cluster mode, it may reside on a different host, meaning "file:" URLs have to exist on that node (and not on the client machine).

    If the provided URI has no scheme, it's considered to be relative to the default file system configured in the Livy server.

    uri

    The location of the file.

    returns

    A future that can be used to monitor the operation.

  5. def addJar(uri: URI): Future[_]

    Permalink

    Adds a jar file to the running remote context.

    Adds a jar file to the running remote context.

    Note that the URL should be reachable by the Spark driver process. If running the driver in cluster mode, it may reside on a different host, meaning "file:" URLs have to exist on that node (and not on the client machine).

    If the provided URI has no scheme, it's considered to be relative to the default file system configured in the Livy server.

    uri

    The location of the jar file.

    returns

    A future that can be used to monitor the operation.

  6. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  7. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  8. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  9. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  10. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  11. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  12. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  13. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  14. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  15. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  16. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  17. def run[T](fn: (ScalaJobContext) ⇒ T): Future[T]

    Permalink

    Asks the remote context to run a job immediately.

    Asks the remote context to run a job immediately.

    Normally, the remote context will queue jobs and execute them based on how many worker threads have been configured. This method will run the submitted job in the same thread processing the RPC message, so that queueing does not apply.

    It's recommended that this method only be used to run code that finishes quickly. This avoids interfering with the normal operation of the context.

    fn

    The job to be executed. It is a function that takes in a ScalaJobContext and returns the result of the execution of the job with that context.

    returns

    A handle that can be used to monitor the job.

  18. def stop(shutdownContext: Boolean): Unit

    Permalink

    Stops the remote context.

    Stops the remote context.

    Any pending jobs will be cancelled, and the remote context will be torn down.

    shutdownContext

    Whether to shutdown the underlying Spark context. If false, the context will keep running and it's still possible to send commands to it, if the backend being used supports it.

  19. def submit[T](fn: (ScalaJobContext) ⇒ T): ScalaJobHandle[T]

    Permalink

    Submits a job for asynchronous execution.

    Submits a job for asynchronous execution.

    fn

    The job to be executed. It is a function that takes in a ScalaJobContext and returns the result of the execution of the job with that context.

    returns

    A handle that can be used to monitor the job.

  20. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  21. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  22. def uploadFile(file: File): Future[_]

    Permalink

    Upload a file to be passed to the Spark application.

    Upload a file to be passed to the Spark application.

    file

    The local file to be uploaded.

    returns

    A future that can be used to monitor this operation.

  23. def uploadJar(jar: File): Future[_]

    Permalink

    Upload a jar to be added to the Spark application classpath.

    Upload a jar to be added to the Spark application classpath.

    jar

    The local file to be uploaded.

    returns

    A future that can be used to monitor this operation.

  24. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  25. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  26. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from AnyRef

Inherited from Any

Ungrouped