Startseite Erkunden Hilfe
Registrieren Anmelden
ylproj
/
livy
2
0
Fork 0
Dateien Issues 0 Pull-Requests 0 Wiki

Keine Beschreibung

Struktur: 59dee63274
Branches Tags
livy
ZIP TAR.GZ
Luciano Resende 59dee63274 [MINOR] Add License header to configuration templates vor 7 Jahren
api 2898d82dd7 [MINOR] Add License header to extra configuration files vor 7 Jahren
assembly 412ccc8fcf LIVY-375. Change Livy code package name to org.apache.livy vor 7 Jahren
bin 9d82f1fdf2 LIVY-364. livy-server writes one log file when set LIVY_MAX_LOG_FILE… vor 7 Jahren
client-common 624bb07b29 Minor. Fix deprecated conf warning log issue vor 7 Jahren
client-http 2898d82dd7 [MINOR] Add License header to extra configuration files vor 7 Jahren
conf 59dee63274 [MINOR] Add License header to configuration templates vor 7 Jahren
core 412ccc8fcf LIVY-375. Change Livy code package name to org.apache.livy vor 7 Jahren
coverage 412ccc8fcf LIVY-375. Change Livy code package name to org.apache.livy vor 7 Jahren
dev feeb8678f4 [MINOR] Change Livy commit message style from LIVY-xxx. to [LIVY-xxx] vor 7 Jahren
docs 60ee047e98 [LIVY-376][DOCS] Update README with new naming and links and move Docs into separate files vor 7 Jahren
examples d4bd76f096 [MINOR] Clean some old package name left in code and configuration file vor 7 Jahren
integration-test d4bd76f096 [MINOR] Clean some old package name left in code and configuration file vor 7 Jahren
python-api 412ccc8fcf LIVY-375. Change Livy code package name to org.apache.livy vor 7 Jahren
repl fc4a9e60aa [LIVY-380] Livy should overwrite spark.sql.catalogImplementation if Hive classes are not present on classpath vor 7 Jahren
rsc 2898d82dd7 [MINOR] Add License header to extra configuration files vor 7 Jahren
scala 412ccc8fcf LIVY-375. Change Livy code package name to org.apache.livy vor 7 Jahren
scala-api d4bd76f096 [MINOR] Clean some old package name left in code and configuration file vor 7 Jahren
server d4bd76f096 [MINOR] Clean some old package name left in code and configuration file vor 7 Jahren
test-lib 412ccc8fcf LIVY-375. Change Livy code package name to org.apache.livy vor 7 Jahren
.gitignore 76d969e585 LIVY-90. Update conf dir to use templates (#301) vor 8 Jahren
.rat-excludes 59dee63274 [MINOR] Add License header to configuration templates vor 7 Jahren
.travis.yml 2898d82dd7 [MINOR] Add License header to extra configuration files vor 7 Jahren
DISCLAIMER cbed3b6a03 [MINOR] Add Incubating DISCLAIMER file vor 7 Jahren
LICENSE 8d14469814 [MINOR] Add LICENSE and NOTICE files vor 7 Jahren
NOTICE 4219e1f260 [MINOR] Remove extension from NOTICE file vor 7 Jahren
README.md 60ee047e98 [LIVY-376][DOCS] Update README with new naming and links and move Docs into separate files vor 7 Jahren
checkstyle-suppressions.xml cb5b8aac50 LIVY-374. Update and unify all the license header to Apache vor 7 Jahren
checkstyle.xml 412ccc8fcf LIVY-375. Change Livy code package name to org.apache.livy vor 7 Jahren
pom.xml ff4002ca37 [LIVY-389][MINOR] Pom updates vor 7 Jahren
scalastyle.xml 412ccc8fcf LIVY-375. Change Livy code package name to org.apache.livy vor 7 Jahren

README.md

Apache Livy

Build Status

Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere. It supports executing snippets of code or programs in a Spark context that runs locally or in Apache Hadoop YARN.

  • Interactive Scala, Python and R shells
  • Batch submissions in Scala, Java, Python
  • Multiple users can share the same server (impersonation support)
  • Can be used for submitting jobs from anywhere with REST
  • Does not require any code change to your programs

Pull requests are welcomed! But before you begin, please check out the Contributing section on the Community page of our website.

Online Documentation

Guides and documentation on getting started using Livy, example code snippets, and Livy API documentation can be found at livy.incubator.apache.org.

Before Building Livy

To build Livy, you will need:

Debian/Ubuntu:

  • mvn (from maven package or maven3 tarball)
  • openjdk-7-jdk (or Oracle Java7 jdk)
  • Python 2.6+
  • R 3.x

Redhat/CentOS:

  • mvn (from maven package or maven3 tarball)
  • java-1.7.0-openjdk (or Oracle Java7 jdk)
  • Python 2.6+
  • R 3.x

MacOS:

  • Xcode command line tools
  • Oracle's JDK 1.7+
  • Maven (Homebrew)
  • Python 2.6+
  • R 3.x

Required python packages for building Livy:

  • cloudpickle
  • requests
  • requests-kerberos
  • flake8
  • flaky
  • pytest

To run Livy, you will also need a Spark installation. You can get Spark releases at https://spark.apache.org/downloads.html.

Livy requires at least Spark 1.6 and supports both Scala 2.10 and 2.11 builds of Spark, Livy will automatically pick repl dependencies through detecting the Scala version of Spark.

Livy also supports Spark 2.0+ for both interactive and batch submission, you could seamlessly switch to different versions of Spark through SPARK_HOME configuration, without needing to rebuild Livy.

Building Livy

Livy is built using Apache Maven. To check out and build Livy, run:

git clone https://github.com/apache/incubator-livy.git
cd livy
mvn package

By default Livy is built against Apache Spark 1.6.2, but the version of Spark used when running Livy does not need to match the version used to build Livy. Livy internally uses reflection to mitigate the gaps between different Spark versions, also Livy package itself does not contain a Spark distribution, so it will work with any supported version of Spark (Spark 1.6+) without needing to rebuild against specific version of Spark.