Run java -version and you should see output like this if the installation was successful: openjdk version '1.8.0_322' You may need to run a slightly different command as Java versions are updated frequently. You can run a command like sdk install java 8.0.322-zulu to install Java 8, a Java version that works well with different version of Spark. Spark works well with the zulu Java vendor. See this blog post for a detailed description on how to work with SDKMAN.Īfter installing SDKMAN, you can run sdk list java to list all the Java versions that are available for installation. SDKMAN, short for the Software Development Kit Manager, makes it easy to install, and switch between, different Java versions. Installing Java can be difficult because there are different vendors and versions. Feel free to skip this section if you’ve already installed Java. You need to install Java to run Spark code. This post will show you how to pin PySpark and Delta Lake versions when creating your environment, to ensure they are compatible.Ĭreating a local PySpark / Delta Lake / Jupyter setup can be a bit tricky, but you’ll find it easy by following the steps in this guide. You also need to pay close attention when setting your PySpark and Delta Lake versions, as they must be compatible.

This guide will teach you how to install Java with a package manager that lets you easily switch between Java versions.

In order to run PySpark, you need to install Java. This setup will let you easily run Delta Lake computations on your local machine in a Jupyter notebook for experimentation or to unit test your business logic. This blog post explains how to install PySpark, Delta Lake, and Jupyter Notebooks on a Mac.

Install pyspark on mac m1