Running the First Spark Program
Explore how to set up and execute your first Apache Spark program using the Spark Java API. This lesson guides you through creating a SparkSession, reading CSV data into DataFrames, and displaying records, while also explaining how to run the application locally using an IDE or Maven command line.
We'll cover the following...
The trivial example
Let’s start our Spark journey with a very trivial example.
Source code
Let’s walk through the code available in the widget below.
mvn install exec:exec
Note: Spark is a pure backend processing tool and doesn’t expose a UI, except for the SparkUI monitoring tool.
For the moment, the Maven project structure with all the necessary dependencies shouldn’t concern us. This is explained in the “Spark Maven-based projects” lesson.
The example application
Our first application is a basic ingest-output program. In layperson terms, this just ...