Introduction to Big Data and Hadoop/

...

/

Parquet: Reading & Writing

Parquet: Reading & Writing

This lesson explains the theory behind reading and writing Parquet records and later demonstrates a Java example to read and write Parquet files.

We'll cover the following...

Reading & Writing

Looking back at our schema, we can come up with maximum definition and repetition levels for the various fields in our record.

message Car {
 required string make;
 required int year;
 
 repeated group part {
     required string name;
     optional int life;
     repeated string oem;   
 }
}

The table below explains the rationale behind the maximum values for definition and repetition levels:

Now, use the definition and repetition levels to write and read back records. Let’s consider the records below:

{
  make        : "Rolls Royce",
  year        : "2025",
  part        :{
                   name: "Tyre",
                   life: "2",
                   oem: "Bridgestone", 
                   oem: "Michelin"
                }

  part        : {
                   name:
...
Access this course and 1400+ top-rated courses and projects.