Avro ↔ JSON Conversion in Java

 

Full Maven Project Template

In this guide, you’ll learn how to:

• Define an Avro schema for a Product (fields: name, code, price)

• Read Avro-serialized data (generic or specific)

• Convert Avro records to JSON via Jackson

• Use a full Maven project setup to tie it all together

---

1. Project Structure

Here’s a suggested directory layout:

product-avro-json/
├── pom.xml
├── src
│   ├── main
│   │   ├── avro
│   │   │   └── product.avsc
│   │   └── java
│   │       └── com/example
│   │           ├── AvroReader.java
│   │           ├── AvroToJsonConverter.java
│   │           └── MainApp.java
│   └── test
│       └── java
│           └── com/example
│               └── AvroJsonTest.java

---

2. pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="
           http://maven.apache.org/POM/4.0.0
           http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.example</groupId>
  <artifactId>product-avro-json</artifactId>
  <version>1.0-SNAPSHOT</version>
  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <maven.compiler.source>11</maven.compiler.source>
    <maven.compiler.target>11</maven.compiler.target>
    <avro.version>1.10.2</avro.version>
    <jackson.version>2.12.3</jackson.version>
  </properties>

  <dependencies>
    <!-- Avro runtime -->
    <dependency>
      <groupId>org.apache.avro</groupId>
      <artifactId>avro</artifactId>
      <version>${avro.version}</version>
    </dependency>
    <!-- Avro compiler (for code generation) -->
    <dependency>
      <groupId>org.apache.avro</groupId>
      <artifactId>avro-compiler</artifactId>
      <version>${avro.version}</version>
    </dependency>
    <!-- Jackson for JSON conversion -->
    <dependency>
      <groupId>com.fasterxml.jackson.core</groupId>
      <artifactId>jackson-databind</artifactId>
      <version>${jackson.version}</version>
    </dependency>
    <!-- JUnit (for testing) -->
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter-api</artifactId>
      <version>5.7.0</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter-engine</artifactId>
      <version>5.7.0</version>
      <scope>test</scope>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <!-- Avro Maven Plugin: generate Java classes from .avsc -->
      <plugin>
        <groupId>org.apache.avro</groupId>
        <artifactId>avro-maven-plugin</artifactId>
        <version>${avro.version}</version>
        <executions>
          <execution>
            <phase>generate-sources</phase>
            <goals>
              <goal>schema</goal>
            </goals>
            <configuration>
              <sourceDirectory>${project.basedir}/src/main/avro</sourceDirectory>
              <outputDirectory>${project.build.directory}/generated-sources/avro</outputDirectory>
            </configuration>
          </execution>
        </executions>
      </plugin>

      <!-- Ensure generated sources are compiled -->
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.8.1</version>
        <configuration>
          <source>11</source>
          <target>11</target>
          <annotationProcessorPaths>
            <!-- include avro plugin if needed -->
          </annotationProcessorPaths>
        </configuration>
        <executions>
          <execution>
            <id>default-compile</id>
            <goals><goal>compile</goal></goals>
          </execution>
          <execution>
            <id>default-testCompile</id>
            <goals><goal>testCompile</goal></goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

This pom.xml does:

• Include avro and avro-compiler

• Use the avro-maven-plugin to generate Java classes from .avsc

• Include Jackson for JSON serialization

• Setup compilation and test dependencies

---

3. Avro Schema: product.avsc

Place this in src/main/avro/product.avsc:

{
  "namespace": "com.example.avro",
  "type": "record",
  "name": "Product",
  "fields": [
    { "name": "name", "type": "string" },
    { "name": "code", "type": "string" },
    { "name": "price", "type": "double" }
  ]
}

After you build the project, the Avro plugin will produce a generated Java class com.example.avro.Product.

---

4. Java Source Files

AvroReader.java

package com.example;

import org.apache.avro.file.DataFileReader;
import org.apache.avro.generic.GenericDatumReader;
import org.apache.avro.generic.GenericRecord;
import org.apache.avro.generic.GenericDatumReader;
import org.apache.avro.io.DatumReader;

import java.io.File;
import java.io.IOException;

public class AvroReader {

    public static Iterable<GenericRecord> readGenericRecords(String avroFilePath) throws IOException {
        File file = new File(avroFilePath);
        DatumReader<GenericRecord> datumReader = new GenericDatumReader<>();
        DataFileReader<GenericRecord> dataFileReader = new DataFileReader<>(file, datumReader);
        return dataFileReader;
    }

    public static <T> Iterable<T> readSpecificRecords(String avroFilePath, Class<T> clazz) throws IOException {
        // Not full implementation; for specific, you’d use SpecificDatumReader
        throw new UnsupportedOperationException("Not implemented");
    }
}

AvroToJsonConverter.java

package com.example;

import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.avro.generic.GenericRecord;

import java.io.IOException;

public class AvroToJsonConverter {
    private static final ObjectMapper objectMapper = new ObjectMapper();

    public static String convertGenericRecordToJson(GenericRecord record) throws IOException {
        return objectMapper.writeValueAsString(record);
    }
}

MainApp.java

package com.example;

import org.apache.avro.generic.GenericRecord;

import java.io.IOException;

public class MainApp {

    public static void main(String[] args) {
        if (args.length != 1) {
            System.err.println("Usage: java -jar product-avro-json.jar <path-to-avro-file>");
            System.exit(1);
        }
        String avroFilePath = args[0];

        try {
            for (GenericRecord rec : AvroReader.readGenericRecords(avroFilePath)) {
                String json = AvroToJsonConverter.convertGenericRecordToJson(rec);
                System.out.println(json);
            }
        } catch (IOException e) {
            System.err.println("Error: " + e.getMessage());
            e.printStackTrace();
        }
    }
}

---

5. (Optional) Test Example: AvroJsonTest.java

You could put a test to check the conversion:

package com.example;

import org.apache.avro.generic.GenericRecord;
import org.junit.jupiter.api.Test;

import java.io.File;
import java.io.IOException;

import static org.junit.jupiter.api.Assertions.*;

public class AvroJsonTest {

    @Test
    public void testConversion() throws IOException {
        String avroFile = "src/test/resources/test-products.avro";
        Iterable<GenericRecord> recs = AvroReader.readGenericRecords(avroFile);
        boolean found = false;
        for (GenericRecord rec : recs) {
            String json = AvroToJsonConverter.convertGenericRecordToJson(rec);
            assertTrue(json.contains("\"name\""));
            assertTrue(json.contains("\"code\""));
            assertTrue(json.contains("\"price\""));
            found = true;
        }
        assertTrue(found, "No record found in test Avro file");
    }
}

You’d have to supply a test Avro file in src/test/resources/test-products.avro.

---

6. How to Build & Run

1. Compile & generate sources:

   mvn clean compile
   

   The Avro plugin will generate Java classes from product.avsc in the target/generated-sources/avro directory.

2. Package into a JAR:

   mvn package
   

   This produces product-avro-json-1.0-SNAPSHOT.jar in target/.

3. Run the application:

   java -jar target/product-avro-json-1.0-SNAPSHOT.jar path/to/your/products.avro
   

   It will print JSON lines corresponding to each record in the Avro file.

---

7. Explanation & Notes

• The Avro Maven plugin reads .avsc files in src/main/avro and generates Java classes under target/generated-sources/avro.

• In this example, we use generic records via GenericRecord to read data; this is flexible and schema-driven.

• We use Jackson to convert Avro GenericRecord to JSON.

• If you prefer specific records (i.e. use the generated Product class directly), you can replace generic reading with SpecificDatumReader<Product> and then do objectMapper.writeValueAsString(productInstance).

• Be careful about null values or optional fields if you extend the schema.