Parse JSON with Manifold

1. Introduction

In this article, we’re going to look at Manifold JSON for JSON Schema-aware parsing of JSON documents. We’ll see what it is, what we can do with it, and how to use it.

Manifold is a suite of compiler plugins that each provide some highly productive features we can use in our applications. In this article, we’re going to explore using it for parsing and building JSON documents based on provided JSON Schema files.

2. Installation

Before we can use Manifold, we need to ensure that it’s available to our compiler. The most important integration is as a plugin to our Java compiler. We do this by configuring our Maven or Gradle project appropriately. In addition, Manifest provides plugins for some IDEs to make our development process easier.

2.1. Compiler Plugin in Maven

Integrating Manifold into our application in Maven requires us to add both a dependency and a compiler plugin. These need to both be the same version, which is 2024.1.20 at the time of writing.

Adding our dependency is the same as any other:

<dependency>
    <groupId>systems.manifold</groupId>
    <artifactId>manifold-json-rt</artifactId>
    <version>2024.1.20</version>
</dependency>

Adding our compiler plugin requires us to configure the maven-compiler-plugin within our module:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-compiler-plugin</artifactId>
    <version>3.8.0</version>
    <configuration>
        <compilerArgs>
            <!-- Configure manifold plugin-->
            <arg>-Xplugin:Manifold</arg>
        </compilerArgs>
        <!-- Add the processor path for the plugin -->
        <annotationProcessorPaths>
            <path>
                <groupId>systems.manifold</groupId>
                <artifactId>manifold-json</artifactId>
                <version>2024.1.20</version>
            </path>
        </annotationProcessorPaths>
    </configuration>
</plugin>

Here we’re adding the -Xplugin:Manifold command-line argument when the compiler is executed, as well as adding the Manifold JSON annotation processor.

This annotation processor is what does all the work of generating code from our JSON schema files.

2.2. Compiler Plugin in Gradle

Integrating Manifold into our application with Gradle needs to achieve the same, but the configuration is slightly simpler. Gradle supports adding annotation processors to the Java compiler in the same way as any other dependency:

dependencies {
    implementation 'systems.manifold:manifold-json-rt:2024.1.20'
    annotationProcessor 'systems.manifold:manifold-json:2024.1.20'
    testAnnotationProcessor 'systems.manifold:manifold-json:2024.1.20'
}

However, we still need to ensure that the -Xplugin:Manifold parameter is passed to the compiler:

tasks.withType(JavaCompile) {
    options.compilerArgs += ['-Xplugin:Manifold']
}

At this point, we’re ready to start using Manifold JSON.

2.3. IDE Plugins

In addition to the plugins in our Maven or Gradle build, Manifold provides plugins for IntelliJ and Android Studio. These can be installed from the standard plugin marketplace:

Once installed, this will allow the compiler plugins that we’ve previously set up in our project to be used when code is compiled from within the IDE, instead of relying only on the Maven or Gradle builds to do the right thing.

3. Defining Classes with JSON Schema

Once we’ve got Manifold JSON set up in our project, we can use it to define classes. We do this by writing JSON Schema files within the src/main/resources (or src/test/resources) directory. The full path to the file then becomes the fully qualified class name.

For example, we can create a com.baeldung.manifold.SimpleUser class by writing our JSON schema in src/main/resources/com/baeldung/manifold/SimpleUser.json:

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "http://baeldung.com/manifold/SimpleUser.json",
  "type": "object",
  "properties": {
    "username": {
      "type": "string",
      "maxLength": 100
    },
    "name": {
      "type": "string",
      "maxLength": 80
    },
    "email": {
      "type": "string",
      "format": "email",
      "maxLength": 260
    }
  },
  "required": [
    "username",
    "name"
  ]
}

This represents a class with three fields – username, name, and email – all of type String.

However, Manifold doesn’t do anything it doesn’t need to do. As such, writing this file won’t generate any code if nothing references it. This reference can be anything that the compiler needs though – even something as simple as a variable definition.

4. Instantiating Manifold Classes

Once we’ve got a class definition, we want to be able to use it. However, we’ll quickly discover that these aren’t regular classes. Manifold generates all of our expected classes as Java interfaces instead. This means that we can’t simply create new instances using new.

However, the generated code provides a static create() method that we can use as a constructor method. This method will need all of the required values in the order they were specified in the required array. For example, the above JSON Schema will produce the following:

public static SimpleUser create(String username, String name);

We then have setters that we can use to populate any of the remaining fields.

So we can then create a new instance of the SimpleUser class with:

SimpleUser user = SimpleUser.create("testuser", "Test User");
user.setEmail("testuser@example.com");

In addition to this, Manifold provides us with a build() method that we can use for the builder pattern. This takes the same set of required parameters, but instead returns a builder object that we can use to populate any other fields:

SimpleUser user = SimpleUser.builder("testuser", "Test User")
  .withEmail("testuser@example.com")
  .build();

If we’re using fields that are both optional and read-only then the builder pattern is the only way that we can provide values – once we’ve got a created instance then we can no longer set the values of these fields.

5. Parsing JSON

Once we’ve got our generated classes, we can use them to interact with JSON sources. Our generated classes provide the means to load data from various types of input and parse it directly into our target class.

The route into this is always triggered using the static load() method on our generated class. This provides us with a manifold.json.rt.api.Loader<> interface that gives us various methods for loading our JSON data.

The simplest of these is the ability to parse a JSON string that has been provided to us in some manner. This is done using the fromJson() method, which takes a string and returns a fully-formed instance of our class:

SimpleUser user = SimpleUser.load().fromJson("""
    {
        "username": "testuser",
        "name": "Test User",
        "email": "testuser@example.com"
    }
    """);

On success, this gives us our desired result. On failure, we get a RuntimeException that wraps a manifold.rt.api.ScriptException indicating exactly what has gone wrong.

We also have various other ways that we can provide the data:

• fromJsonFile() which reads the data from a java.io.File.
• fromJsonReader() which reads the data from a java.io.Reader.
• fromJsonUrl() which reads the data from a URL – in which case Manifold will go and fetch the data this URL points at.

Note however that the fromJsonUrl() method ultimately uses java.net.URL.openStream() to read the data, which may not be as efficient as we’d like.

All of these work in the same way – we call them with the appropriate source of data and it returns us a fully formed object, or throws a RuntimeException if the data can’t be parsed:

InputStream is = getClass().getResourceAsStream("/com/baeldung/manifold/simpleUserData.json");
InputStreamReader reader = new InputStreamReader(is);
SimpleUser user = SimpleUser.load().fromJsonReader(reader);

6. Generating JSON

As well as being able to parse JSON into our objects, we can go the other way and generate JSON from our objects.

In the same way that Manifold produces a load() method to load JSON into our objects, we also have a write() method to write JSON from our object. This returns us an instance of manifold.json.rt.api.Writer which gives us methods for writing our JSON from our object.

The simplest of these is the toJson() method, which returns the JSON as a String, from which we can do whatever we want:

SimpleUser user = SimpleUser.builder("testuser", "Test User")
  .withEmail("testuser@example.com")
  .build();
String json = user.write().toJson();

Alternatively, we can write JSON to anything that implements the Appendable interface. This includes, amongst many other things, the java.io.Writer interface or the StringBuilder and StringBuffer classes:

SimpleUser user = SimpleUser.builder("testuser", "Test User")
  .withEmail("testuser@example.com")
  .build();
user.write().toJson(writer);

This writer can then write to any target – including memory buffers, files, or network connections.

7. Alternative Formats

The primary purpose of manifold-json is to be able to parse and generate JSON content. However, we also have the ability for other formats – CSV, XML, and YAML.

For each of these, we need to add a particular dependency to our project – systems.manifold:manifold-csv-rt for CSV, systems.manifold:manifold-xml-rt for XML or systems.manifold:manifold-yaml-rt for YAML. We can add as many or as few of these as we need.

Once this is done, we can use the appropriate methods on the Loader and Writer interfaces that Manifold provides to us:

SimpleUser user = SimpleUser.load().fromXmlReader(reader);
String yaml = user.write().toYaml();

8. JSON Schema Features

Manifold uses JSON Schema files to describe the structure of our classes. We can use these files to describe the classes to generate and the fields to be present in them. However, we can describe more than just this with JSON Schema, and Manifold JSON supports some of these extra features.

8.1. Read-Only and Write-Only Fields

Fields that are marked as read-only in the schema are generated without setters. This means that we can populate them at construction time, or from parsing input files, but can never change the values afterward:

"username": {
    "type": "string",
    "readOnly": true
},

Conversely, the generator creates fields marked as write-only in the schema without getters. This means that we can populate them however we wish – at construction time, parsing from input files, or using setters – but we can never get the value back out.

"mfaCode": {
    "type": "string",
    "writeOnly": true
},

Note that the system still renders write-only properties in any generated output, so we can access them through that route. However, we cannot read these properties from the Java classes.

8.2. Formatted Types

Some types in a JSON Schema allow us to provide additional format information. For example, we can specify that a field is a string but has a format of date-time. This is a hint to anything using the schema of the type of data that we should see in these fields.

Manifold JSON will do its best to understand these formats and produce appropriate Java types for them. For example, a string that is formatted as date-time will be generated as a java.time.LocalDateTime in our code.

8.3. Additional Properties

JSON Schema lets us define open-ended schemas through the use of additionalProperties and patternProperties.

The additionalProperties flag indicates that the type can have an arbitrary number of extra properties of any type. Essentially it means that the schema allows any other JSON to match. Manifold JSON defaults to having this defined as true, but we can explicitly set it to false in our schema if we wish.

If this is set to true then Manifold will provide two additional methods on our generated class – get(name) and put(name, value). Using these we can work with any arbitrary fields that we wish:

SimpleUser user = SimpleUser.builder("testuser", "Test User")
  .withEmail("testuser@example.com")
  .build();
user.put("note", "This isn't specified in the schema");

Note that no validation is done on these values. This includes checking if the name collides with other defined fields. As such, this can be used to overwrite fields defined in our schema – including ignoring validation rules such as type or readOnly.

JSON Schema also has support for a more advanced concept called “pattern properties”. We define these as full property definitions, but we use a regex to define the property name instead of a fixed string. For example, this definition will allow us to specify fields note0 up to note9 all as type string:

"patternProperties": {
  "note[0-9]": {
    "type": "string"
  }
}

Manifold JSON has partial support for this. Instead of generating explicit code to handle this, it will treat the presence of any patternProperties in a type the same as if additionalProperties were set to true. This includes if we’d explicitly set additionalProperties to false.

8.4. Nested Types

JSON Schema has support for us to define one type nested within another, instead of needing to define everything at the top level and use references. This can be very useful for providing localized structure to our JSON. For example:

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "http://baeldung.com/manifold/User.json",
  "type": "object",
  "properties": {
    "email": {
      "type": "object",
      "properties": {
        "address": {
          "type": "string",
          "maxLength": 260
        },
        "verified": {
          "type": "boolean"
        }
      },
      "required": ["address", "verified"]
    }
  }
}

Here we’ve defined one object where one of the fields is another object. We’d then represent this with JSON looking like:

{
    "email": {
        "address": "testuser@example.com",
        "verified": false
    }
}

Manifold JSON has full support for this, and will automatically generate appropriately named inner classes. We can then use those classes exactly as any other:

User user = User.builder()
  .withEmail(User.email.builder("testuser@example.com", false).build())
  .build();

8.5. Composition

JSON Schema supports composing different types together, using the allOf, anyOf, or oneOf keywords. Each of these takes a collection of other schemas – either by reference or directly specified – and the resulting schema must match all of them, at least one of them, or exactly one of them respectively. Manifold JSON has a level of support for these keywords.

When using allOf, Manifold will generate a class that includes all of the definitions from the composed types. If we define these inline, the system will create a single class with all of the definitions:

"allOf": [
  {
    "type": "object",
    "properties": {
      "username": {
        "type": "string"
      }
    }
  },
  {
    "type": "object",
    "properties": {
      "roles": {
        "type": "array",
        "items": {
          "type": "string"
        }
      }
    }
  }
]

Alternatively, if we compose using references, the resulting interface will extend all of the types that we referenced:

"allOf": [
  {"$ref":  "#/definitions/user"},
  {"$ref":  "#/definitions/adminUser"}
]

In both cases, we can use the generated class as if it fits both sets of definitions:

Composed.user.builder()
    .withUsername("testuser")
    .withRoles(List.of("admin"))
    .build()

If instead, we use anyOf or oneOf then Manifold will generate code that can accept the alternative options in a typesafe manner. This requires us to use references so that Manifold can infer type names:

"anyOf": [
  {"$ref":  "#/definitions/Dog"},
  {"$ref":  "#/definitions/Cat"}
]

When doing this, our types gain typesafe methods for interacting with the different options:

Composed composed = Composed.builder()
  .withAnimalAsCat(Composed.Cat.builder()
    .withColor("ginger")
    .build())
  .build();
assertEquals("ginger", composed.getAnimalAsCat().getColor());

Here we can see that our builder has gained a withAnimalAsCat method – where the “Animal” portion is the field name within the outer object and the “Cat” portion is the inferred type name from our definition. Our actual object has also gained getAnimalAsCat and setAnimalAsCat methods in the same way.

9. Conclusion

In this article, we’ve given a broad introduction to Manifold JSON. This library can do much more, so why not try it out and see?

All of the examples are available over on GitHub.