Quantcast
Channel: Baeldung
Viewing all 3777 articles
Browse latest View live

How to Access an Iteration Counter in a For Each Loop

$
0
0

1. Overview

While iterating over data in Java, we may wish to access both the current item and its position in the data source.

This is very easy to achieve in a classic for loop, where the position is usually the focus of the loop's calculations, but it requires a little more work when we use constructs like for each loop or stream.

In this short tutorial, we'll look at a few ways that for each operation can include a counter.

2. Implementing a Counter

Let's start with a simple example. We'll take an ordered list of movies and output them with their ranking.

List<String> IMDB_TOP_MOVIES = Arrays.asList("The Shawshank Redemption",
  "The Godfather", "The Godfather II", "The Dark Knight");

2.1. for Loop

A for loop uses a counter to reference the current item, so it's an easy way to operate over both the data and its index in the list:

List rankings = new ArrayList<>();
for (int i = 0; i < movies.size(); i++) {
    String ranking = (i + 1) + ": " + movies.get(i);
    rankings.add(ranking);
}

As this List is probably an ArrayList, the get operation is efficient, and the above code is a simple solution to our problem.

assertThat(getRankingsWithForLoop(IMDB_TOP_MOVIES))
  .containsExactly("1: The Shawshank Redemption",
      "2: The Godfather", "3: The Godfather II", "4: The Dark Knight");

However, not all data sources in Java can be iterated over this way. Sometimes get is a time-intensive operation, or we can only process the next element of a data source using Stream or Iterable.

2.2. for Each loop

We'll continue using our list of movies, but let's pretend that we can only iterate over it using Java's for each construct:

for (String movie : IMDB_TOP_MOVIES) {
   // use movie value
}

Here we need to use a separate variable to track the current index. We can construct that outside of the loop, and increment it inside:

int i = 0;
for (String movie : movies) {
    String ranking = (i + 1) + ": " + movie;
    rankings.add(ranking);
    i++;
}

We should note that we have to increment the counter after it has been used within the loop.

3. A Functional for Each

Writing the counter extension every time we need it might result in code duplication and may risk accidental bugs concerning when to update the counter variable. We can, therefore, generalize the above using Java's functional interfaces.

First, we should think of the behavior inside the loop as a consumer of both the item in the collection and also the index. This can be modeled using BiConsumer, which defines an accept function that takes two parameters

@FunctionalInterface
public interface BiConsumer<T, U> {
   void accept(T t, U u);
}

As the inside of our loop is something that uses two values, we could write a general looping operation. It could take the Iterable of the source data, over which the for each loop will run, and the BiConsumer for the operation to perform on each item and its index. We can make this generic with the type parameter T:

static <T> void forEachWithCounter(Iterable<T> source, BiConsumer<Integer, T> consumer) {
    int i = 0;
    for (T item : source) {
        consumer.accept(i, item);
        i++;
    }
}

We can use this with our movie rankings example by providing the implementation for the BiConsumer as a lambda:

List rankings = new ArrayList<>();
forEachWithCounter(movies, (i, movie) -> {
    String ranking = (i + 1) + ": " + movies.get(i);
    rankings.add(ranking);
});

4. Adding a Counter to forEach with Stream

The Java Stream API allows us to express how our data passes through filters and transformations. It also provides a forEach function. Let's try to convert that into an operation that includes the counter.

The Stream forEach function takes a Consumer to process the next item. We could, however, create that Consumer to keep track of the counter and pass the item onto a BiConsumer:

public static <T> Consumer<T> withCounter(BiConsumer<Integer, T> consumer) {
    AtomicInteger counter = new AtomicInteger(0);
    return item -> consumer.accept(counter.getAndIncrement(), item);
}

This function returns a new lambda. That lambda uses the AtomicInteger object to keep track of the counter during iteration. The getAndIncrement function is called every time there's a new item.

The lambda created by this function delegates to the BiConsumer passed in so that the algorithm can process both the item and its index.

Let's see this in use by our movie ranking example against a Stream called movies:

List rankings = new ArrayList<>();
movies.forEach(withCounter((i, movie) -> {
    String ranking = (i + 1) + ": " + movie;
    rankings.add(ranking);
}));

Inside the forEach is a call to the withCounter function to create an object which both tracks the count and acts as the Consumer that the forEach operation passes its values to.

5. Conclusion

In this short article, we've looked at three ways to attach a counter to Java for each operation.

We saw how to track the index of the current item on each implementation of them for a loop. We then looked at how to generalize this pattern and how to add it to streaming operations.

As always the example code for this article is available over on GitHub.

The post How to Access an Iteration Counter in a For Each Loop first appeared on Baeldung.

        

Java Weekly, Issue 361

$
0
0

1. Spring and Java

>> Announcing GraalVM 20.3 [medium.com]

Improved container awareness, G1 GC for native images, and many more performance improvements, all in a new GraalVM version!

>> Vanilla Java: Using Java SE as a Framework [renato.athaydes.com]

A minimal server framework on top of Java SE: hot-reloading, annotation processors and code generation, native images, and JUnit integration.

>> What We Know About Java 16 and 17 So Far [infoq.com]

Speculating over the Loom and Valhalla release: delaying the Java 17 LTS release or delivering small chunks of big features?

Also worth reading:

Webinars and presentations:

Time to upgrade:

2. Technical

>> Integrating PIT Mutation Testing and GitHub Pages with GitHub Actions [4comprehension.com]

Publishing PIT mutation test reports to GitHub pages: a practical guide to integrating GitHub actions with PIT.

Also worth reading:

3. Musings

>> Humility [benjiweber.co.uk]

In praise of Humility: enabling compassion, keeping things simple, valuing feedback, and courage!

Also worth reading:

4. Comics

And my favorite Dilberts of the week:

>> Dilbert Not On Mute [dilbert.com]

>> Working At Home Benefits [dilbert.com]

>> Online Class Muted [dilbert.com]

5. Pick of the Week

Not exactly a launch but a couple of weeks ago, I introduced our longer-term COVID pricing for 2020:

>> All Courses on Baeldung

Here's the plan – I'm going to keep the courses at 25% off their standard price in November and then 20% off in December.

Hope that helps.

The post Java Weekly, Issue 361 first appeared on Baeldung.

        

Guide to the System Stubs Library

$
0
0

1. Overview

It can be hard to test our software when it depends on system resources like environment variables, system properties, or uses process-level operations like System.exit.

Java doesn't provide a direct method for setting environment variables, and we run the risk of the values set in one test affecting the execution of another. Similarly, we may find ourselves avoiding writing JUnit tests for code that might perform a System.exit as there's a chance it would abort the tests.

The System Rules and System Lambda Libraries were early solutions to these problems. In this tutorial, we'll look at a new fork of System Lambda called System Stubs, which provides a JUnit 5 alternative.

2. Why System Stubs?

2.1. System Lambda is Not a JUnit Plugin

The original System Rules library was only usable with JUnit 4. It could still be used with JUnit Vintage under JUnit 5, but that required the continued creation of JUnit 4 tests. The creators of the library produced a test framework agnostic version called System Lambda, which was intended for use inside each test method:

@Test
void aSingleSystemLambda() throws Exception {
    restoreSystemProperties(() -> {
        System.setProperty("log_dir", "test/resources");
        assertEquals("test/resources", System.getProperty("log_dir"));
    });
    // more test code here
}

The test code is expressed as a lambda, passed to a method that sets up the necessary stubbing. The cleanup happens just before control is returned to the rest of the test method.

Though this works well in some cases, the approach has a few disadvantages.

2.2. Avoiding Extra Code

The benefit of the System Lambda approach is that there are some common recipes inside its factory class for performing specific types of tests. However, this leads to some code bloat when we want to use it across many test cases.

Firstly, even if the test code itself doesn't throw a checked exception, the wrapper method does, so all methods gain a throws Exception. Secondly, setting up the same rule across multiple tests requires code duplication. Each test needs to perform the same configuration independently.

However, the most cumbersome aspect of this approach comes when we try to set up more than one tool at a time. Let's say we want to set some environment variables and system properties. We end up needing two levels of nesting before our test code starts:

@Test
void multipleSystemLambdas() throws Exception {
    restoreSystemProperties(() -> {
        withEnvironmentVariable("URL", "https://www.baeldung.com")
            .execute(() -> {
                System.setProperty("log_dir", "test/resources");
                assertEquals("test/resources", System.getProperty("log_dir"));
                assertEquals("https://www.baeldung.com", System.getenv("URL"));
            });
    });
}

This is where a JUnit plugin or extension can help us cut down the amount of code we need in our tests.

2.3. Using Less Boilerplate

We should expect to be able to write our tests with a minimum of boilerplate:

@SystemStub
private EnvironmentVariables environmentVariables = ...;
@SystemStub
private SystemProperties restoreSystemProperties;
@Test
void multipleSystemStubs() {
    System.setProperty("log_dir", "test/resources");
    assertEquals("test/resources", System.getProperty("log_dir"));
    assertEquals("https://www.baeldung.com", System.getenv("ADDRESS"));
}

This approach is provided by the SystemStubs JUnit 5 extension and allows our tests to be composed with less code.

2.4. Test Lifecycle Hooks

When the only available tool is the execute-around pattern, it's impossible to hook in the stubbing behavior to all parts of the test lifecycle. This is particularly challenging when trying to combine it with other JUnit extensions, like @SpringBootTest.

If we wanted to set up some environment variables around a Spring Boot test, then there is no way we could reasonably embed that whole test eco-system inside a single test method. We would need a way to activate the test set-up around a test suite.

This was never going to be possible with the methodology employed by System Lambda and was one of the main reasons to create System Stubs.

2.5. Encourage Dynamic Properties

Other frameworks for setting system properties, such as JUnit Pioneer, emphasize configurations known at compile time. In modern tests, where we may be using Testcontainers or Wiremock, we need to set up our system properties based on random runtime settings after those tools startup. This works best with a test library that can be used across the whole test lifecycle.

2.6. More Configurability

It's beneficial having ready-made test recipes, like catchSystemExit, which wrap around test code to do a single job. However, this relies on the test library developers to provide each variation of configuration option we might need.

Configuration by composition is more flexible and is a large part of the new System Stubs implementation.

However, System Stubs supports the original test constructs from System Lambda for backward compatibility. Additionally, it provides a new JUnit 5 extension, a set of JUnit 4 rules, and many more configuration options. Though based on the original code, it has been heavily refactored and modularised to provide a richer set of features.

Let's learn more about it.

3. Getting Started

3.1. Dependencies

The JUnit 5 extension requires a reasonably up to date version of JUnit 5:

<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter</artifactId>
    <version>5.6.2</version>
    <scope>test</scope>
</dependency>

Let's add all the System Stubs library dependencies to our pom.xml:

<!-- for testing with only lambda pattern -->
<dependency>
    <groupId>uk.org.webcompere</groupId>
    <artifactId>system-stubs-core</artifactId>
    <version>1.1.0</version>
    <scope>test</scope>
</dependency>
<!-- for JUnit 4 -->
<dependency>
    <groupId>uk.org.webcompere</groupId>
    <artifactId>system-stubs-junit4</artifactId>
    <version>1.1.0</version>
    <scope>test</scope>
</dependency>
<!-- for JUnit 5 -->
<dependency>
    <groupId>uk.org.webcompere</groupId>
    <artifactId>system-stubs-jupiter</artifactId>
    <version>1.1.0</version>
    <scope>test</scope>
</dependency>

We should note that we only need to import as many of these as we need for the test framework we're using. Indeed, both the latter two transitively include the core dependency.

Now let's write our first test.

3.2. JUnit 4 Environment Variables

We can control environment variables by declaring a JUnit 4 @Rule annotated field in our test class of type EnvironmentVariablesRule. This will be activated by JUnit 4 when our tests run and will allow us to set environment variables inside the test:

@Rule
public EnvironmentVariablesRule environmentVariablesRule = new EnvironmentVariablesRule();
@Test
public void givenEnvironmentCanBeModified_whenSetEnvironment_thenItIsSet() {
    environmentVariablesRule.set("ENV", "value1");
    assertThat(System.getenv("ENV")).isEqualTo("value1");
}

In practice, we may prefer to set the environment variable values in a @Before method so that the set-up can be shared across all tests:

@Before
public void before() {
    environmentVariablesRule.set("ENV", "value1")
      .set("ENV2", "value2");
}

Here we should note using the fluent set method, which makes setting multiple values easy through method chaining.

We can also use the constructor of the EnvironmentVariablesRule object to provide values on construction:

@Rule
public EnvironmentVariablesRule environmentVariablesRule =
  new EnvironmentVariablesRule("ENV", "value1",
    "ENV2", "value2");

There are several overloads of the constructor, allowing variables to be provided in different forms. The one in the above example allows any number of name-value pairs to be provided using varargs.

Each of the System Stubs JUnit 4 rules is a subclass of one of the core stubbing objects. They can also be used across the lifecycle of a whole test class with the @ClassRule annotation on a static field, which will cause them to be activated before the first test, and then cleaned up just after the last.

3.3. JUnit 5 Environment Variables

Before we use System Stubs objects inside a JUnit 5 test, we must add the extension to our test class:

@ExtendWith(SystemStubsExtension.class)
class EnvironmentVariablesJUnit5 {
    // tests
}

Then we can create a field in the test class for JUnit 5 to manage for us. We annotate this with @SystemStub so that the extension knows to activate it:

@SystemStub
private EnvironmentVariables environmentVariables;

The extension will only manage objects marked with @SystemStub, which allows us to use other System Stubs objects in the test manually if we prefer.

Here, we haven't provided any construction of the stub object. The extension constructs one for us, in the same way as the Mockito extension constructs mocks.

We can now use the object to help us set environment variables inside one of our tests:

@Test
void givenEnvironmentCanBeModified_whenSetEnvironment_thenItIsSet() {
    environmentVariables.set("ENV", "value1");
    assertThat(System.getenv("ENV")).isEqualTo("value1");
}

If we wanted to provide the environment variables that apply to all tests from outside of the test method, we can do that inside a @BeforeEach method or can use the constructor of EnvironmentVariables to set our values:

@SystemStub
private EnvironmentVariables environmentVariables =
  new EnvironmentVariables("ENV", "value1");

As with EnvironmentVariablesRule there are several overloads of the constructor, allowing us many ways to set the desired variables. We can also use the set method fluently to set values if we prefer:

@SystemStub
private EnvironmentVariables environmentVariables =
  new EnvironmentVariables()
    .set("ENV", "value1")
    .set("ENV2", "value2");

We can also make our fields static for them to be managed as part of the @BeforeAll/@AfterAll lifecycle.

3.4. JUnit 5 Parameter Injection

While placing the stub objects in fields is useful when using them for all of our tests, we may prefer to use them only for selected ones. This can be achieved by JUnit 5 parameter injection:

@Test
void givenEnvironmentCanBeModified(EnvironmentVariables environmentVariables) {
    environmentVariables.set("ENV", "value1");
    assertThat(System.getenv("ENV")).isEqualTo("value1");
}

In this case, the EnvironmentVariables object was constructed for us with its default constructor, allowing us to use it within a single test. The object has also been activated so that it is operating on the runtime environment. It will be tidied up when the test is finished.

All of the System Stubs objects have a default constructor and the ability to be reconfigured while running. We can inject as many as we need into our tests.

3.5. Execute-Around Environment Variables

The original System Lambda facade methods for creating stubs are also available via the SystemStubs class. Internally they are implemented by creating instances of the stubbing objects. Sometimes the object returned from the recipe is a stub object for further configuration and use:

withEnvironmentVariable("ENV3", "val")
    .execute(() -> {
        assertThat(System.getenv("ENV3")).isEqualTo("val");
    });

Behind the scenes, withEnvironmentVariable is doing the equivalent of:

return new EnvironmentVariables().set("ENV3", "val");

The execute method is common to all SystemStub objects. It sets up the stubbing defined by the object, then executes the lambda passed in. Afterward, it tidies up and returns control to the surrounding test.

If the test code returns a value, then that value can be returned by execute:

String extracted = new EnvironmentVariables("PROXY", "none")
  .execute(() -> System.getenv("PROXY"));
assertThat(extracted).isEqualTo("none");

This can be useful when the code we are testing needs to have access to environment settings to construct something. It's commonly used when testing things like AWS Lambda handlers, which are often configured through environment variables.

The advantage of this pattern for occasional tests is that we have to set up the stubbing explicitly, only where needed. Therefore it can be more precise and visible. However, it does not allow us to share the setup between tests and can be more long-winded.

3.6. Multiple System Stubs

We have already seen how the JUnit 4 and JUnit 5 plugins construct and activate stubbing objects for us. If there are multiple stubs, they are set up and torn down appropriately by the framework code.

However, when we construct stubbing objects for the execute-around pattern, we need our test code to run inside them all.

This can be achieved using the with/execute methods. These work by creating a composite from multiple stubbing objects used with a single execute:

with(new EnvironmentVariables("FOO", "bar"), new SystemProperties("prop", "val"))
  .execute(() -> {
      assertThat(System.getenv("FOO")).isEqualTo("bar");
      assertThat(System.getProperty("prop")).isEqualTo("val");
  });

Now we have seen the general form of using the System Stubs objects, both with and without JUnit framework support, let's look at the rest of the library's capabilities.

4. System Properties

We can call System.setProperty at any time in Java. However, this runs the risk of leaking the settings out of one test into another. The primary aim of SystemProperties stubbing is to restore the system properties to their original settings after the test is complete. However, it's also useful for common setup code to define which system properties should be used before the test starts.

4.1. JUnit 4 System Properties

By adding the rule to the JUnit 4 test class, we can insulate each test from any System.setProperty calls made in other test methods. We can also provide some upfront properties via the constructor:

@Rule
public SystemPropertiesRule systemProperties =
  new SystemPropertiesRule("db.connection", "false");

With this object, we can also set some additional properties in the JUnit @Before method:

@Before
public void before() {
    systemProperties.set("before.prop", "before");
}

We can also use the set method in the body of a test or use System.setProperty if we wish. We must only use set in creating the SystemPropertiesRule, or in the @Before method, as it stores the setting in the rule, ready for applying later.

4.2. JUnit 5 System Properties

We have two main use cases for using the SystemProperties object. We may wish to reset the system properties after each test case, or we may wish to prepare some common system properties in a central place for each test case to use.

Restoring system properties requires us to add both the JUnit 5 extension and a SystemProperties field to our test class:

@ExtendWith(SystemStubsExtension.class)
class RestoreSystemProperties {
    @SystemStub
    private SystemProperties systemProperties;
}

Now, each test will have any system properties it changes cleaned up afterward.

We can also do this for selected tests by parameter injection:

@Test
void willRestorePropertiesAfter(SystemProperties systemProperties) {
}

If we want the test to have properties set in it, then we can either assign those properties in the construction of our SystemProperties object or use a @BeforeEach method:

@ExtendWith(SystemStubsExtension.class)
class SetSomeSystemProperties {
    @SystemStub
    private SystemProperties systemProperties;
    @BeforeEach
    void before() {
        systemProperties.set("beforeProperty", "before");
    }
}

Again, let's note that the JUnit 5 test needs to be annotated with @ExtendWith(SystemStubsExtension.class). The extension will create the System Stubs object if we do not provide a new statement in the initializer list.

4.3. System Properties with Execute Around

The SystemStubs class provides a restoreSystemProperties method to allow us to run test code with properties restored:

restoreSystemProperties(() -> {
    // test code
    System.setProperty("unrestored", "true");
});
assertThat(System.getProperty("unrestored")).isNull();

This takes a lambda that returns nothing. If we wish to use a common set-up function to create properties, get a return value from the test method, or combine SystemProperties with other stubs via with/execute, then we can create the object explicitly:

String result = new SystemProperties()
  .execute(() -> {
      System.setProperty("unrestored", "true");
      return "it works";
  });
assertThat(result).isEqualTo("it works");
assertThat(System.getProperty("unrestored")).isNull();

4.4. Properties in Files

Both the SystemProperties and EnvironmentVariables objects can be constructed from a Map. This allows Java's Properties object to be provided as the source of either system properties or environment variables.

There are helper methods inside the PropertySource class for loading up Java properties from files or resources. These properties files are name/value pairs:

name=baeldung
version=1.0

We can load from the resource test.properties by using the fromResource function:

SystemProperties systemProperties =
  new SystemProperties(PropertySource.fromResource("test.properties"));

There are similar convenience methods in PropertySource for other sources, such as fromFile or fromInputStream.

5. System Out and System Err

When our application writes to System.out, it can be hard to test. This is sometimes solved by using an interface as the target of output and mocking that at test time:

interface LogOutput {
   void write(String line);
}
class Component {
    private LogOutput log;
    public void method() {
        log.write("Some output");
    }
}

Techniques like this work well with Mockito mocks but are not necessary if we can just trap System.out itself.

5.1. JUnit 4 SystemOutRule and SystemErrRule

To trap output to System.out in a JUnit 4 test, we add the SystemOutRule:

@Rule
public SystemOutRule systemOutRule = new SystemOutRule();

After that, any output to System.out can be read within the test:

System.out.println("line1");
System.out.println("line2");
assertThat(systemOutRule.getLines())
  .containsExactly("line1", "line2");

We have a choice of formats for the text. The above example uses the Stream<String> provided by getLines. We may also choose to get the whole block of text:

assertThat(systemOutRule.getText())
  .startsWith("line1");

However, we should note that this text will have newline characters that vary between platforms. We can replace newlines with \n on every platform by using the normalized form:

assertThat(systemOutRule.getLinesNormalized())
  .isEqualTo("line1\nline2\n");

The SystemErrRule works in the same way for System.err as its System.out counterpart:

@Rule
public SystemErrRule systemErrRule = new SystemErrRule();
@Test
public void whenCodeWritesToSystemErr_itCanBeRead() {
    System.err.println("line1");
    System.err.println("line2");
    assertThat(systemErrRule.getLines())
      .containsExactly("line1", "line2");
}

There is also a SystemErrAndOutRule class, which taps both System.out and System.err simultaneously into a single buffer.

5.2. JUnit 5 Example

As with the other System Stubs objects, we only need to declare a field or parameter of type SystemOut or SystemErr. This will provide us with a capture of the output:

@SystemStub
private SystemOut systemOut;
@SystemStub
private SystemErr systemErr;
@Test
void whenWriteToOutput_thenItCanBeAsserted() {
    System.out.println("to out");
    System.err.println("to err");
    assertThat(systemOut.getLines()).containsExactly("to out");
    assertThat(systemErr.getLines()).containsExactly("to err");
}

We can also use the SystemErrAndOut class to direct both sets of output into the same buffer.

5.3. Execute-Around Example

The SystemStubs facade provides some functions for tapping the output and returning it as a String:

@Test
void givenTapOutput_thenGetOutput() throws Exception {
    String output = tapSystemOutNormalized(() -> {
        System.out.println("a");
        System.out.println("b");
    });
    assertThat(output).isEqualTo("a\nb\n");
}

We should note that these methods do not provide as rich an interface as the raw objects themselves. The capture of output can't easily be combined with other stubbing, such as setting environment variables.

However, the SystemOut, SystemErr, and SystemErrAndOut objects can be used directly. For example, we could combine them with some SystemProperties:

SystemOut systemOut = new SystemOut();
SystemProperties systemProperties = new SystemProperties("a", "!");
with(systemOut, systemProperties)
  .execute(()  -> {
    System.out.println("a: " + System.getProperty("a"));
});
assertThat(systemOut.getLines()).containsExactly("a: !");

5.4. Muting

Sometimes our aim is not to capture output but to keep it from cluttering our test run logs. We can achieve this using the muteSystemOut or muteSystemErr functions:

muteSystemOut(() -> {
    System.out.println("nothing is output");
});

We can achieve the same thing across all tests via the JUnit 4 SystemOutRule:

@Rule
public SystemOutRule systemOutRule = new SystemOutRule(new NoopStream());

In JUnit 5, we can use the same technique:

@SystemStub
private SystemOut systemOut = new SystemOut(new NoopStream());

5.5. Customization

As we have seen, there are several variations for intercepting output. They all share a common base class in the library. For convenience, several helper methods and types, like SystemErrAndOut, help do common things. However, the library itself is easily customized.

We could provide our own target for capturing the output as an implementation of Output. We've already seen the Output class TapStream in use in the first examples. NoopStream is used for muting. We also have DisallowWriteStream that throws an error if something writes to it:

// throws an exception:
new SystemOut(new DisallowWriteStream())
  .execute(() -> System.out.println("boo"));

6. Mocking System In

We may have an application that reads input on stdin. Testing this could involve extracting the algorithm into a function that reads from any InputStream and then feeding it with a pre-preprepared input stream. In general, modular code is better, so this is a good pattern.

However, if we only test the core functions, we lose test coverage on the code which provides System.in as the source.

In any case, it can be inconvenient to construct our own streams. Luckily, System Stubs has solutions for all of these.

6.1. Test Input Streams

System Stubs provides a family of AltInputStream classes as alternative inputs for any code that reads from an InputStream:

LinesAltStream testInput = new LinesAltStream("line1", "line2");
Scanner scanner = new Scanner(testInput);
assertThat(scanner.nextLine()).isEqualTo("line1");

In this example, we've used an array of strings to construct LinesAltStream, but we could have supplied the input from a Stream<String>, allowing this to be used with any source of text data without necessarily loading it all into memory at once.

6.2. JUnit 4 Example

We can provide lines for input in a JUnit 4 test using the SystemInRule:

@Rule
public SystemInRule systemInRule =
  new SystemInRule("line1", "line2", "line3");

Then, the test code can read this input from System.in:

@Test
public void givenInput_canReadFirstLine() {
    assertThat(new Scanner(System.in).nextLine())
      .isEqualTo("line1");
}

6.3. JUnit 5 Example

For JUnit 5 tests, we create a SystemIn field:

@SystemStub
private SystemIn systemIn = new SystemIn("line1", "line2", "line3");

Then our tests will run with System.in providing these lines as input.

6.4. Execute-Around Example

The SystemStubs facade provides withTextFromSystemIn as a factory method that creates a SystemIn object for use with its execute method:

withTextFromSystemIn("line1", "line2", "line3")
  .execute(() -> {
      assertThat(new Scanner(System.in).nextLine())
        .isEqualTo("line1");
  });

6.5. Customization

More features can be added to the SystemIn object either on construction or while it is running within a test.

We can call andExceptionThrownOnInputEnd, which causes reading from System.in to throw an exception when it runs out of text. This can simulate an interrupted read from a file.

We can also set the input stream to come from any InputStream, like FileInputStream, by using setInputStream. We also have LinesAltStream and TextAltStream, which operate on input text.

7. Mocking System.Exit

As mentioned previously, if our code can call System.exit, it can make for dangerous and hard to debug test faults. One of our aims in stubbing System.exit is to make an accidental call into a traceable error. Another motivation is to test intentional exits from the software.

7.1. JUnit 4 Example

Let's add the SystemExitRule to a test class as a safety measure to prevent any System.exit from stopping the JVM:

@Rule
public SystemExitRule systemExitRule = new SystemExitRule();

However, we may also wish to see if the right exit code was used. For that, we need to assert that the code throws the AbortExecutionException, which is the System Stubs signal that System.exit was called.

@Test
public void whenExit_thenExitCodeIsAvailable() {
    assertThatThrownBy(() -> {
        System.exit(123);
    }).isInstanceOf(AbortExecutionException.class);
    assertThat(systemExitRule.getExitCode()).isEqualTo(123);
}

In this example, we've used assertThatThrownBy from AssertJ to catch and check the exception signaling exit occurred. Then we looked at getExitCode from the SystemExitRule to assert the exit code.

7.2. JUnit 5 Example

For JUnit 5 tests, we declare the @SystemStub field:

@SystemStub
private SystemExit systemExit;

Then we use the SystemExit class in the same way as SystemExitRule in JUnit 4. Given that the SystemExitRule class is a subclass of SystemExit, they have the same interface.

7.3. Execute-Around Example

The SystemStubs class provides catchSystemExit, which internally uses SystemExit‘s execute function:

int exitCode = catchSystemExit(() -> {
    System.exit(123);
});
assertThat(exitCode).isEqualTo(123);

Compared with the JUnit plugin examples, this code does not throw an exception to indicate a system exit. Instead, it catches the error and records the exit code. With the facade method, it returns the exit code.

When we use the execute method directly, exit is caught, and the exit code is set inside the SystemExit object. We can then call getExitCode to get the exit code, or null if there was none.

8. Custom Test Resources in JUnit 5

JUnit 4 already provides a simple structure for creating test rules like the ones used in System Stubs. If we want to make a new test rule for some resource, with a setup and teardown, we can subclass ExternalResource and provide overrides of the before and after methods.

JUnit 5 has a more complex pattern for resource management. For simple use cases, it's possible to use the System Stubs library as a starting point. The SystemStubsExtension operates on anything that satisfies the TestResource interface.

8.1. Creating a TestResource

We can create a subclass of TestResource and then use our custom objects in the same way we use System Stubs ones. We should note that we need to provide a default constructor if we want to use the automatic creation of fields and parameters.

Let's say we wanted to open a connection to a database for some tests and close it afterward:

public class FakeDatabaseTestResource implements TestResource {
    // let's pretend this is a database connection
    private String databaseConnection = "closed";
    @Override
    public void setup() throws Exception {
        databaseConnection = "open";
    }
    @Override
    public void teardown() throws Exception {
        databaseConnection = "closed";
    }
    public String getDatabaseConnection() {
        return databaseConnection;
    }
}

We're using the databaseConnection string as an illustration of a resource like a database connection. We modify the state of the resource in the setup and teardown methods.

8.2. Execute-Around is Built-In

Now let's try using this with the execute-around pattern:

FakeDatabaseTestResource fake = new FakeDatabaseTestResource();
assertThat(fake.getDatabaseConnection()).isEqualTo("closed");
fake.execute(() -> {
    assertThat(fake.getDatabaseConnection()).isEqualTo("open");
});

As we can see, the TestResource interface gave it the execute-around capabilities of the other objects.

8.3. Custom TestResource in JUnit 5 Test

We can also use this inside a JUnit 5 test:

@ExtendWith(SystemStubsExtension.class)
class FakeDatabaseJUnit5UnitTest {
    @Test
    void useFakeDatabase(FakeDatabaseTestResource fakeDatabase) {
        assertThat(fakeDatabase.getDatabaseConnection()).isEqualTo("open");
    }
}

So, it is easy to create additional test objects that follow the System Stubs design.

9. Environment and Property Overrides for JUnit 5 Spring Tests

Setting environment variables for Spring tests can be difficult. We might compose a custom rule for integration testing to set some system properties for Spring to pick up.

We may also use an ApplicationContextInitializer class to plug into our Spring Context, providing extra properties for the test.

As many Spring applications are controlled by system property or environment variable overrides, it may be easier to use System Stubs to set these in an outer test, with the Spring test running as an inner class.

There's a full example provided in the System Stubs documentation. We start by creating an outer class:

@ExtendWith(SystemStubsExtension.class)
public class SpringAppWithDynamicPropertiesTest {
    // sets the environment before Spring even starts
    @SystemStub
    private static EnvironmentVariables environmentVariables;
}

In this instance, the @SystemStub field is static and is initialized in the @BeforeAll method:

@BeforeAll
static void beforeAll() {
     String baseUrl = ...;
     environmentVariables.set("SERVER_URL", baseUrl);
}

This point in the test lifecycle allows some global resources to be created and applied to the running environment before the Spring test runs.

Then, we can put the Spring test into a @Nested class. This causes it to be run only when the parent class is set up:

@Nested
@SpringBootTest(classes = {RestApi.class, App.class},
    webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
class InnerSpringTest {
    @LocalServerPort
    private int serverPort;
    // Test methods
}

The Spring context is created against the state of the environment set by the @SystemStub objects in the outer class.

This technique also allows us to control the configuration of any other libraries that depend on the state of system properties or environment variables that may be running behind Spring Beans.

This can allow us to hook into the test lifecycle to modify things like proxy settings or HTTP connection pool parameters before a Spring test runs.

10. Conclusion

In this article, we've looked at the importance of being able to mock system resources and how System Stubs allows for complex configurations of stubbing with a minimum of code repetition through its JUnit 4 and JUnit 5 plugins.

We saw how to provide and isolate environment variables and system properties in our tests. Then we looked at capturing the output and controlling the input on the standard streams. We also looked at capturing and asserting calls to System.exit.

Finally, we looked at how to create custom test resources and how to use System Stubs with Spring.

As always, the full source code of the examples is available over on GitHub.

The post Guide to the System Stubs Library first appeared on Baeldung.

        

Digital Certificate: How to Import .cer File into Truststore File

$
0
0

1. Overview

The SSL protocol is usually the preferred choice whenever applications need to communicate with clients over the network. Together with encryption of data, SSL makes it mandatory for an application, like a browser, to exchange asymmetric keys during the handshake in order to establish a secure connection.

Generally, applications share the asymmetric keys in X.509 certificates format. Therefore, before SSL handshaking, clients must import such certificates into their truststore files. 

In this article, we'll discuss a few tools that we can use to import certificates in .cer format into the client's truststore.

2. The keytool Command

The JDK distribution provides a keytool utility that we can use to manage Java keystores (JKS). The most important purpose of this command is to generate self-signed X.509 certificates for testing SSL communication between a client and a server.

We can also import self-signed or CA-signed certificates into a JKS file and use it as a truststore:

keytool -importcert -alias trustme -file baeldung.cer -keystore cacerts
Enter keystore password:
Trust this certificate? [no]:  yes
Certificate was added to keystore

Here, we've imported a self-signed baeldung.cer certificate using the keytool command. We can import this certificate into any Java keystore. For example, the one shown here is adding the certificate in the cacerts keystore in the JDK.

If we now list the certificates in the keystore, we'll see an alias trustme:

keytool -list -keystore cacerts
trustme, Oct 31, 2020, trustedCertEntry,
Certificate fingerprint (SHA1): 04:40:6C:B0:06:65:EE:80:9A:90:A5:E9:DA:19:05:4A:AA:F2:CF:A4

3. The openssl Command

Until now, we've only discussed importing the certificates into a JKS file. Such keystores can only be used with Java applications. If we have to implement an SSL library in other languages or use the same certificate across multiple language platforms, we're more likely to use PKCS12 keystores.

To import a certificate into a PKCS12 keystore, we can also use openssl :

openssl pkcs12 -export -in baeldung.cer -inkey baeldung.key -out baeldung.keystore -name trustme

This command will import a certificate named baeldung.cer into a keystore baeldung.keystore with an alias trustme. 

We can see the imported certificate in the keystore:

openssl pkcs12 -info -in baeldung.keystore
Enter Import Password:
MAC: sha1, Iteration 2048
MAC length: 20, salt length: 8
PKCS7 Encrypted data: pbeWithSHA1And40BitRC2-CBC, Iteration 2048
Certificate bag
Bag Attributes
    friendlyName: trustme
    localKeyID: F4 36 4E 19 E4 E4 E7 65 74 56 FB 50 40 02 68 8B EC F0 4D B3
subject=C = IN, ST = DE, L = DC, O = BA, OU = AU, CN = baeldung.com
issuer=C = IN, ST = DE, L = DC, O = BA, OU = AU, CN = baeldung.com
-----BEGIN CERTIFICATE-----
MIIFkTCCA3mgAwIBAgIUL/OjGExnppeZkiNNh0i2+TPHaCQwDQYJKoZIhvcNAQEL
BQAwWDELMAkGA1UEBhMCSU4xCzAJBgNVBAgMAkRFMQswCQYDVQQHDAJEQzELMAkG
A1UECgwCQkExCzAJBgNVBAsMAkFVMRUwEwYDVQQDDAxiYWVsZHVuZy5jb20wHhcN
MjAxMTAzMTIwMjI5WhcNMjExMTAzMTIwMjI5WjBYMQswCQYDVQQGEwJJTjELMAkG
A1UECAwCREUxCzAJBgNVBAcMAkRDMQswCQYDVQQKDAJCQTELMAkGA1UECwwCQVUx
FTATBgNVBAMMDGJhZWxkdW5nLmNvbTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCC
AgoCggIBAK/XF/xmqQRJlTx2Vtq70x1KFwkHJEcZOyFbQP7O9RgicvMTAnbZtKpS
BSVjwroklIr4OVK4wmwdaTnlIm22CsFrbn+iBVL00tVs+sBYEcgO5nphVWGFbvHl
Q3PO4vTedSyH1qIyYrrhAn8wYvzdmr2g6tRwBX8K5H948Zb32Xbp5r9aR5M2i8Qz
fc0QasJUM5b71TNt8Qcsru3pFKj5hUMBTNrGCQrr6vrADTcG0YHuVSMeJId7f67h
l0vEY0BmRPnWNwGe+Sg/jqOWH9WWvkk/umkEQNWCQZaXZNZZ8jl5WMKFnmA7sPQ+
UjZPabNOTxhz6fJv5nJu7aMS/6tUWO0SdQ+ctO3HgR42wtBPoEOOuFMP6OqHI4hf
CXFTYg6aLwxFJP7LngfRvETgzVlsb9L/m++JBeoWRqpWaQUEgxDYJGFGA5dwQJaf
f24d042i44X0WqBBoWLjSQd/JFVH5MF17waiYpxFBOgpz3XEM/1j+juJPVut2k96
3ecgR54iKILbibizPUojn7t3AFT1Ug8exdefHdf+QsL8/L5+8/xOYkp/pnglQJJl
W0Lq4Sh9LWiux9XVdY6n2UYf/crgLSHatVkPa26cysdXhiiEPn4yYr2AdYVf0Xr5
W5PULufdi0HW2Eja/TfeXoBQtkdidqP8SMW+DwqafX80s37bZZBvAgMBAAGjUzBR
MB0GA1UdDgQWBBQPHIpCFhAy3kGAbzHpXMjXCMVQRzAfBgNVHSMEGDAWgBQPHIpC
FhAy3kGAbzHpXMjXCMVQRzAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUA
A4ICAQBzOK52I7lDX+7CHy6cQ8PnLZjAD4S5qC1P9dMy50E9N6Tmw2TiPbWl9CnU
7a/kVO6xDJDmNMnqRbHmlZclJaTFv6naXSX27PdIWjhwAfLjNa+FO9JNwMgiP25I
ISVjyrA3HwbhFnMs5FyBW9hbxfQ+X2Q2ooa+J3TKU7FImuDRKF3Sdb63+/j0go8S
5/TsoYpQxg86xbWf6IYGYwegd2SPSWUZ0HQSobZ7fRA7Y0EyPKgyqsBbmDtJ+X1g
P8Kep4N1oocc7ZkkX4pNfXTgXib9fUkKMAfRJz8w62z8I1OM61bciW7V2VSp/Y5p
iTihyuwO0aHG+YTnsr3qFrSFQLQUjCeBvx+euQelsGm8W9xM9YfASXiaEwCmb9PO
i/umD70J1e0HFDay9FW6mMoCCEBTZIF9ARqzhHgg9fi9iH2ctrsxadFAlOTFp5+/
p+nxrencfvc4CP6aHoqkE45HpMBoNDAxRMVd/FRzIG2as0q5At873MNFXP6WxmQV
4KGIhteNLyrXk82yHdHfm1EENTI7OEst/Fc8O3fFy9wn0OvoHIuCv1FVCq4Gnjmq
vNnBnGldrYruCNvj5KT6U14FFdK/5Zng0nSky4oMTs49zt392bdYi31SHWeATdw/
PscFRdig2stoI8ku6R+K7XvmseGzPmUW4K2IWU0zdRP2a4YyvA==
-----END CERTIFICATE-----
PKCS7 Data
Shrouded Keybag: pbeWithSHA1And3-KeyTripleDES-CBC, Iteration 2048
Bag Attributes
    friendlyName: trustme
    localKeyID: F4 36 4E 19 E4 E4 E7 65 74 56 FB 50 40 02 68 8B EC F0 4D B3
Key Attributes: <No Attributes>

So, we've successfully imported our certificate into the PKCS12 keystore. As a result, this keystore can now be used as a truststore file in SSL client applications like HTTP client libraries. Likewise, this file can also be used as a keystore in SSL server applications like Tomcat.

4. Conclusion

In this article, we discussed two popular SSL tools for managing digital certificates — OpenSSL and Java Keytool. We further used the keytool and openssl commands to import a certificate in .cer format into JKS and PKCS12 files, respectively.

The post Digital Certificate: How to Import .cer File into Truststore File first appeared on Baeldung.

        

Security Context Basics: User, Subject and Principal

$
0
0

1. Overview

Security is a fundamental part of any Java application. Also, we can find many security frameworks that can handle security concerns. Additionally, we use a few terms commonly like the subject, principal, and user in these frameworks.

In this tutorial, we're going to explain these basic concepts of security frameworks. Also, we'll show their relationships and differences.

2. Subject

In a security context, the subject represents the source of a request. The subject is an entity that obtains information about resources or modifies resources. Additionally, a subject can also be a user, a program, a process, a file, a computer, a database, etc.

For example, a person needs to authorize access to resources and applications to authenticate the request source. In this case, this person is a subject.

Let's take a look at our example that implemented base on the JAAS framework:

Subject subject = loginContext.getSubject();
PrivilegedAction privilegedAction = new ResourceAction();
Subject.doAsPrivileged(subject, privilegedAction, null);

3. Principal

After successful authentication, we have a populated subject with many associated identities, such as roles, social security number(SSN), etc. In other words, these identifiers are principals, and the subject represents them.

For instance, a person may have an account number principal (“87654-3210”) and other unique identifiers, distinguishing it from other subjects.

Let's see how to create an UserPrincipal after a successful login and add it to a Subject:

@Override
public boolean commit() throws LoginException {
    if (!loginSucceeded) {
        return false;
    }
    userPrincipal = new UserPrincipal(username);
    subject.getPrincipals().add(userPrincipal);
    return true;
}

4. User

Typically, a user represents a person who accesses resources to perform some action or accomplish a work task.

Also, we can use a user as a principal, and on the other hand, a principal is an identity assigned to a user. UserPrincipal is an excellent example of a user in the JAAS framework discussed in the previous section.

5. Difference Between Subject, Principal, and User

As we saw in the above sections, we can represent different aspects of the same user's identity by using principals. They are subsets of subjects, and users are subsets of principals that are referring to the end-user or interactive operators.

6. Conclusion

In this tutorial, we discussed the definition of the subject, principal, and user that they are common in most of the security frameworks. Also, we showed the difference between them.

The implementation of all these examples and code snippets can be found in the GitHub project.

The post Security Context Basics: User, Subject and Principal first appeared on Baeldung.

        

Thymeleaf Variables

$
0
0

1. Introduction

In this tutorial, we're going to take a look at variables in Thymeleaf. We'll create a Spring Boot example that will fetch a list of Baeldung articles and display them in a Thymeleaf HTML template.

2. Maven Dependencies

To work with Thymeleaf, we'll need to add the spring-boot-starter-thymeleaf and spring-boot-starter-web dependencies:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-thymeleaf</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
</dependency>

3. Web Controller

First, we'll create a web controller with a GET endpoint that returns a page with a list of Baeldung articles.

The method annotated with @GetMapping will take a single parameter – the Model. It holds all the global variables that can be further used inside the Thymeleaf template. In our case, the model will have just one parameter – the list of articles.

The Article class will consist of two String fields, name and url:

public class Article {
    private String name;
    private String url;
    // constructor, getters and setters
}

The return value of our controller's method should be the name of the desired Thymeleaf template. This name should correspond to the HTML file located in the src/resource/template directory. In our case, it'll be src/resource/template/articles-list.html.

Let's take a quick look at our Spring controller:

@Controller
@RequestMapping("/api/articles")
public class ArticlesController {
    @GetMapping
    public String allArticles(Model model) {
        model.addAttribute("articles", fetchArticles());
        return "articles-list";
    }
    private List<Article> fetchArticles() {
        return Arrays.asList(
          new Article(
            "Introduction to Using Thymeleaf in Spring",
            "https://www.baeldung.com/thymeleaf-in-spring-mvc"
          ),
          // a few other articles
        );
    }
}

After running the application, the articles page will be available at http://localhost:8080/articles.

4. Thymeleaf Template

Now, let's move into the Thymeleaf HTML template. It should have the standard HTML document structure with just the additional Thymeleaf namespace definition:

<html xmlns:th="http://www.thymeleaf.org">

We'll use this as a template in further examples, where we'll be replacing just the content of the <main> tag:

<!DOCTYPE HTML>
<html xmlns:th="http://www.thymeleaf.org">
<head>
    <title>Thymeleaf Variables</title>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
</head>
<body>
    <main>
        ...
    </main>
</body>
</html>

5. Define Variables

There are two ways we can define a variable in a Thymeleaf template. The first option is to take a single element while iterating over an array:

<div th:each="article : ${articles}">
    <a th:text="${article.name}" th:href="https://feeds.feedblitz.com/~/t/0/0/baeldung/~${article.url}"></a>
</div>

As a result, we'll get a <div> with several <a> elements corresponding to the number of articles in the articles variable.

Another way is to define a new variable based on another one. For example, we can take the first element of the articles array:

<div th:with="firstArticle=${articles[0]}">
    <a th:text="${firstArticle.name}" th:href="https://feeds.feedblitz.com/~/t/0/0/baeldung/~${firstArticle.url}"></a>
</div>

Or we can create a new variable that holds just the article's name:

<div th:each="article : ${articles}", th:with="articleName=${article.name}">
    <a th:text="${articleName}" th:href="https://feeds.feedblitz.com/~/t/0/0/baeldung/~${article.url}"></a>
</div>

In the above example, the ${article.name} and ${articleName} fragments are replaceable.

It's also possible to define multiple variables. For example, we can create two separate variables to hold the article name and URL:

<div th:each="article : ${articles}" th:with="articleName=${article.name}, articleUrl=${article.url}">
    <a th:text="${articleName}" th:href="https://feeds.feedblitz.com/~/t/0/0/baeldung/~${articleUrl}"></a>
</div>

6. Variables Scope

Variables passed to the Model in a controller have a global scope. This means they can be used in every place of our HTML templates.

On the other hand, variables defined in the HTML template have a local scope. They can be used only within the range of the element that they were defined in.

For example, the below code is correct as the <a> element is within the firstDiv:

<div id="firstDiv" th:with="firstArticle=${articles[0]}">
    <a th:text="${firstArticle.name}" th:href="https://feeds.feedblitz.com/~/t/0/0/baeldung/~${firstArticle.url}"></a>
</div>

On the other hand, when we try to use the firstArticle in another div:

<div id="firstDiv" th:with="firstArticle=${articles[0]}">
    <a th:text="${firstArticle.name}" th:href="https://feeds.feedblitz.com/~/t/0/0/baeldung/~${firstArticle.url}"></a>
</div>
<div id="secondDiv">
    <h2 th:text="${firstArticle.name}"></h2>
</div>

We'll get an exception during compile-time saying that the firstArticle is null:

org.springframework.expression.spel.SpelEvaluationException: EL1007E: Property or field 'name' cannot be found on null

This is because the <h2> element is trying to use a variable defined in the firstDiv, which is out of scope.

If we still need to use the firstArticle variable inside the secondDiv, we would need to define it again in secondDiv or wrap these two div tags in a common element and define the firstArticle in it.

7. Changing a Variable's Value

It's also possible to overwrite a variable's value in a given scope:

<div id="mainDiv" th:with="articles = ${ { articles[0], articles[1] } }">
    <div th:each="article : ${articles}">
        <a th:text="${article.name}" th:href="https://feeds.feedblitz.com/~/t/0/0/baeldung/~${article.url}"></a>
    </div>
</div>

In the above example, we redefined the articles variable to have just two first elements.

Note that outside of the mainDiv, the articles variable will still have its original value passed in the controller.

8. Conclusion

In this tutorial, we've learned how to define and use variables in Thymeleaf. As always, all the source code is available over on GitHub.

The post Thymeleaf Variables first appeared on Baeldung.

        

NoSuchFieldError in Java

$
0
0

1. Overview

In this article, we'll demonstrate the reason behind NoSuchFieldError and discover how to resolve it.

2. NoSuchFieldError

As the name suggests, NoSuchFieldError occurs when a specified field doesn't exist. NoSuchFieldError extends the IncompatibleClassChangeError class and is thrown when the application tries to access or modify a field of an object or a static field of a class but the object or class no longer has that field.

IncompatibleClassChangeError class extends the LinkageError class and occurs when we perform incompatible class definition changes. Finally, LinkageError extends Error and shows that a class has some dependency on another incompatibly changed class.

Let's see this error in action with the help of an example. As a first step, let's create a Dependency class:

public class Dependency {
    public static String message = "Hello Baeldung!!";
}

Then we'll create a FieldErrorExample class that refers to a field of our Dependency class:

public class FieldErrorExample {
    public static String getDependentMessage() {
        return Dependency.message;
    }
}

Let's also add code to check whether we're getting a message from the Dependency class:

public static void fetchAndPrint() {
    System.out.println(getDependentMessage());
}

Now, we can compile these files using the javac command, and upon execution of the FieldErrorExample class using the java command, it will print the specified message.

However, if we comment out, remove, or change the attribute name in the Dependency class and recompile it, then we'll run into our error.

For example, let's change the attribute name in our Dependency class:

public class Dependency {
    public static String msg = "Hello Baeldung!!";
}

Now, if we recompile only our Dependency class, and then execute FieldErrorExample again, we'll encounter the NoSuchFieldError:

Exception in thread "main" java.lang.NoSuchFieldError: message

The above error occurred because the FieldErrorExample class still refers to the static field message of the Dependency class, but it no longer exists — we've made an incompatible change to the Dependency class.

3. Resolving the Error

To avoid this error, we need to clean and compile the existing files. We can do this using the javac command or with Maven by running mvn clean install. By performing this step, we'll have all the latest compiled files, and we'll avoid running into the error.

If the error persists, then the problem might be multiple JAR files: one while compiling, and another while running. This often happens when the application depends on external JARs. Here, we should validate the order of the JARs in the build path to identify the inconsistent JAR.

If we have to investigate further, it's helpful to run the application with -verbose: class option to check the loaded classes. This can help us identify the outdated class.

Sometimes a third-party JAR might be internally referring to another version, which results in NoSuchFieldError. If this happens, we can use mvn dependency:tree -Dverbose. This generates the maven dependency tree and helps us in identifying the inconsistent JAR.

4. Conclusion

In this short tutorial, we have shown why NoSuchFieldError occurs and looked at how we can resolve it.

As always, the code is available over on GitHub.

The post NoSuchFieldError in Java first appeared on Baeldung.

        

Java AES Encryption and Decryption

$
0
0

1. Overview

The symmetric-key block cipher plays an important role in data encryption. It means that the same key is used for both encryption and decryption. The Advanced Encryption Standard (AES) is a widely used symmetric-key encryption algorithm.

In this tutorial, we’ll see how to implement AES encryption and decryption using the Java Cryptography Architecture (JCA) within the JDK.

2. AES Algorithm

The AES algorithm is an iterative, symmetric-key block cipher that supports cryptographic keys (secret keys) of 128, 192, and 256 bits to encrypt and decrypt data in blocks of 128 bits. The below figure shows the high-level AES algorithm:

High Level AES Algorithm

If the data to be encrypted does not meet the block size of 128 bits requirement, it must be padded. Padding is a process of filling up the last block to 128 bits.

3. AES Variations

The AES algorithm has six modes of operation:

  1. ECB (Electronic Code Book)
  2. CBC (Cipher Block Chaining)
  3. CFB (Cipher FeedBack)
  4. OFB (Output FeedBack)
  5. CTR (Counter)
  6. GCM (Galois/Counter Mode)

The mode of operation may be applied in order to strengthen the effect of the encryption algorithm. Moreover, the mode of operation may convert the block cipher into a stream cipher. Each mode has its strength and weakness. Let’s have a quick review.

3.1. ECB

This mode of operation is the simplest of all. The plaintext is divided into blocks with a size of 128 bits. Then each block will be encrypted with the same key and algorithm. Therefore, it produces the same result for the same block. This is the main weakness of this mode and it is not recommended for encryption. It requires padding data.

3.2. CBC

In order to overcome the ECB weakness, CBC mode uses an Initialization Vector (IV) to augment the encryption. First, CBC uses the plaintext block xor with the IV. Then it encrypts the result to the ciphertext block. In the next block, it uses the encryption result to xor with the plaintext block until the last block.

In this mode, encryption can not be parallelized, but decryption can be parallelized. It also requires padding data.

3.3. CFB

This mode can be used as a stream cipher. First, it encrypts the IV, then it will xor with the plaintext block to get ciphertext. Then CFB encrypts the encryption result to xor the plaintext. It needs an IV.

In this mode, decryption can be parallelized but encryption can not be parallelized.

3.4. OFB

This mode can be used as a stream cipher. First, it encrypts the IV. Then it uses the encryption results to xor the plaintext to get ciphertext.

It doesn’t require padding data and will not be affected by the noisy block.

3.5. CTR

This mode uses the value of a counter as an IV. It's very similar to OFB, but it uses the counter to be encrypted every time instead of the IV.

This mode has two strengths, including encryption/decryption parallelization, and noise in one block does not affect other blocks.

3.6. GCM

This mode is an extension of the CTR mode. The GCM has received significant attention and is recommended by NIST. The GCM mode outputs ciphertext and an authentication tag. The main advantage of this mode, compared to other operation modes of the algorithm, is its efficiency.

In this tutorial, we'll use the AES/CBC/PKCS5Padding algorithm because it is widely used in many projects.

3.7. Size of Data After Encryption

As mentioned earlier, the AES has a block size of 128 bits or 16 bytes. The AES does not change the size, and the ciphertext size is equal to the cleartext size. Also, in ECB and CBC modes, we should use a padding algorithm likes PKCS 5. So, the size of data after encryption is:

ciphertext_size (bytes) = cleartext_size + (16 - (cleartext_size % 16))

For storing IV with ciphertext, we need to add 16 more bytes.

4. AES Parameters

In the AES algorithm, we need three parameters: input data, secret key, and IV. IV is not used in ECB mode.

4.1. Input Data

The input data to the AES can be string, file, object, and password-based.

4.2. Secret Key

There are two ways for generating a secret key in the AES: generating from a random number or deriving from a given password.

In the first approach, the secret key should be generated from a Cryptographically Secure (Pseudo-)Random Number Generator like the SecureRandom class.

For generating a secret key, we can use the KeyGenerator class. Let’s define a method for generating the AES key with the size of n (128, 192, and 256) bits:

public static SecretKey generateKey(int n) throws NoSuchAlgorithmException {
    KeyGenerator keyGenerator = KeyGenerator.getInstance("AES");
    keyGenerator.init(n);
    SecretKey key = keyGenerator.generateKey();
    return key;
}

In the second approach, the AES secret key can be derived from a given password using a password-based key derivation function like PBKDF2. We also need a salt value for turning a password into a secret key. The salt is also a random value.

We can use the SecretKeyFactory class with the PBKDF2WithHmacSHA256 algorithm for generating a key from a given password.

Let’s define a method for generating the AES key from a given password with 65,536 iterations and a key length of 256 bits:

public static SecretKey getKeyFromPassword(String password, String salt)
    throws NoSuchAlgorithmException, InvalidKeySpecException {
    
    SecretKeyFactory factory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA256");
    KeySpec spec = new PBEKeySpec(password.toCharArray(), salt.getBytes(), 65536, 256);
    SecretKey secret = new SecretKeySpec(factory.generateSecret(spec)
        .getEncoded(), "AES");
    return secret;
}

4.3. Initialization Vector (IV)

IV is a pseudo-random value and has the same size as the block that is encrypted. We can use the SecureRandom class to generate a random IV.

Let’s define a method for generating an IV:

public static IvParameterSpec generateIv() {
    byte[] iv = new byte[16];
    new SecureRandom().nextBytes(iv);
    return new IvParameterSpec(iv);
}

5. Encryption and Decryption

5.1. String

To implement input string encryption, we first need to generate the secret key and IV according to the previous section. As the next step, we create an instance from the Cipher class by using the getInstance() method.

Additionally, we configure a cipher instance using the init() method with a secret key, IV, and encryption mode. Finally, we encrypt the input string by invoking the doFinal() method. This method gets bytes of input and returns ciphertext in bytes:

public static String encrypt(String algorithm, String input, SecretKey key,
    IvParameterSpec iv) throws NoSuchPaddingException, NoSuchAlgorithmException,
    InvalidAlgorithmParameterException, InvalidKeyException,
    BadPaddingException, IllegalBlockSizeException {
    
    Cipher cipher = Cipher.getInstance(algorithm);
    cipher.init(Cipher.ENCRYPT_MODE, key, iv);
    byte[] cipherText = cipher.doFinal(input.getBytes());
    return Base64.getEncoder()
        .encodeToString(cipherText);
}

For decrypting an input string, we can initialize our cipher using the DECRYPT_MODE to decrypt the content:

public static String decrypt(String algorithm, String cipherText, SecretKey key,
    IvParameterSpec iv) throws NoSuchPaddingException, NoSuchAlgorithmException,
    InvalidAlgorithmParameterException, InvalidKeyException,
    BadPaddingException, IllegalBlockSizeException {
    
    Cipher cipher = Cipher.getInstance(algorithm);
    cipher.init(Cipher.DECRYPT_MODE, key, iv);
    byte[] plainText = cipher.doFinal(Base64.getDecoder()
        .decode(cipherText));
    return new String(plainText);
}

Let's write a test method for encrypting and decrypting a string input:

@Test
void givenString_whenEncrypt_thenSuccess()
    throws NoSuchAlgorithmException, IllegalBlockSizeException, InvalidKeyException,
    BadPaddingException, InvalidAlgorithmParameterException, NoSuchPaddingException { 
    
    String input = "baeldung";
    SecretKey key = AESUtil.generateKey(128);
    IvParameterSpec ivParameterSpec = AESUtil.generateIv();
    String algorithm = "AES/CBC/PKCS5Padding";
    String cipherText = AESUtil.encrypt(algorithm, input, key, ivParameterSpec);
    String plainText = AESUtil.decrypt(algorithm, cipherText, key, ivParameterSpec);
    Assertions.assertEquals(input, plainText);
}

5.2. File

Now let's encrypt a file using the AES algorithm. The steps are the same, but we need some IO classes to work with the files. Let's encrypt a text file:

public static void encryptFile(String algorithm, SecretKey key, IvParameterSpec iv,
    File inputFile, File outputFile) throws IOException, NoSuchPaddingException,
    NoSuchAlgorithmException, InvalidAlgorithmParameterException, InvalidKeyException,
    BadPaddingException, IllegalBlockSizeException {
    
    Cipher cipher = Cipher.getInstance(algorithm);
    cipher.init(Cipher.ENCRYPT_MODE, key, iv);
    FileInputStream inputStream = new FileInputStream(inputFile);
    FileOutputStream outputStream = new FileOutputStream(outputFile);
    byte[] buffer = new byte[64];
    int bytesRead;
    while ((bytesRead = inputStream.read(buffer)) != -1) {
        byte[] output = cipher.update(buffer, 0, bytesRead);
        if (output != null) {
            outputStream.write(output);
        }
    }
    byte[] outputBytes = cipher.doFinal();
    if (outputBytes != null) {
        outputStream.write(outputBytes);
    }
    inputStream.close();
    outputStream.close();
}

Please note that trying to read the entire file – particularly if it is large – into memory is not recommended. Instead, we encrypt a buffer at a time.

For decrypting a file, we use similar steps and initialize our cipher using DECRYPT_MODE as we saw before.

Again, let's define a test method for encrypting and decrypting a text file. In this method, we read the baeldung.txt file from the test resource directory, encrypt it into a file called baeldung.encrypted, and then decrypt the file into a new file:

@Test
void givenFile_whenEncrypt_thenSuccess() 
    throws NoSuchAlgorithmException, IOException, IllegalBlockSizeException, 
    InvalidKeyException, BadPaddingException, InvalidAlgorithmParameterException, 
    NoSuchPaddingException {
    
    SecretKey key = AESUtil.generateKey(128);
    String algorithm = "AES/CBC/PKCS5Padding";
    IvParameterSpec ivParameterSpec = AESUtil.generateIv();
    Resource resource = new ClassPathResource("inputFile/baeldung.txt");
    File inputFile = resource.getFile();
    File encryptedFile = new File("classpath:baeldung.encrypted");
    File decryptedFile = new File("document.decrypted");
    AESUtil.encryptFile(algorithm, key, ivParameterSpec, inputFile, encryptedFile);
    AESUtil.decryptFile(
      algorithm, key, ivParameterSpec, encryptedFile, decryptedFile);
    assertThat(inputFile).hasSameTextualContentAs(decryptedFile);
}

5.3. Password-Based

We can do the AES encryption and decryption using the secret key that is derived from a given password.

For generating a secret key, we use the getKeyFromPassword() method. The encryption and decryption steps are the same as those shown in the string input section. We can then use the instantiated cipher and the provided secret key to perform the encryption.

Let's write a test method:

@Test
void givenPassword_whenEncrypt_thenSuccess() 
    throws InvalidKeySpecException, NoSuchAlgorithmException, 
    IllegalBlockSizeException, InvalidKeyException, BadPaddingException, 
    InvalidAlgorithmParameterException, NoSuchPaddingException {
    
    String plainText = "www.baeldung.com";
    String password = "baeldung";
    String salt = "12345678";
    IvParameterSpec ivParameterSpec = AESUtil.generateIv();
    SecretKey key = AESUtil.getKeyFromPassword(password,salt);
    String cipherText = AESUtil.encryptPasswordBased(plainText, key, ivParameterSpec);
    String decryptedCipherText = AESUtil.decryptPasswordBased(
      cipherText, key, ivParameterSpec);
    Assertions.assertEquals(plainText, decryptedCipherText);
}

5.4. Object

For encrypting a Java object, we need to use the SealedObject class. The object should be Serializable. Let's begin by defining a Student class:

public class Student implements Serializable {
    private String name;
    private int age;
    // standard setters and getters
}

Next, let's encrypt the Student object :

public static SealedObject encryptObject(String algorithm, Serializable object,
    SecretKey key, IvParameterSpec iv) throws NoSuchPaddingException,
    NoSuchAlgorithmException, InvalidAlgorithmParameterException, 
    InvalidKeyException, IOException, IllegalBlockSizeException {
    
    Cipher cipher = Cipher.getInstance(algorithm);
    cipher.init(Cipher.ENCRYPT_MODE, key, iv);
    SealedObject sealedObject = new SealedObject(object, cipher);
    return sealedObject;
}

The encrypted object can later be decrypted using the correct cipher:

public static Serializable decryptObject(String algorithm, SealedObject sealedObject,
    SecretKey key, IvParameterSpec iv) throws NoSuchPaddingException,
    NoSuchAlgorithmException, InvalidAlgorithmParameterException, InvalidKeyException,
    ClassNotFoundException, BadPaddingException, IllegalBlockSizeException,
    IOException {
    
    Cipher cipher = Cipher.getInstance(algorithm);
    cipher.init(Cipher.DECRYPT_MODE, key, iv);
    Serializable unsealObject = (Serializable) sealedObject.getObject(cipher);
    return unsealObject;
}

Let's write a test case:

@Test
void givenObject_whenEncrypt_thenSuccess() 
    throws NoSuchAlgorithmException, IllegalBlockSizeException, InvalidKeyException,
    InvalidAlgorithmParameterException, NoSuchPaddingException, IOException, 
    BadPaddingException, ClassNotFoundException {
    
    Student student = new Student("Baeldung", 20);
    SecretKey key = AESUtil.generateKey(128);
    IvParameterSpec ivParameterSpec = AESUtil.generateIv();
    String algorithm = "AES/CBC/PKCS5Padding";
    SealedObject sealedObject = AESUtil.encryptObject(
      algorithm, student, key, ivParameterSpec);
    Student object = (Student) AESUtil.decryptObject(
      algorithm, sealedObject, key, ivParameterSpec);
    assertThat(student).isEqualToComparingFieldByField(object);
}

6. Conclusion

In summary, we've learned how to encrypt and decrypt input data like strings, files, objects, and password-based data, using the AES algorithm in Java. Additionally, we've discussed the AES variations and the size of data after encryption.

As always, the full source code of the article is available over on GitHub.

The post Java AES Encryption and Decryption first appeared on Baeldung.

        

Introduction to Servlets and Servlet Containers

$
0
0

1. Overview

In this tutorial, we'll understand conceptually what servlets and servlet containers are and how they work.

We'll also see them in the context of a request, response, session objects, shared variables, and multithreading.

2. What are Servlets and Their Containers

Servlets are a component of the JEE framework used for web development. They are basically Java programs that run inside the boundaries of a container. On the whole, they are responsible for accepting a request, processing it, and sending a response back. Introduction to Java servlets provides a good basic understanding of the subject.

To use them, servlets need to be registered first so that a container, either JEE or Spring-based, can pick them up at start-up. In the beginning, the container instantiates a servlet by calling its init() method.

Once its initialization is complete, the servlet is ready to accept incoming requests. Subsequently, the container directs these requests for processing in the servlet's service() method. After that, it further delegates the request to the appropriate method such as doGet() or doPost() based on the HTTP request type.

With destroy(), the container tears the servlet down, and it can no longer accept incoming requests. We call this cycle of init-service-destroy as the lifecycle of a servlet.

Now let's look at this from the point of view of a container, such as Apache Tomcat or Jetty. At start-up, it creates an object of ServletContext. The job of the ServletContext is to function as the server or container's memory and remember all the servlets, filters, and listeners associated with the web application, as described in its web.xml or equivalent annotations. Until we stop or terminate the container, ServletContext stays with it.

However, the servlet's load-on-startup parameter plays an important role here. If this parameter has a value greater than zero, only then the server initializes it at start-up. If this parameter is not specified, then the servlet's init() is called when a request hits it for the very first time.

3. Request, Response, and Session

In the previous section, we talked about sending requests and receiving responses, which basically is the cornerstone of any client-server application. Now, let's look at them in detail with respect to servlets.

In this case, a request would be represented by HttpServletRequest and response with HttpServletResponse.

Whenever a client such as a browser, or a curl command, sends in a request, the container creates a new HttpServletRequest and HttpServletResponse object. It then passes on these new objects to the servlet's service method. Based on the HttpServletRequest‘s method attribute, this method determines which of the doXXX methods should be called.

Apart from the information about the method, the request object also carries other information such as headers, parameters, and body. Similarly, the HttpServletResponse object also carries headers, parameters, and body – we can set them up in our servlet's doXXX method.

These objects are short-lived. When the client gets the response back, the server marks the request and response objects for garbage collection.

How would we then maintain a state between subsequent client requests or connections? HttpSession is the answer to this riddle.

This basically binds objects to a user session, so that information pertaining to a particular user can be persisted across multiple requests. This is generally achieved using the concept of cookies, using JSESSIONID as a unique identifier for a given session. We can specify the timeout for the session in web.xml:

<session-config>
    <session-timeout>10</session-timeout>
</session-config>

This means if our session has been idle for 10 minutes, the server will discard it. Any subsequent request would create a new session.

4. How do Servlets Share Data

There're various ways in which servlets can share data, based on the required scope.

As we saw in the earlier sections, different objects have different lifetimes. HttpServletRequest and HttpServletResponse objects only live between one servlet call. HttpSession lives as long as it's active and hasn't timed out.

ServletContext‘s lifespan is the longest. It's born with the web application and gets destroyed only when the application itself shuts down. Since servlet, filter, and listener instances are tied to the context, they also live as long as the web application is up and running.

Consequently, if our requirement is to share data between all servlets, let's say if we want to count the number of visitors to our site, then we should put the variable in the ServletContext. If we need to share data within a session, then we'd save it in the session scope. A user's name would be an example in this case.

Lastly, there's the request scope pertaining to data for a single request, such as the request payload.

5. Handling Multithreading

Multiple HttpServletRequest objects share servlets among each other such that each request operates with its own thread of the servlet instance.

What that effectively means in terms of thread-safety is that we should not assign a request or session scoped data as an instance variable of the servlet.

For example, let's consider this snippet:

public class ExampleThree extends HttpServlet {
    
    private String instanceMessage;
    @Override
    protected void doGet(HttpServletRequest request, HttpServletResponse response) 
      throws ServletException, IOException {
        String message = request.getParameter("message");
        instanceMessage = request.getParameter("message");
        request.setAttribute("text", message);
        request.setAttribute("unsafeText", instanceMessage);
        request.getRequestDispatcher("/jsp/ExampleThree.jsp").forward(request, response);
    }
}

In this case, all requests in the session share instanceMessage, whereas message is unique to a given request object. Consequently, in the case of concurrent requests, the data in instanceMessage could be inconsistent.

6. Conclusion

In this tutorial, we looked at some concepts around servlets, their containers, and a few essential objects they revolve around. We also saw how servlets share data and how multi-threading affects them.

As always, source code is available over on GitHub.

The post Introduction to Servlets and Servlet Containers first appeared on Baeldung.

        

Running Spring Boot with PostgreSQL in Docker Compose

$
0
0

1. Introduction

In this tutorial, we want to run a Spring Boot application with the popular open-source database PostgreSQL. In a previous article, we looked at Docker Compose to handle multiple containers at once. So instead of installing PostgreSQL as a separate application, we'll use Docker Compose to run Spring Boot and PostgreSQL.

2. Creating the Spring Boot Project

Let’s go to the Spring Initializer and create our Spring Boot project. We’ll add the PostgreSQL Driver and Spring Data JPA modules. After we download the resulting ZIP file and extract it to a folder, we can run our new application:

./mvnw spring-boot:run

The application fails because it can't connect to the database:

***************************
APPLICATION FAILED TO START
***************************
Description:
Failed to configure a DataSource: 'url' attribute is not specified and no embedded datasource could be configured.
Reason: Failed to determine a suitable driver class

3. Dockerfile

Before we can start PostgreSQL with Docker Compose, we need to turn our Spring Boot application into a Docker image. The first step is to package the application as a JAR file:

./mvnw clean package -DskipTests

Here, we first clean-up our previous builds before packaging the application. In addition, we skip the tests because they fail without PostgreSQL.

We now have an application JAR file in the target directory. That file has the project name and version number in its name and ends with -SNAPSHOT.jar. So its name could be docker-spring-boot-postgres-0.0.1-SNAPSHOT.jar.

Let's make the new src/main/docker directory. After that, we copy the application JAR file there:

cp target/docker-spring-boot-postgres-0.0.1-SNAPSHOT.jar src/main/docker

Finally, we create this Dockerfile in that same directory:

FROM adoptopenjdk:11-jre-hotspot
ARG JAR_FILE=*.jar
COPY ${JAR_FILE} application.jar
ENTRYPOINT ["java", "-jar", "application.jar"]

This file describes how Docker should run our Spring Boot application. It uses Java 11 from AdoptOpenJDK and copies the application JAR file to application.jar. It then runs that JAR file to start our Spring Boot application.

4. Docker Compose File

Now let's write our Docker Compose file, docker-compose.yml, and save it in src/main/docker:

version: '2'
services:
  app:
    image: 'docker-spring-boot-postgres:latest'
    build:
      context: .
    container_name: app
    depends_on:
      - db
    environment:
      - SPRING_DATASOURCE_URL=jdbc:postgresql://db:5432/compose-postgres
      - SPRING_DATASOURCE_USERNAME=compose-postgres
      - SPRING_DATASOURCE_PASSWORD=compose-postgres
      - SPRING_JPA_HIBERNATE_DDL_AUTO=update
          
  db:
    image: 'postgres:13.1-alpine'
    container_name: db
    environment:
      - POSTGRES_USER=compose-postgres
      - POSTGRES_PASSWORD=compose-postgres

Our application's name is app. It's the first of two services (lines 4-15):

  • The Spring Boot Docker image has the name docker-spring-boot-postgres:latest (line 5). Docker builds that image from the Dockerfile in the current directory (lines 6-7)
  • The container name is app (line 8). It depends on the db service (line 10). That's why it starts after the db container
  • Our application uses the db PostgreSQL container as the data source (line 12). The database name, the user name, and the password are all compose-postgres (lines 12-14)
  • Hibernate will automatically create or update any database tables needed (line 15)

The PostgreSQL database has the name db and is the second service (lines 17-22):

  • We use PostgreSQL 13.1 (line 18)
  • The container name is db (line 19)
  • The user name and password are both compose-postgres (lines 21-22)

5. Running with Docker Compose

Let's run our Spring Boot application and PostgreSQL with Docker Compose:

docker-compose up

Firstly, this will build the Docker Image for our Spring Boot application. Next, it will start a PostgreSQL container. Finally, it will launch our application Docker image. This time, our application runs fine:

Starting DemoApplication v0.0.1-SNAPSHOT using Java 11.0.9 on f94e79a2c9fc with PID 1 (/application.jar started by root in /)
[...]
Finished Spring Data repository scanning in 28 ms. Found 0 JPA repository interfaces.
[...]
Started DemoApplication in 4.751 seconds (JVM running for 6.512)

As we can see, Spring Data found no repository interface. That is correct – we didn't create one yet!

If we want to stop all containers, we need to press [Ctrl-C] first. Then we can stop Docker Compose:

docker-compose down

6. Creating a Customer Entity and Repository

To use the PostgreSQL database in our application, we'll create a simple customer entity:

@Entity
@Table(name = "customer")
public class Customer {
    @Id
    @GeneratedValue
    private long id;
    
    @Column(name = "first_name", nullable = false)
    private String firstName;
    
    @Column(name = "last_name", nullable = false)
    private String lastName;

The Customer has a generated id attribute and two mandatory attributes: firstName and lastName.

Now, we can write the repository interface for this entity:

public interface CustomerRepository extends JpaRepository<Customer, Long> { }

By simply extending JpaRepository, we inherit methods for creating and querying our Customer entity.

Finally, we'll use these methods in our application:

@SpringBootApplication
public class DemoApplication {
    @Autowired 
    private CustomerRepository repository; 
  
    @EventListener(ApplicationReadyEvent.class)
    public void runAfterStartup() {
        List allCustomers = this.repository.findAll(); 
        logger.info("Number of customers: " + allCustomers.size());
 
        Customer newCustomer = new Customer(); 
        newCustomer.setFirstName("John"); 
        newCustomer.setLastName("Doe"); 
        logger.info("Saving new customer..."); 
        this.repository.save(newCustomer); 
 
        allCustomers = this.repository.findAll(); 
        logger.info("Number of customers: " + allCustomers.size());
    }
}
  • We access our Customer repository through dependency injection
  • We query the number of existing customers with the repository — this will be zero
  • Then we create and save a customer
  • When we then query the existing customers again, we expect to find the one we just created

7. Running with Docker Compose Again

To run the updated Spring Boot application, we need to rebuild it first. Therefore, we execute these commands once more in the project root directory:

./mvnw clean package -DskipTests
cp target/docker-spring-boot-postgres-0.0.1-SNAPSHOT.jar src/main/docker

How do we rebuild our Docker image with this updated application JAR file? The best way is to remove the existing Docker image whose name we specified in the docker-compose.yml. This forces Docker to build the image again the next time we start our Docker Compose file:

cd src/main/docker
docker-compose down
docker rmi docker-spring-boot-postgres:latest
docker-compose up

So after stopping our containers, we delete the application Docker image. We then start our Docker Compose file again, which rebuilds the application image.

Here's the application output:

Finished Spring Data repository scanning in 180 ms. Found 1 JPA repository interfaces.
[...]
Number of customers: 0
Saving new customer...
Number of customers: 1

Spring Boot finds our empty customer repository. Therefore, we start with no customer but then successfully create one.

8. Conclusion

In this short tutorial, we started by creating a Spring Boot application for PostgreSQL. Next, we wrote a Docker Compose file to run our application container with a PostgreSQL container.

Finally, we created a customer entity and repository, which allowed us to save a customer to PostgreSQL.

As usual, the source code for this tutorial can be found over on GitHub.

The post Running Spring Boot with PostgreSQL in Docker Compose first appeared on Baeldung.

        

Java Weekly, Issue 362

$
0
0

1. Spring and Java

>> Metaspace in OpenJDK 16 [lkorinth.github.io]

More efficient Metaspace in JDK 16: reduced fragmentation and uncommitting the unused memory.

>> JEP proposed to target JDK 16: Warnings for Value-Based Classes [openjdk.java.net]

One step closer to value types: considering primitive wrappers as value types and deprecating their constructors!

>> Cloud Native Buildpacks with Paketo.io & layered jars for Spring Boot [blog.codecentric.de]

Goodbye Dockerfiles: transforming source codes to Docker images without Dockerfiles with the added benefits of layered Jars.

Also worth reading:

Webinars and presentations:

Time to upgrade:

2. Technical

>> HATEOAS without links [mscharhag.com]

Discovering the next steps on the client-side: decoupling client and servers of a REST API using HATEOAS.

Also worth reading:

3. Musings

>> Remote work challenges as a team lead [arnoldgalovics.com]

On being an effective remote leader: embracing async communications, better meetings, and team building.

Also worth reading:

4. Comics

And my favorite Dilberts of the week:

>> Boss Hires Stalker [dilbert.com]

>> Zoom Happy Hour [dilbert.com]

5. Pick of the Week

>> How to Get Lucky: Focus On The Fat Tails [taylorpearson.me]

The post Java Weekly, Issue 362 first appeared on Baeldung.

        

Light-Weight Concurrency in Java and Kotlin

$
0
0

1. Introduction

In this tutorial, we'll explore the basic concepts of concurrency and how different programming languages address them, particularly Java and Kotlin.

We'll focus primarily on the light-weight concurrency models and compare coroutines in Kotlin with the upcoming proposals in Java as part of Project Loom.

2. Basics of Concurrency

Concurrency is the ability to decompose a program into components that are order-independent or partially ordered. The objective here is to have multiple independent processes working together without affecting the outcome.

Within the operating system kernel, we refer to an instance of a program as a process. The kernel isolates processes by assigning them different address spaces for security and fault tolerance. Since each process has its own address space, open file handles, etc., they are quite expensive to create.

Moreover, since processes cannot access each other's memory, inter-process communication becomes non-trivial.

This is where kernel-level threads bring relief for concurrent programming:

Threads are separate lines of execution within a process. A process can typically have multiple threads. While threads share the same file handles and address spaces, they maintain their own programming stacks. This makes inter-thread communication much easier.

The operating system kernel supports and manages the kernel-level threads directly. The kernel does provide system calls to create and manage these threads from outside. However, the kernel has full control of these threads, including their scheduling. This makes kernel-level threads slow and inefficient, resulting in costly thread operations.

On the other hand, we also have user-level threads that are supported in the user-space, part of the system memory allocated to the running applications:

There are various models that map user-level threads to kernel-level threads like one-to-one or many-to-one. But a runtime system like a virtual machine directly manages user-level threads.

The kernel isn't aware of user-level threads. Hence thread operations on user-level threads are much faster. Of course, this requires coordination between the user-level thread scheduler and the kernel.

3. Concurrency in Programming Languages

We discussed broadly the concurrency primitives that the operating systems provide us. But, how do concurrency abstractions available in different programming languages make use of them? While a detailed analysis is beyond this tutorial's scope, we'll discuss some of the popular patterns here.

Most modern programming languages support concurrent programming and provide one or more primitives to work with. For instance, Java has the first-class support for concurrency through an abstraction called the Thread class. This provides a system-independent definition for a thread in Java. However, under-the-hood, Java maps every thread to the kernel level thread through system calls.

As we've already seen, while kernel threads are easier to program with, they are quite bulky and inefficient. The alternative, in fact, is to use the user-level threads. Many programming languages support the concept of light-weight threads natively, while there are several external libraries to enable this as well.

The fundamental approach is to handle the scheduling of these light-weight threads within the execution environment. The scheduling here is cooperative rather than preemptive, which makes it much more efficient. Also, as we manage these threads in the user-space, we can multiplex them on just a few kernel threads, reducing the kernel threads' overall cost.

Different programming languages have different names for them. For instance, we have coroutines in Kotlin, goroutines in Golang, processes in Erlang, and threads in Haskell, to name a few. Although there's no native support for them in Java, this is in the active proposal under Project Loom. We'll examine some of them later in this tutorial.

4. Additional Approaches Towards Concurrency

The concurrency models discussed so far have a commonality that we can mostly reason about the program's flow in a synchronous manner. Even though they provide asynchronicity, fundamental primitives like threads, or coroutines, abstract it mostly.

However, with more explicit asynchronous programming, we break this abstraction and allow parts of the program to run arbitrarily.

For instance, reactive programming sees concurrency with a completely different perspective. It transforms the program flow as a sequence of events that occur asynchronously. Hence, the program code becomes functions that listen to these asynchronous events, process them, and, if necessary, publish new events.

We often depict these graphically as marble diagrams:

More importantly, the thread on which we publish or subscribe to these events is actually not significant in reactive programming. The reactor core process uses a limited number of threads, typically matching the available CPU cores. It executes a function on a free thread from the pool and releases it back.

So, if we can avoid using any blocking code, it can result in a program that executes much more efficiently, even on a single thread. Also, it addresses some of the pain points like call-back hell typically associated with other asynchronous programming styles.

However, it increases the level of difficulty in reading and writing the program, making it difficult to test and maintain.

5. A Case for Structured Concurrency

A typical concurrent application with more than one execution path is difficult to reason about. Part of the problem is that it lacks abstraction. For instance, if we call a function in such an application, we can't guarantee that processing has terminated when the faction terminates. This is because the function may have spawned multiple concurrent execution paths, of which we're completely unaware.

A sequential flow of the program is much easier to read and write. Of course, to support concurrency, this flow needs to branch out. But, it's much simpler to comprehend if all the branches terminate back into the main flow:

So, maintaining the abstraction, we don't really care how the function internally decomposes the program. It's all fine, so far, as all lines of execution terminate with the function. Alternatively, the scopes of concurrent executions are cleanly nested. This is the fundamental premise of structured concurrency. It emphasizes that if control splits into concurrent tasks, they must join up again.

If we see some of the asynchronous programming models like reactive programming, we'll understand that it's difficult to achieve structured concurrency. In fact, concurrent programming has mostly involved arbitrary jumps, even with simpler primitives like threads.

However, we can achieve structured concurrency in Kotlin with a solution like coroutines.

We'll see how, later in this tutorial.

6. Kotlin: How Do They Do It?

Now we've gathered enough background to examine how Kotlin solves the problem of concurrency while keeping most of the issues at bay. Kotlin is an open-source programming language that was started by JetBrains back in 2010. Kotlin targets the JVM along with other platforms like JavaScript, and even Native. Hence, it can produce Java-compatible bytecode.

Kotlin provides the support for light-weight threads in the form of coroutines, which are implemented as a rich librarykotlinx.coroutines. Interestingly, the JVM does not have native support for a light-weight concurrency construct like coroutine — well, at least yet! Nonetheless, Kotlin introduced coroutines as an experimental language feature quite early, and they became official in version 1.3.

We'll see how Kotlin implements coroutines and how we can use them to write concurrent applications with the benefits of structured concurrency.

6.1. What Exactly Is a Coroutine?

Generally speaking, coroutines are parts of a computer program or generalized subroutines that can suspend and resume their execution at any point. It first appeared as a method in assembly languages way back in the 1950s. Coroutines can have several interesting applications.

When we use them for concurrency, they appear to be similar to kernel threads. However, there are subtle differences. For instance, a scheduler manages kernel threads preemptively, while coroutines voluntarily yield control, resulting in cooperative multitasking.

Let's see a general construction of coroutines:

coroutine
    loop
        while some_condition
            some_action
        yield

Here, as we can see, we have a coroutine that performs some action in a loop but cooperatively yields the control on every step instead of blocking. This can help to utilize underlying kernel threads much more efficiently. This is exactly what other asynchronous programming styles like reactive programming do, but without the complexities.

While a coroutine can choose to yield to a specific coroutine, there can also be a controller that schedules multiple coroutines. More interestingly, we can multiplex thousands of coroutines on just a single underlying kernel thread.

But consequently, coroutines don't necessarily provide parallelism, even on a multi-core system.

6.2. Kotlin Coroutine in Action

Kotlin provides many coroutine builders to create a coroutine, like launch, async, and runBlocking. Further, coroutines in Kotlin are always bound to a coroutine scope. The coroutine scope contains the coroutine context and sets the new coroutine scope that is launched by a coroutine builder.

We'll see shortly how to launch a coroutine, but let's first understand suspending functions. Kotlin provides a special keyword called suspend to mark such functions. This allows the compiler to sprinkle some magic into these functions, which we'll see later.

Let's create a suspending function:

suspend fun doSomething() {
    // heavy computation here
}

Barring the use of this keyword, we can see that these are just regular functions. However, there is an important limitation: Suspending functions can only be invoked from within a coroutine or from another suspending function.

So, let's use one of the coroutine builders to launch a coroutine and call our simple but suspending function:

GlobalScope.launch {
    doSomething() // does some heavy computation in the background
    ... do other stuff
}

Here, we're starting a new coroutine with the launch coroutine builder.

6.3. Structured Concurrency with Coroutines

Now, we should avoid launching a coroutine bound to the GlobalScope, unless intended for the right reasons. This is because such coroutines operate on the whole application lifecycle and, more importantly, deviate from the principles of structured concurrency.

To adhere to structured concurrency, we should rather create an application-specific CoroutineScope and use coroutine builder on its instance:

var job = Job()
val coroutineScope = CoroutineScope(Dispatchers.Main + job)
coroutineScope.launch { 
    doSomething() // does some heavy computation in the background 
    ... do other stuff 
}

To create an instance of CoroutineScope, we have to define a Dispatcher, which controls which thread runs a coroutine. The Job here is responsible for the coroutine's lifecycle, cancellation, and parent-child relations.

All the coroutines launched using this CoroutineScope can be simply canceled by canceling this parent Job. This prevents coroutines from leaking unintentionally. This also avoids having side-effects of launching coroutines from a suspending function. Hence, we achieve structured concurrency, which we've discussed before.

6.4. Looking Under the Hood

So, the question now is: How does Kotlin implement coroutines? Broadly speaking, coroutines are implemented in Kotlin as a finite state machine with suspension points and continuations. For those of us uninitiated in this area, this may not make any sense! But we'll try to describe them briefly.

Let's first understand some of the terms we've just introduced. A suspension point is a point in the suspending function at which we want to suspend our execution and resume later. At the same time, a continuation is actually the encapsulation of the state of a function at a suspension point. Basically, a continuation captures the rest of the execution after the suspension point.

Now Kotlin, upon compilation, transforms all suspending functions to add a parameter, which is the continuation object. The compiler will transform the signature of our suspending function from the previous section:

fun doSomething(continuation: Continuation): Any?

This programming style is typical of functional programming and is known as Continuation Passing Style (CPS). Here, the control is passed explicitly in the form of a continuation. This is somewhat similar to the asynchronous programming style where we pass a callback function to get notified. However, with coroutines in Kotlin, the compiler implicitly handles the continuations.

The Kotlin compiler identifies all possible suspension points in a suspending function and creates states with labels for everything delimited by the suspension points. The resulting continuation is nothing but a huge switch statement with these states as labels.

Hence, we can think of continuation as packing this as a finite state machine.

7. Java: What Is the Proposal?

Java has had first-class support for concurrency since the early days of its inception. However, Java does not have native support for what we know as light-weight threads. Although there have been several attempts to build such support outside the core Java, none of them could find enough success.

For the last couple of years, OpenJDK has been working on Project Loom to bridge this gap.

7.1. A Brief History of Concurrency in Java

Since JDK 1.0, the class Thread has provided a core abstraction for concurrency in Java. It was intended to run on all platforms alike, to match the promise “write once, run anywhere”. Unfortunately, some of the target platforms didn't have native support for threads back then. Hence, Java had to implement something called green threads to deliver that promise.

Basically, green threads are the implementation of threads that are managed in the user-space and scheduled by the virtual machine. We've already seen such threads' general definition and discussed how coroutines in Kotlin or goroutines in Golang and similar concepts. Although green threads may vary in terms of the implementation, the basic idea was actually quite similar.

In the initial days, Java struggled to refine the implementation of green threads. It was difficult to scale green threads over multiple processors and hence benefit from parallelism on multi-core systems. To get around this problem and simplify the concurrent programming model, Java decided to abandon green threads in version 1.3.

So, Java decided to map every thread to a separate native kernel thread. Essentially the JVM threads became a thin wrapper around the operating system threads. This simplified the programming model, and Java could leverage the benefits of parallelism with preemptive scheduling of threads by the kernel across multiple cores.

7.2. Problems with the Java Concurrency Model

The concurrency model in Java was actually quite easy to use and has been improved substantially with the introduction of ExecutorService and CompletableFuture. This also worked well for a large period of time. However, the problem is how concurrent applications that were written with this model have to face an unprecedented scale today.

For instance, typical servlet containers are written in the thread-per-request model. But, it's impossible to create as many threads on a system as the number of concurrent requests we expect them to handle. This calls for alternate programming models like event-loop or reactive programming that are inherently non-blocking, but they have their own share of issues.

7.3. Proposals of Project Loom

By now, it should not be difficult for us to guess that perhaps it's time for Java to bring back the support for light-weight threads. This is actually the motivation behind Project Loom. The purpose of this project is to explore and incubate a light-weight concurrency model on the Java platform. The idea is to build support for light-weight threads on top of the JVM threads and fundamentally decouple the JVM threads from the native kernel threads.

The current proposal is to introduce support for some core concurrency related constructs right at the level of JVM. These include virtual threads (previously called fibers), delimited continuation, and tail-call elimination. The current construct of the thread is basically a composition of continuation and scheduler. The idea is to separate these concerns and support virtual threads on top of these building blocks.

As the current JVM thread is just a wrapper over the underlying kernel thread, it relies on the kernel to provide the implementation for both continuation and scheduler. However, by exposing continuation as a construct within the Java platform, it's possible to combine it with a global scheduler. This gives rise to virtual threads as light-weight threads managed entirely within the JVM.

Of course, the idea behind Project Loom is not just to provide a construct like the virtual thread in Java but also to address some of the other issues that arise due to them. For instance, a flexible mechanism to pass data among a large number of virtual threads. A more intuitive way to organize and supervise so many virtual threads, a concept close to structured concurrency. Or managing context-data for so many virtual threads, similar to what we have as thread-local for current threads.

7.4. Understanding Continuations

Let's understand what we actually mean by delimited continuations in the scope of Project Loom. Actually, the basic idea behind a delimited continuation is nothing different from a coroutine that we've already discussed before. Hence, we can see a delimited continuation as a sequential code that can suspend its execution at any point and resume again from the same point.

In Java, however, the proposal is to expose continuations as a public API. The proposed API may look like the following:

class _Continuation {
    public _Continuation(_Scope scope, Runnable target) 
    public boolean run()
    public static _Continuation suspend(_Scope scope, Consumer<_Continuation> ccc)
    public ? getStackTrace()
}

Please note that continuation is a general construct and has nothing specific to virtual threads. Although virtual threads require continuations for implementation, there are other possible uses of continuations as well. For instance, we can use it to implement a generator, which is an iterator that yields after producing a single value.

7.5. Implementation of Virtual Threads

The focus of Project Loom is to provide support for virtual threads as a basic construct. Virtual threads are the higher-level construct that is proposed to provide the capabilities of user-mode threads in Java. Basically, virtual threads should allow us to run an arbitrary code concurrently with the ability to suspend and resume execution.

As we can already guess, continuations will be used to create higher-level constructs like virtual threads. The idea is that the virtual thread will hold a private instance of the continuation class along with other necessary parts:

class _VirtualThread {
    private final _Continuation continuation;
    private final Executor scheduler;
    private volatile State state;
    private final Runnable task;
​
    private enum State { NEW, LEASED, RUNNABLE, PAUSED, DONE; }
  
    public _VirtualThread(Runnable target, Executor scheduler) {
        .....
    }
  
    public void start() {
        .....
    }
  
    public static void park() {
        _Continuation.suspend(_FIBER_SCOPE, null);
    }
  
    public void unpark() {
        .....
    }
}

Above is a simple representation of how we can compose a virtual thread with low-level primitive, like continuations. Also, note that schedulers are an essential part of implementing the virtual thread. However, the initial default global scheduler for virtual threads will be the ForkJoinPool that already exists in Java and implements a work-stealing algorithm.

More importantly, the proposal is to keep the API of virtual threads very close to that of the current heavy-weight threads. The heavy-weight thread as it exists today will continue to exist. So the conformity of the API that heavy-weight or the new light-weight threads support will lead to a better user experience.

7.6. A Sneak Peek into the Current State

Project Loom has been in progress for a couple of years now, and some parts of the proposal may be available as part of Java 16 in 2021. However, early-access builds are available for some time to experiment with the new features and provide feedback.

So, first, let's see how working with heavy-weight threads, or the threads as we know them currently, will change:

Runnable printThread = () -> System.out.println(Thread.currentThread());
ThreadFactory kernelThreadFactory = Thread.builder().factory();
Thread kernelThread = kernelThreadFactory.newThread(printThread);
kernelThread.start();

As we can see, we have a new interface called Thread.Builder, which is a mutable builder for Thread or ThreadFactory. This is to facilitate creating a kernel thread, as we're doing here, or a virtual thread. Everything else is quite similar to what exists today.

So, let's see how to create and use a virtual thread instead:

Runnable printThread = () -> System.out.println(Thread.currentThread());
ThreadFactory virtualThreadFactory = Thread.builder().virtual().factory();
Thread virtualThread = virtualThreadFactory.newThread(printThread);
virtualThread.start();

Apart from the fact that there's a different thread factory to create virtual threads, there is actually no difference! This is because the current implementation of virtual threads does not introduce a new class but just a new implementation of the Thread class.

Apart from the fact that this new implementation of Thread differs in scheduling, there are other aspects that will not work the same for them. For instance, the behavior and implications of some of the existing constructs like ThreadGroup and ThreadLocal will be different for the virtual threads.

8. How Are Java Virtual Threads Different from Kotlin Coroutines?

We've discussed in detail the support for the light-weight concurrency model that Kotlin has in terms of coroutines and the model Java is proposing to bring as virtual threads.

The obvious question is, how do they compare against each other, and is it possible to benefit from both of them when they target the same JVM. In this section, we'll explore some of the important aspects like continuations and scheduling.

8.1. Stackful vs. Stackless Continuations

Since continuations form the basis of any form of user-mode thread implementation, let's begin by examining their implementation in Kotlin and how they are different from the proposal in Java. Broadly speaking about design choice, Kotlin coroutines are stackless, whereas continuations in Java are proposed to be stackful.

As the name suggests, stackful continuations or coroutines maintain their own function call stack. A stack here is a contiguous block of memory that is needed to store the local variables and the function arguments. On the contrary, stackless coroutines do not maintain their stack and rely on the caller. This makes them strongly connected to the caller.

As an immediate fallout, stackless coroutines can suspend themselves only from the top-level function. So, all functions called from the coroutine must finish before suspending the coroutine. In comparison, a stackful continuation or coroutine can suspend at any nested depth of the call stack. So, stackful coroutines are more powerful and general-purpose than stackless coroutines.

However, since stackless coroutines have a lower memory footprint than stackful coroutines, they prove to be more efficient. This is because context switching between stackless coroutines comes out to be less expensive. Moreover, the compiler locally handles the code transformations for stackless coroutines with very little support from the runtime.

8.2. Preemptive vs. Cooperative Scheduling

Apart from continuations, another important part of the implementation of a light-weight thread is scheduling. We've seen how the operating system scheduler schedules the kernel threads preemptively. This is, in fact, one of the reasons why kernel threads prove to be inefficient. So, typically, the approach for scheduling light-weight threads is more structured than arbitrary.

As we've seen earlier, the scheduling in Kotlin coroutines is cooperative where coroutines voluntarily yield the control at logical points. For instance, we can decide to wrap a computationally heavy or blocking operation in a suspending function. When we call such functions from a coroutine or another suspending function, these become natural suspension points.

However, the current proposal in Java is to keep the scheduling preemptive rather than cooperative. Hence, it's not possible to define suspension points in Java virtual threads. So, does that mean it will carry the burden of kernel scheduler? Not really. Note that the kernel threads are preempted arbitrarily, based on the notion of time-slice.

However, the proposal for virtual thread scheduler in Java is to preempt them when they block on I/O or synchronization.

Regardless of how they're scheduled, light-weight threads are finally executed on the underlying kernel threads. In the case of Kotlin coroutines, coroutine context includes a coroutine dispatcher. The coroutine dispatcher decides which kernel thread the coroutine uses for its execution.

On the other hand, the Java virtual thread scheduler maintains a pool of kernel threads as workers and mounts a runnable virtual thread on one of the available workers.

9. Conclusion

In this tutorial, we understood the basic concepts of concurrency and how light-weight concurrency differs from heavy-weight concurrency. We also touched upon how concurrency is generally approached in programming languages and what we mean by structured concurrency.

Further, we understood how light-weight concurrency is supported in Kotlin as coroutines and how Java is proposing to introduce virtual threads in that regard. We discussed these constructs in some detail and then touched upon how their implementations differ from each other.

The post Light-Weight Concurrency in Java and Kotlin first appeared on Baeldung.

        

Spring Boot With Spring Batch

$
0
0

1. Overview

Spring Batch is a powerful framework for developing robust batch applications. In our previous tutorial, we introduced Spring Batch.

In this tutorial, we'll build on the previous one and learn how to set up and create a basic batch-driven application using Spring Boot.

2. Maven Dependencies

First, let’s add the spring-boot-starter-batch to our pom.xml:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-batch</artifactId>
    <version>2.4.0.RELEASE</version>
</dependency>

We'll also add the org.hsqldb dependency, which is available from Maven Central as well:

<dependency>
    <groupId>org.hsqldb</groupId>
    <artifactId>hsqldb</artifactId>
    <version>2.5.1</version>
    <scope>runtime</scope>
</dependency>

3. Defining a Simple Spring Batch Job

We're going to build a job that imports a coffee list from a CSV file, transforms it using a custom processor, and stores the final results in an in-memory database.

3.1. Getting Started

Let's start by defining our application entry point:

@SpringBootApplication
public class SpringBootBatchProcessingApplication {
    public static void main(String[] args) {
        SpringApplication.run(SpringBootBatchProcessingApplication.class, args);
    }
}

As we can see, this is a standard Spring Boot application. As we want to use default configuration values where possible, we're going to use a very light set of application configuration properties.

We'll define these properties in our src/main/resources/application.properties file:

file.input=coffee-list.csv

This property contains the location of our input coffee list. Each line contains the brand, origin, and some characteristics of our coffee:

Blue Mountain,Jamaica,Fruity
Lavazza,Colombia,Strong
Folgers,America,Smokey

As we're going to see, this is a flat CSV file, which means Spring can handle it without any special customization.

Next, we'll add a SQL script schema-all.sql to create our coffee table to store the data:

DROP TABLE coffee IF EXISTS;
CREATE TABLE coffee  (
    coffee_id BIGINT IDENTITY NOT NULL PRIMARY KEY,
    brand VARCHAR(20),
    origin VARCHAR(20),
    characteristics VARCHAR(30)
);

Conveniently Spring Boot will run this script automatically during startup.

3.2. Coffee Domain Class

Subsequently, we'll need a simple domain class to hold our coffee items:

public class Coffee {
    private String brand;
    private String origin;
    private String characteristics;
    public Coffee(String brand, String origin, String characteristics) {
        this.brand = brand;
        this.origin = origin;
        this.characteristics = characteristics;
    }
    // getters and setters
}

As previously mentioned, our Coffee object contains three properties:

  • A brand
  • An origin
  • Some additional characteristics

4. Job Configuration

Now, on to the key component, our job configuration. We'll go step by step, building up our configuration and explaining each part along the way:

@Configuration
@EnableBatchProcessing
public class BatchConfiguration {
    @Autowired
    public JobBuilderFactory jobBuilderFactory;
    @Autowired
    public StepBuilderFactory stepBuilderFactory;
    
    @Value("${file.input}")
    private String fileInput;
    
    // ...
}

Firstly, we start with a standard Spring @Configuration class. Next, we add a @EnableBatchProcessing annotation to our class. Notably, this gives us access to many useful beans that support jobs and will save us a lot of leg work.

Furthermore, using this annotation also provides us with access to two useful factories that we'll use later when building our job configuration and jobs steps.

For the last part of our initial configuration, we include a reference to the file.input property we declared previously.

4.1. A Reader and Writer for Our Job

Now, we can go ahead and define a reader bean in our configuration:

@Bean
public FlatFileItemReader reader() {
    return new FlatFileItemReaderBuilder().name("coffeeItemReader")
      .resource(new ClassPathResource(fileInput))
      .delimited()
      .names(new String[] { "brand", "origin", "characteristics" })
      .fieldSetMapper(new BeanWrapperFieldSetMapper() {{
          setTargetType(Coffee.class);
      }})
      .build();
}

In short, our reader bean defined above looks for a file called coffee-list.csv and parses each line item into a Coffee object.

Likewise, we define a writer bean:

@Bean
public JdbcBatchItemWriter writer(DataSource dataSource) {
    return new JdbcBatchItemWriterBuilder()
      .itemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<>())
      .sql("INSERT INTO coffee (brand, origin, characteristics) VALUES (:brand, :origin, :characteristics)")
      .dataSource(dataSource)
      .build();
}

This time around, we include the SQL statement needed to insert a single coffee item into our database, driven by the Java bean properties of our Coffee object. Handily the dataSource is automatically created by @EnableBatchProcessing annotation.

4.2. Putting Our Job Together

Lastly, we need to add the actual job steps and configuration:

@Bean
public Job importUserJob(JobCompletionNotificationListener listener, Step step1) {
    return jobBuilderFactory.get("importUserJob")
      .incrementer(new RunIdIncrementer())
      .listener(listener)
      .flow(step1)
      .end()
      .build();
}
@Bean
public Step step1(JdbcBatchItemWriter writer) {
    return stepBuilderFactory.get("step1")
      .<Coffee, Coffee> chunk(10)
      .reader(reader())
      .processor(processor())
      .writer(writer)
      .build();
}
@Bean
public CoffeeItemProcessor processor() {
    return new CoffeeItemProcessor();
}

As we can see, our job is relatively simple and consists of one step defined in the step1 method.

Let's take a look at what this step is doing:

  • First, we configure our step so that it will write up to ten records at a time using the chunk(10) declaration
  • Then, we read in the coffee data using our reader bean, which we set using the reader method
  • Next, we pass each of our coffee items to a custom processor where we apply some custom business logic
  • Finally, we write each coffee item to the database using the writer we saw previously

On the other hand, our importUserJob contains our job definition, which contains an id using the build-in RunIdIncrementer class. We also set a JobCompletionNotificationListener, which we use to get notified when the job completes.

To complete our job configuration, we list each step (though this job has only one step). We now have a perfectly configured job!

5. A Custom Coffee Processor

Let's take a look in detail at the custom processor we defined previously in our job configuration:

public class CoffeeItemProcessor implements ItemProcessor<Coffee, Coffee> {
    private static final Logger LOGGER = LoggerFactory.getLogger(CoffeeItemProcessor.class);
    @Override
    public Coffee process(final Coffee coffee) throws Exception {
        String brand = coffee.getBrand().toUpperCase();
        String origin = coffee.getOrigin().toUpperCase();
        String chracteristics = coffee.getCharacteristics().toUpperCase();
        Coffee transformedCoffee = new Coffee(brand, origin, chracteristics);
        LOGGER.info("Converting ( {} ) into ( {} )", coffee, transformedCoffee);
        return transformedCoffee;
    }
}

Of particular interest, the ItemProcessor interface provides us with a mechanism to apply some specific business logic during our job execution.

To keep things simple, we define our CoffeeItemProcessor, which takes an input Coffee object and transforms each of the properties to uppercase.

6. Job Completion

Additionally, we're also going to write a JobCompletionNotificationListener to provide some feedback when our job finishes:

@Override
public void afterJob(JobExecution jobExecution) {
    if (jobExecution.getStatus() == BatchStatus.COMPLETED) {
        LOGGER.info("!!! JOB FINISHED! Time to verify the results");
        String query = "SELECT brand, origin, characteristics FROM coffee";
        jdbcTemplate.query(query, (rs, row) -> new Coffee(rs.getString(1), rs.getString(2), rs.getString(3)))
          .forEach(coffee -> LOGGER.info("Found < {} > in the database.", coffee));
    }
}

In the above example, we override the afterJob method and check the job completed successfully. Moreover, we run a trivial query to check that each coffee item was stored in the database successfully.

7. Running Our Job

Now that we have everything in place to run our job, here comes the fun part. Let's go ahead and run our job:

...
17:41:16.336 [main] INFO  c.b.b.JobCompletionNotificationListener -
  !!! JOB FINISHED! Time to verify the results
17:41:16.336 [main] INFO  c.b.b.JobCompletionNotificationListener -
  Found < Coffee [brand=BLUE MOUNTAIN, origin=JAMAICA, characteristics=FRUITY] > in the database.
17:41:16.337 [main] INFO  c.b.b.JobCompletionNotificationListener -
  Found < Coffee [brand=LAVAZZA, origin=COLOMBIA, characteristics=STRONG] > in the database.
17:41:16.337 [main] INFO  c.b.b.JobCompletionNotificationListener -
  Found < Coffee [brand=FOLGERS, origin=AMERICA, characteristics=SMOKEY] > in the database.
...

As we can see, our job ran successfully, and each coffee item was stored in the database as expected.

8. Conclusion

In this article, we've learned how to create a simple Spring Batch job using Spring Boot. First, we started by defining some basic configuration.

Then, we saw how to add a file reader and database writer. Finally, we took a look at how to apply some custom processing and check our job was executed successfully.

As always, the full source code of the article is available over on GitHub.

The post Spring Boot With Spring Batch first appeared on Baeldung.

        

A Guide to MultipleBagFetchException in Hibernate

$
0
0

1. Overview

In this tutorial, we'll talk about the MultipleBagFetchException. We'll begin with the necessary terms to understand, and then we'll explore some workarounds until we reach the ideal solution.

We'll create a simple music app's domain to demonstrate each of the solutions.

2. What is a Bag in Hibernate?

A Bag, similar to a List, is a collection that can contain duplicate elements. However, it is not in order. Moreover, a Bag is a Hibernate term and isn't part of the Java Collections Framework.

Given the earlier definition, it's worth highlighting that both List and Bag uses java.util.List. Although in Hibernate, both are treated differently. To differentiate a Bag from a List, let's look at it in actual code.

A Bag:

// @ any collection mapping annotation
private List<T> collection;

A List:

// @ any collection mapping annotation
@OrderColumn(name = "position")
private List<T> collection;

3. Cause of MultipleBagFetchException

Fetching two or more Bags at the same time on an Entity could form a Cartesian Product. Since a Bag doesn't have an order, Hibernate would not be able to map the right columns to the right entities. Hence, in this case, it throws a MultipleBagFetchException.

Let's have some concrete examples that lead to MultipleBagFetchException.

For the first example, let's try to create a simple entity that has 2 bags and both with eager fetch type. An Artist might be a good example. It can have a collection of songs and offers.

Given that, let's create the Artist entity:

@Entity
class Artist {
    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    private Long id;
    private String name;
    @OneToMany(mappedBy = "artist", fetch = FetchType.EAGER)
    private List<Song> songs;
    @OneToMany(mappedBy = "artist", fetch = FetchType.EAGER)
    private List<Offer> offers;
    // constructor, equals, hashCode
}

If we try to run a test, we'll encounter a MultipleBagFetchException immediately, and it won't be able to build Hibernate SessionFactory. Having that said, let's not do this.

Instead, let's convert one or both of the collections' fetch type to lazy:

@OneToMany(mappedBy = "artist")
private List<Song> songs;
@OneToMany(mappedBy = "artist")
private List<Offer> offers;

Now, we'll be able to create and run a test. Although, if we try to fetch both of these bag collections at the same time, it would still lead to MultipleBagFetchException.

4. Simulate a MultipleBagFetchException

In the previous section, we've seen the causes of MultipleBagFetchException. Here, let's verify those claims by creating an integration test.

For simplicity, let's use the Artist entity that we've previously created.

Now, let's create the integration test, and let's try to fetch both songs and offers at the same time using JPQL:

@Test
public void whenFetchingMoreThanOneBag_thenThrowAnException() {
    IllegalArgumentException exception =
      assertThrows(IllegalArgumentException.class, () -> {
        String jpql = "SELECT artist FROM Artist artist "
          + "JOIN FETCH artist.songs "
          + "JOIN FETCH artist.offers ";
        entityManager.createQuery(jpql);
    });
    final String expectedMessagePart = "MultipleBagFetchException";
    final String actualMessage = exception.getMessage();
    assertTrue(actualMessage.contains(expectedMessagePart));
}

From the assertion, we have encountered an IllegalArgumentException, which has a root cause of MultipleBagFetchException.

5. Domain Model

Before proceeding to possible solutions, let's look at the necessary domain models, which we'll use as a reference later on.

Suppose we're dealing with a music app's domain. Given that, let's narrow our focus toward certain entities: Album, Artist, and User. 

We've already seen the Artist entity, so let's proceed with the other two entities instead.

First, let's look at the Album entity:

@Entity
class Album {
    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    private Long id;
    private String name;
    @OneToMany(mappedBy = "album")
    private List<Song> songs;
    @ManyToMany(mappedBy = "followingAlbums")
    private Set<Follower> followers;
    // constructor, equals, hashCode
}

An Album has a collection of songs, and at the same time, could have a set of followers. 

Next, here's the User entity:

@Entity
class User {
    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    private Long id;
    private String name;
    @OneToMany(mappedBy = "createdBy", cascade = CascadeType.PERSIST)
    private List<Playlist> playlists;
    @OneToMany(mappedBy = "user", cascade = CascadeType.PERSIST)
    @OrderColumn(name = "arrangement_index")
    private List<FavoriteSong> favoriteSongs;
    
    // constructor, equals, hashCode
}

A User can create many playlists. Additionally, a User has a separate List for favoriteSongs wherein its order is based on the arrangement index.

6. Workaround: Using a Set in a single JPQL query

Before anything else, let's emphasize that this approach would generate a cartesian product, which makes this a mere workaround. It's because we'll be fetching two collections simultaneously in a single JPQL query. In contrast, there's nothing wrong with using a Set. It is the appropriate choice if we don't need our collection to have an order or any duplicated elements.

To demonstrate this approach, let's reference the Album entity from our domain model. 

An Album entity has two collections: songs and followers. The collection of songs is of type bag. However, for the followers, we're using a Set. Having that said, we won't encounter a MultipleBagFetchException even if we try to fetch both collections at the same time.

Using an integration test, let's try to retrieve an Album by its id while fetching both of its collections in a single JPQL query:

@Test
public void whenFetchingOneBagAndSet_thenRetrieveSuccess() {
    String jpql = "SELECT DISTINCT album FROM Album album "
      + "LEFT JOIN FETCH album.songs "
      + "LEFT JOIN FETCH album.followers "
      + "WHERE album.id = 1";
    Query query = entityManager.createQuery(jpql)
      .setHint(QueryHints.HINT_PASS_DISTINCT_THROUGH, false);
    assertEquals(1, query.getResultList().size());
}

As we can see, we have successfully retrieved an Album. It's because only the list of songs is a Bag. On the other hand, the collection of followers is a Set.

On a side note, it's worth highlighting that we're making use of QueryHints.HINT_PASS_DISTINCT_THROUGH. Since we're using an entity JPQL query, it prevents the DISTINCT keyword from being included in the actual SQL query. Thus, we'll use this query hint for the remaining approaches as well. 

7. Workaround: Using a List in a single JPQL query

Similar to the previous section, this would also generate a cartesian product, which could lead to performance issues. Again, there's nothing wrong with using a List, Set, or Bag for the data type. The purpose of this section is to demonstrate further that Hibernate can fetch collections simultaneously given there's no more than one of type Bag.

For this approach, let's use the User entity from our domain model.

As mentioned earlier, a User has two collections: playlists and favoriteSongs. The playlists have no defined order, making it a bag collection. However, for the List of favoriteSongs, its order depends on how the User arranges it. If we look closely at the FavoriteSong entity, the arrangementIndex property made it possible to do so.

Again, using a single JPQL query, let's try to verify if we'll be able to retrieve all the users while fetching both collections of playlists and favoriteSongs at the same time.

To demonstrate, let's create an integration test:

@Test
public void whenFetchingOneBagAndOneList_thenRetrieveSuccess() {
    String jpql = "SELECT DISTINCT user FROM User user "
      + "LEFT JOIN FETCH user.playlists "
      + "LEFT JOIN FETCH user.favoriteSongs ";
    List<User> users = entityManager.createQuery(jpql, User.class)
      .setHint(QueryHints.HINT_PASS_DISTINCT_THROUGH, false)
      .getResultList();
    assertEquals(3, users.size());
}

From the assertion, we can see that we have successfully retrieved all users. Moreover, we didn't encounter a MultipleBagFetchException. It's because even though we're fetching two collections at the same time, only the playlists is a bag collection.

8. Ideal Solution: Using Multiple Queries

We've seen from the previous workarounds the use of a single JPQL query for the simultaneous retrieval of collections. Unfortunately, it generates a cartesian product. We know that it's not ideal. So here, let's solve the MultipleBagFetchException without having to sacrifice performance.

Suppose we're dealing with an entity that has more than one bag collection. In our case, it is the Artist entity. It has two bag collections: songs and offers.

Given this situation, we won't even be able to fetch both collections at the same time using a single JPQL query. Doing so will lead to a MultipleBagFetchException. Instead, let's split it into two JPQL queries.

With this approach, we're expecting to fetch both bag collections successfully, one at a time.

Again, for the last time, let's quickly create an integration test for the retrieval of all artists:

@Test
public void whenUsingMultipleQueries_thenRetrieveSuccess() {
    String jpql = "SELECT DISTINCT artist FROM Artist artist "
      + "LEFT JOIN FETCH artist.songs ";
    List<Artist> artists = entityManager.createQuery(jpql, Artist.class)
      .setHint(QueryHints.HINT_PASS_DISTINCT_THROUGH, false)
      .getResultList();
    jpql = "SELECT DISTINCT artist FROM Artist artist "
      + "LEFT JOIN FETCH artist.offers "
      + "WHERE artist IN :artists ";
    artists = entityManager.createQuery(jpql, Artist.class)
      .setParameter("artists", artists)
      .setHint(QueryHints.HINT_PASS_DISTINCT_THROUGH, false)
      .getResultList();
    assertEquals(2, artists.size());
}

From the test, we first retrieved all artists while fetching its collection of songs.

Then, we created another query to fetch the artists' offers.

Using this approach, we avoided the MultipleBagFetchException as well as the formation of a cartesian product.

9. Conclusion

In this article, we've explored MultipleBagFetchException in detail. We discussed the necessary vocabulary and the causes of this exception. We then simulated it. After that, we talked about a simple music app's domain to have different scenarios for each of our workarounds and ideal solution. Lastly, we set up several integration tests to verify each of the approaches.

As always, the full source code of the article is available over on GitHub.

The post A Guide to MultipleBagFetchException in Hibernate first appeared on Baeldung.

        

Get a Filename Without the Extension in Java

$
0
0

1. Overview

When we work with files in Java, we often need to handle filenames. For example, sometimes we want to get the name without the extension from a given filename. In other words, we want to remove the extension of a filename.

In this tutorial, we'll discuss the generic way to remove the extension from a filename.

2. Scenarios of Removing the Extension From a Filename

When we take a first look at it, we may think that removing the extension from a filename is a pretty easy problem.

However, if we take a closer look at the problem, it could be more complicated than we thought.

First of all, let's have a look at the types a filename can be:

  • Without any extension, for example, “baeldung”
  • With a single extension, this is the most usual case, for example, “baeldung.txt
  • With multiple extensions, like “baeldung.tar.gz
  • Dotfile without an extension, such as “.baeldung
  • Dotfile with a single extension, for instance, “.baeldung.conf
  • Dotfile with multiple extensions, for example, “.baeldung.conf.bak

Next, we'll list expecting results of the examples above after removing the extension(s):

  • baeldung“: The filename doesn't have an extension. Therefore, the filename should not be changed, and we should get “baeldung
  • baeldung.txt“: This is a straightforward case. The correct result is “baeldung
  • baeldung.tar.gz“: This filename contains two extensions. If we want to remove only one extension, “baeldung.tar” should be the result. But if we want to remove all extensions from the filename, “baeldung” is the correct result
  • .baeldung“: Since this filename doesn't have any extension either, the filename shouldn't be changed either. Thus, we're expecting to see “.baeldung” in the result
  • .baeldung.conf“: The result should be “.baeldung
  • .baeldung.conf.bak“: The result should be “.baeldung.conf” if we only want to remove one extension. Otherwise, “.baeldung” is the expected output if we'll remove all extensions

In this tutorial, we'll test if the utility methods provided by Guava and Apache Commons IO can handle all the cases listed above.

Further, we'll also discuss a generic way to solve the problem of removing the extension (or extensions) from a given filename.

3. Testing the Guava Library

Since version 14.0, Guava has introduced the Files.getNameWithoutExtension() method. It allows us to remove the extension from the given filename easily.

To use the utility method, we need to add the Guava library into our classpath. For example, if we use Maven as the build tool, we can add the Guava dependency to our pom.xml file:

<dependency>
    <groupId>com.google.guava</groupId>
    <artifactId>guava</artifactId>
    <version>30.0-jre</version>
</dependency>

First, let's have a look at the implementation of this method:

public static String getNameWithoutExtension(String file) {
   ...
   int dotIndex = fileName.lastIndexOf('.');
   return (dotIndex == -1) ? fileName : fileName.substring(0, dotIndex);
 }

The implementation is pretty straightforward. If the filename contains dots, the method cuts from the last dot to the end of the filename. Otherwise, if the filename doesn't contain a dot, the original filename will be returned without any change.

Therefore, Guava's getNameWithoutExtension() method won't work for dotfiles without an extension. Let's write a test to prove that:

@Test
public void givenDotFileWithoutExt_whenCallGuavaMethod_thenCannotGetDesiredResult() {
    //negative assertion
    assertNotEquals(".baeldung", Files.getNameWithoutExtension(".baeldung"));
}

When we handle a filename with multiple extensions, this method doesn't provide an option to remove all extensions from the filename:

@Test
public void givenFileWithoutMultipleExt_whenCallGuavaMethod_thenCannotRemoveAllExtensions() {
    //negative assertion
    assertNotEquals("baeldung", Files.getNameWithoutExtension("baeldung.tar.gz"));
}

4. Testing the Apache Commons IO Library

Like the Guava library, the popular Apache Commons IO library provides a removeExtension() method in the FilenameUtils class to quickly remove the filename's extension.

Before we have a look at this method, let's add the Apache Commons IO dependency into our pom.xml:

<dependency>
    <groupId>commons-io</groupId>
    <artifactId>commons-io</artifactId>
    <version>2.8.0</version>
</dependency>

The implementation is similar to Guava's getNameWithoutExtension() method:

public static String removeExtension(final String filename) {
    ...
    final int index = indexOfExtension(filename); //used the String.lastIndexOf() method
    if (index == NOT_FOUND) {
  	return filename;
    } else {
	return filename.substring(0, index);
    }
}

Therefore, the Apache Commons IO's method won't work with dotfiles either:

@Test
public void givenDotFileWithoutExt_whenCallApacheCommonsMethod_thenCannotGetDesiredResult() {
    //negative assertion
    assertNotEquals(".baeldung", FilenameUtils.removeExtension(".baeldung"));
}

If a filename has multiple extensions, the removeExtension() method cannot remove all extensions:

@Test
public void givenFileWithoutMultipleExt_whenCallApacheCommonsMethod_thenCannotRemoveAllExtensions() {
    //negative assertion
    assertNotEquals("baeldung", FilenameUtils.removeExtension("baeldung.tar.gz"));
}

5. Removing the Extension(s) From a Filename

So far, we've seen utility methods for removing the extension from a filename in two widely used libraries. Both methods are pretty handy and work for the most common cases.

However, on the other hand, they have some shortcomings:

  • They won't work for dotfiles, for example, “.baeldung
  • When a filename has multiple extensions, they don't provide an option to remove the last extension only or all extensions

Next, let's build a method to cover all cases:

public static String removeFileExtension(String filename, boolean removeAllExtensions) {
    if (filename == null || filename.isEmpty()) {
        return filename;
    }
    String extPattern = "(?<!^)[.]" + (removeAllExtensions ? ".*" : "[^.]*$");
    return filename.replaceAll(extPattern, "");
}

We added a boolean parameter removeAllExtensions to provide the option to remove all extensions or only the last extension from a filename.

The core part of this method is the regex pattern. So let's understand what does this regex pattern do:

  • “(?<!^)[.]” – We use a negative-lookbehind in this regex. It matches a dot “.” that is not at the beginning of the filename
  • (?<!^)[.].*” – If the removeAllExtensions option is set, this will match the first matched dot until the end of the filename
  • (?<!^)[.][^.]*$” – This pattern matches only the last extension

Finally, let's write some test methods to verify if our method works for all different cases:

@Test
public void givenFilenameNoExt_whenCallFilenameUtilMethod_thenGetExpectedFilename() {
    assertEquals("baeldung", MyFilenameUtil.removeFileExtension("baeldung", true));
    assertEquals("baeldung", MyFilenameUtil.removeFileExtension("baeldung", false));
}
@Test
public void givenSingleExt_whenCallFilenameUtilMethod_thenGetExpectedFilename() {
    assertEquals("baeldung", MyFilenameUtil.removeFileExtension("baeldung.txt", true));
    assertEquals("baeldung", MyFilenameUtil.removeFileExtension("baeldung.txt", false));
}
@Test
public void givenDotFile_whenCallFilenameUtilMethod_thenGetExpectedFilename() {
    assertEquals(".baeldung", MyFilenameUtil.removeFileExtension(".baeldung", true));
    assertEquals(".baeldung", MyFilenameUtil.removeFileExtension(".baeldung", false));
}
@Test
public void givenDotFileWithExt_whenCallFilenameUtilMethod_thenGetExpectedFilename() {
    assertEquals(".baeldung", MyFilenameUtil.removeFileExtension(".baeldung.conf", true));
    assertEquals(".baeldung", MyFilenameUtil.removeFileExtension(".baeldung.conf", false));
}
@Test
public void givenDoubleExt_whenCallFilenameUtilMethod_thenGetExpectedFilename() {
    assertEquals("baeldung", MyFilenameUtil.removeFileExtension("baeldung.tar.gz", true));
    assertEquals("baeldung.tar", MyFilenameUtil.removeFileExtension("baeldung.tar.gz", false));
}
@Test
public void givenDotFileWithDoubleExt_whenCallFilenameUtilMethod_thenGetExpectedFilename() {
    assertEquals(".baeldung", MyFilenameUtil.removeFileExtension(".baeldung.conf.bak", true));
    assertEquals(".baeldung.conf", MyFilenameUtil.removeFileExtension(".baeldung.conf.bak", false));
}

6. Conclusion

In this article, we've talked about how to remove extensions from a given filename.

First, we discussed the different scenarios of removing extensions.

Next, we've introduced the methods provided by two widely used libraries: Guava and Apache Commons IO. They are pretty handy and work for common cases but cannot work for dotfiles. Also, they don't provide an option to remove a single extension or all extensions.

Finally, we built a method to cover all requirements.

As always, the full source code of the article is available over on GitHub.

The post Get a Filename Without the Extension in Java first appeared on Baeldung.

        

Comparing Doubles in Java

$
0
0

1. Overview

In this tutorial, we'll talk about the different ways of comparing double values in Java. In particular, it isn't as easy as comparing other primitive types. As a matter of fact, it's problematic in many other languages, not only Java.

First, we'll explain why using the simple == operator is inaccurate and might cause difficult to trace bugs in the runtime. Then, we'll show how to compare doubles in plain Java and common third-party libraries correctly.

2. Using the == Operator

Inaccuracy with comparisons using the == operator is caused by the way double values are stored in a computer's memory. We need to remember that there is an infinite number of values that must fit in limited memory space, usually 64 bits. As a result, we can't have an exact representation of most double values in our computers. They must be rounded to be saved.

Because of the rounding inaccuracy, interesting errors might occur:

double d1 = 0;
for (int i = 1; i <= 8; i++) {
    d1 += 0.1;
 }
double d2 = 0.1 * 8;
System.out.println(d1);
System.out.println(d2);

Both variables, d1 and d2, should equal 0.8. However, when we run the code above, we'll see the following results:

0.7999999999999999
0.8

In that case, comparing both values with the == operator would produce a wrong result. For this reason, we must use a more complex comparison algorithm.

If we want to have the best precision and control over the rounding mechanism, we can use java.math.BigDecimal class.

3. Comparing Doubles in Plain Java

The recommended algorithm to compare double values in plain Java is a threshold comparison method. In this case, we need to check whether the difference between both numbers is within the specified tolerance, commonly called epsilon:

double epsilon = 0.000001d;
assertThat(Math.abs(d1 - d2) < epsilon).isTrue();

The smaller the epsilon's value, the greater the comparison accuracy. However, if we specify the tolerance value too small, we'll get the same false result as in the simple == comparison. In general, epsilon's value with 5 and 6 decimals is usually a good place to start.

Unfortunately, there is no utility from the standard JDK that we could use to compare double values in the recommended and precise way. Luckily, we don't need to write it by ourselves. We can use a variety of dedicated methods provided by free and widely known third-party libraries.

4. Using Apache Commons Math

Apache Commons Math is one of the biggest open-source library dedicated to mathematics and statistics components. From the variety of different classes and methods, we'll focus on org.apache.commons.math3.util.Precision class in particular. It contains 2 helpful equals() methods to compare double values correctly:

double epsilon = 0.000001d;
assertThat(Precision.equals(d1, d2, epsilon)).isTrue();
assertThat(Precision.equals(d1, d2)).isTrue();

The epsilon variable used here has the same meaning as in the previous example. It is an amount of allowed absolute error. However, it's not the only similarity to the threshold algorithm. In particular, both equals methods use the same approach under the hood.

The two-argument function version is just a shortcut for the equals(d1, d2, 1) method call. The epsilon's value in that version is quite high. Therefore we shouldn't use it and always specify the tolerance value by ourselves.

5. Using Guava

Google's Guava is a big set of core Java libraries that extend the standard JDK capabilities. It contains a big number of useful math utils in the com.google.common.math package. To compare double values correctly in Guava, let's implement the fuzzyEquals() method from the DoubleMath class:

double epsilon = 0.000001d;
assertThat(DoubleMath.fuzzyEquals(d1, d2, epsilon)).isTrue();

The method name is different than in the Apache Commons Math, but it works practically identically under the hood. The only difference is that there is no overloaded method with the epsilon's default value.

6. Using JUnit

JUnit is one of the most widely used unit testing frameworks for Java. In general, every unit test usually ends with analyzing the difference between expected and actual values. Therefore, the testing framework must have correct and precise comparison algorithms. In fact, JUnit provides a set of comparing methods for common objects, collections, and primitive types, including dedicated methods to check double values equality:

double epsilon = 0.000001d;
assertEquals(d1, d2, epsilon);

As a matter of fact, it works the same as Guava's and Apache Commons's methods previously described.

It's important to point out that there is also a deprecated, two-argument version without the epsilon argument. However, if we want to be sure our results are always correct, we should stick with the three-argument version.

7. Conclusion

In this article, we've explored different ways of comparing double values in Java.

We've explained why simple comparison might cause difficult to trace bugs in the runtime. Then, we've shown how to compare values in plain Java and common libraries correctly.

As always, the source code for the examples can be found over on GitHub.

The post Comparing Doubles in Java first appeared on Baeldung.

        

Adding Parameters to HttpClient Requests

$
0
0

1. Introduction

HttpClient is part of the Apache HttpComponents project that provides a toolset of low-level Java components focused on HTTP and associated protocols. The most essential function of HttpClient is to execute HTTP methods.

In this short tutorial, we'll discuss adding parameters to HttpClient requests. We'll learn how to use UriBuilder with String name-value pairs and also NameValuePairs. Similarly, we'll see how to pass parameters using UrlEncodedFormEntity.

2. Add Parameters to HttpClient Requests Using UriBuilder

UriBuilder helps us to easily create URIs and add parameters via builder pattern. We can add parameters using String name-value pairs, or utilize NameValuePairs class for that purpose.

In this example, a final URL should look like this:

https://example.com?param1=value1&param2=value2

Let's see how to use String name-value pairs:

public CloseableHttpResponse sendHttpRequest() {
    HttpGet httpGet = new HttpGet("https://example.com");
    URI uri = new URIBuilder(httpGet.getURI())
      .addParameter("param1", "value1")
      .addParameter("param2", "value2")
      .build();
   ((HttpRequestBase) httpGet).setURI(uri);
    CloseableHttpResponse response = client.execute(httpGet);
    client.close();
}

Also, we can go with the NameValuePair list for HttpClient request:

public CloseableHttpResponse sendHttpRequest() {
    List nameValuePairs = new ArrayList();
    nameValuePairs.add(new BasicNameValuePair("param1", "value1"));
    nameValuePairs.add(new BasicNameValuePair("param2", "value2"));
    HttpGet httpGet = new HttpGet("https://example.com");
    URI uri = new URIBuilder(httpGet.getURI())
      .addParameters(nameValuePairs)
      .build();
   ((HttpRequestBase) httpGet).setURI(uri);
    CloseableHttpResponse response = client.execute(httpGet);
    client.close();
}

Similarly, UriBuilder can be used to add parameters to other HttpClient request methods.

3. Add Parameters to HttpClient Request Using UrlEncodedFormEntity

Another approach would be to utilize UrlEncodedFormEntity:

public CloseableHttpResponse sendHttpRequest() {
    List nameValuePairs = new ArrayList();
    nameValuePairs.add(new BasicNameValuePair("param1", "value1"));
    nameValuePairs.add(new BasicNameValuePair("param2", "value2"));
    HttpPost httpPost = new HttpPost("https://example.com");
    httpPost.setEntity(new UrlEncodedFormEntity(nameValuePairs, StandardCharsets.UTF_8));
    CloseableHttpResponse response = client.execute(httpPost);
    client.close();
}

Notice that UrlEncodedFormEntity couldn't be used for GET requests, since GET request does not have a body that could contain an entity.

4. Conclusion

In this example, we showed how to add parameters to HttpClient requests. Also, the implementation of all these examples and code snippets are available over on GitHub.

The post Adding Parameters to HttpClient Requests first appeared on Baeldung.

        

Java Weekly, Issue 363

$
0
0

1. Spring and Java

>> Incubator Support for HTTP/3 in Netty [netty.io]

Say hello to HTTP/3 and QUIC – the first incubator support of QUIC in Netty based on Cloudflare's quiche implementation!

>> Towards OpenJDK 17 [cl4es.github.io]

Faster startup times and static images: using the JVM as the basis for Project Leyden!

>> Introducing Hibernate Reactive [in.relation.to]

Meet the new Reactive API for the Hibernate ORM: communicating with the relational databases using the non-blocking IO.

Also worth reading:

Webinars and presentations:

Time to upgrade:

2. Technical

>> Don't Panic: Kubernetes and Docker [kubernetes.io]

Kubernetes deprecates Docker as its container runtime: towards more lightweight and CRI compatible container runtimes.

Also worth reading:

3. Musings

>> Team Manager's Toolkit for 1-on-1s [phauer.com]

In praise of 1-on-1 sessions: a recipe for maintaining growth, development, and becoming and staying happy!

Also worth reading:

4. Comics

And my favorite Dilberts of the week:

>> 5 G Is 4 G [dilbert.com]

>> Married Zoomers [dilbert.com]

>> Thought Leader [dilbert.com]

5. Pick of the Week

>> If Self-Discipline Feels Difficult, Then You’re Doing It Wrong [markmanson.net]

The post Java Weekly, Issue 363 first appeared on Baeldung.

        

ArrayList vs. LinkedList vs. HashMap in Java

$
0
0

1. Overview

Collections in Java are based on a couple of core interfaces and more than a dozen implementation classes. The wide selection of different implementations can sometimes lead to confusion.

Deciding on which collection type to use for a particular use case is not a trivial task. That decision can have a great impact on our code readability and performance.

Instead of explaining all types of collections in a single article, we'll explain three of the most common ones: ArrayList, LinkedList, and HashMap. In this tutorial, we'll look at how they store data, their performance, and recommend when to use them.

2. Collections

A collection is simply a Java object that groups other objects together. The Java Collections Framework contains a set of data structures and algorithms for representing and manipulating collections. If applied correctly, the provided data structures help reduce programming effort and increase performance.

2.1. Interfaces

The Java Collections Framework contains four basic interfaces: List, Set, Map, and Queue. It is important to understand the intended usage of these interfaces before looking at the implementation classes.

Let's have a quick look at three of the four core interfaces that we'll use in this article:

  • The List interface is dedicated to storing ordered collections of objects. It allows us to positionally access and insert new elements, as well as save duplicate values
  • The Map interface supports a key-value pair mapping of the data. To access a certain value, we need to know its unique key
  • The Queue interface enables the storage of data based on the first-in-first-out order. Similar to a real-world queue line

HashMap implements the Map interface. The List interface is implemented by both ArrayList and LinkedList. LinkedList additionally implements the Queue interface.

2.2. List vs. Map

A common antipattern we sometimes encounter is trying to maintain order using a map. Thus, not making use of other collection types more suitable for the job.

Just because we can solve many problems with a single collection type doesn't mean we should.

Let's look at a bad example, where we use a map to save data based on the positional key:

Map<Integer, String> map = new HashMap<>();
map.put(1, "Daniel");
map.put(2, "Marko");
for (String name : map.values()) {
    assertThat(name).isIn(map.values());
}
assertThat(map.values()).containsExactlyInAnyOrder("Daniel", "Marko");

When we iterate through the map values, we're not guaranteed to retrieve them in the same order we put them in. That is simply because a map wasn't designed for maintaining the order of elements.

We can rewrite this example in a much more readable way using a list. Lists are ordered by definition, so we can iterate through the items in the same order that we inserted them:

List<String> list = new ArrayList<>();
list.add("Daniel");
list.add("Marko");
for (String name : list) {
    assertThat(name).isIn(list);
}
assertThat(list).containsExactly("Daniel", "Marko");

Maps are designed for quick access and search based on unique keys. When we want to maintain order or work with position-based indexes, lists are a natural choice.

3. ArrayList

ArrayList is the most commonly used implementation of the List interface in Java. It is based on built-in arrays but can dynamically grow and shrink as we add or remove elements.

We use indexes that start from zero to access list elements. We can insert a new element either at the end, or the specific position of the list:

List<String> list = new ArrayList<>();
list.add("Daniel");
list.add(0, "Marko");
assertThat(list).hasSize(2);
assertThat(list.get(0)).isEqualTo("Marko");

To remove an element from the list, we need to provide the object reference or its index:

List<String> list = new ArrayList<>(Arrays.asList("Daniel", "Marko"));
list.remove(1);
assertThat(list).hasSize(1);
assertThat(list).doesNotContain("Marko");

3.1. Performance

ArrayList provides us with dynamic arrays in Java. Although slower than the built-in arrays, ArrayList helps us save some programming effort and improve code readability.

When we talk about time complexity, we make use of the Big-O notation. The notation describes how the time to perform the algorithm grows with the size of the input.

ArrayList allows random access since arrays are based on indexes. That means that accessing any element always takes a constant time O(1).

Adding new elements also takes O(1) time, except when adding an element on a specific position/index, then it takes O(n). Checking if a specific element exists in the given list runs in linear O(n) time.

The same is true for the removal of elements. We need to iterate the entire array to find the element selected for removal.

3.2. Usage

Whenever we're unsure what collection type to use, it's probably a good idea to start with an ArrayList. Keep in mind that accessing items based on indexes will be very fast. However, searching for items based on their value or adding/removing items at a specific position will be expensive.

Using ArrayList makes sense when it is important to maintain the same order of items, and quick access time based on the position/index is an important criterion.

Avoid using ArrayList when the order of items is not important. Also, try to avoid it when items often need to be added at a specific position. Likewise, bear in mind that ArrayList may not be the best option when searching for specific item values is an important requirement, especially if the list is large.

4. LinkedList

LinkedList is a doubly-linked list implementation. Implementing both the List and Deque (an extension of Queue) interfaces. Unlike ArrayList, when we store data in a LinkedList, every element maintains a link to the previous one.

Besides standard List insertion methods, LinkedList supports additional methods which can add an element at the beginning or the end of the list:

LinkedList<String> list = new LinkedList<>();
list.addLast("Daniel");
list.addFirst("Marko");
assertThat(list).hasSize(2);
assertThat(list.getLast()).isEqualTo("Daniel");

This list implementation also offers methods for removing elements from the beginning or at the end of the list:

LinkedList<String> list = new LinkedList<>(Arrays.asList("Daniel", "Marko", "David"));
list.removeFirst();
list.removeLast();
assertThat(list).hasSize(1);
assertThat(list).containsExactly("Marko");

The implemented Deque interface provides queue-like methods for retrieving, adding, and deleting elements:

LinkedList<String> list = new LinkedList<>();
list.push("Daniel");
list.push("Marko");
assertThat(list.poll()).isEqualTo("Marko");
assertThat(list).hasSize(1);

4.1. Performance

LinkedList consumes a bit more memory than an ArrayList since every node stores two references to the previous and next element.

The insertion, addition, and removal operations are faster in a LinkedList because there is no resizing of an array done in the background. When a new item is added somewhere in the middle of the list, only references in surrounding elements need to change.

LinkedList supports O(1) constant-time insertion at any position in the collection. However, it is less efficient at accessing items in a specific position, taking O(n) time.

Removing an element also takes O(1) constant-time, since we just need to modify a few pointers. Checking if a specific element exists in the given list takes O(n) linear time, same as for an ArrayList.

4.2. Usage

Most of the time we can use ArrayList as the default List implementation. However, in certain use-cases, we should make use of LinkedList. Those include when we prefer constant insertion and deletion time, over constant access time, and effective memory usage.

Using LinkedList makes sense when maintaining the same order of items and quick insertion time (adding and removing items at any position) is an important criterion.

Like an ArrayList, we should avoid using LinkedList when the order of items is not important. LinkedList is not the best option when fast access time or searching for items is an important requirement.

5. HashMap

Unlike ArrayList and LinkedList, HashMap implements the Map interface. That means that every key is mapped to exactly one value. We always need to know the key to retrieve the corresponding value from the collection:

Map<String, String> map = new HashMap<>();
map.put("123456", "Daniel");
map.put("654321", "Marko");
assertThat(map.get("654321")).isEqualTo("Marko");

Similarly, we can only delete a value from the collection using its key:

Map<String, String> map = new HashMap<>();
map.put("123456", "Daniel");
map.put("654321", "Marko");
map.remove("654321");
assertThat(map).hasSize(1);

5.1. Performance

One might ask, why not simply use a List and get rid of the keys all together? Especially since HashMap consumes more memory for saving keys and its entries are not ordered. The answer lies in the performance benefits for searching elements.

HashMap is very efficient at checking if a key exists or retrieving a value based on a key. Those operations take O(1) on average.

Adding and removing elements from a HashMap based on a key takes O(1) constant-time. Checking for an element without knowing the key takes linear time O(n), as it's necessary to loop over all the elements.

5.2. Usage

Along with ArrayListHashMap is one of the most frequently used data structures in Java. Unlike different list implementations, HashMap makes use of indexing to perform a jump to a specific value, making the search time constant, even for large collections.

Using HashMap makes sense only when unique keys are available for the data we want to store. We should use it when searching for items based on a key and quick access time is an important requirement.

We should avoid using HashMap when it is important to maintain the same order of items in a collection.

6. Conclusion

In this article, we explored three common collection types in Java: ArrayList, LinkedList, and HashMap. We looked at their performance for adding, removing, and searching for items. Based on that, we provided recommendations on when to apply each of them in our Java applications.

In the examples, we covered only basic methods for adding and removing items. For a more detailed look at each implementation API, please visit our dedicated ArrayList, ArrayList, and HashMap articles.

As always, the complete source code is available over on GitHub.

The post ArrayList vs. LinkedList vs. HashMap in Java first appeared on Baeldung.

        

JDBC URL Format For Different Databases

$
0
0

1. Overview

When we work with a database in Java, usually we connect to the database with JDBC.

The JDBC URL is an important parameter to establish the connection between our Java application and the database. However, the JDBC URL format can be different for different database systems.

In this tutorial, we'll take a closer look at the JDBC URL formats of several widely used databases: Oracle, MySQL, Microsoft SQL Server, and PostgreSQL.

2. JDBC URL Formats for Oracle

Oracle database systems are widely used in enterprise Java applications. Before we can take a look at the format of the JDBC URL to connect Oracle databases, we should first make sure the Oracle Thin database driver is in our classpath.

For example, if our project is managed by Maven, we need to add the ojdbc14 dependency in our pom.xml:

<dependency>
    <groupId>com.oracle</groupId>
    <artifactId>ojdbc14</artifactId>
    <version>10.2.0.4.0</version>
</dependency>

Due to some license issues, the Maven Central repository only points to the POM file of this artifact. Therefore, we need to download the jar and manually install it in our Maven repository.

The Thin driver offers several kinds of JDBC URL formats:

Next, we'll go through each of these formats.

2.1. Connect to Oracle Database SID

In some older versions of the Oracle database, the database is defined as a SID. Let's see the JDBC URL format for connecting to a SID:

jdbc:oracle:thin:[<user>/<password>]@<host>[:<port>]:<SID>

For example, assuming we have an Oracle database server host “myoracle.db.server:1521“, and the name of the SID is “my_sid“, we can follow the format above to build the connection URL and connect to the database:

@Test
public void givenOracleSID_thenCreateConnectionObject() {
    String oracleJdbcUrl = "jdbc:oracle:thin:@myoracle.db.server:1521:my_sid";
    String username = "dbUser";
    String password = "1234567";
    try (Connection conn = DriverManager.getConnection(oracleJdbcUrl, username, password)) {
        assertNotNull(conn);
    } catch (SQLException e) {
        System.err.format("SQL State: %s\n%s", e.getSQLState(), e.getMessage());
    }
}

2.2. Connect to Oracle Database Service Name

The format of the JDBC URL to connect Oracle databases via service name is pretty similar to the one we used to connect via SID:

jdbc:oracle:thin:[<user>/<password>]@//<host>[:<port>]/<service>

We can connect to the service “my_servicename” on the Oracle database server “myoracle.db.server:1521“:

@Test
public void givenOracleServiceName_thenCreateConnectionObject() {
    String oracleJdbcUrl = "jdbc:oracle:thin:@//myoracle.db.server:1521/my_servicename";
    ...
    try (Connection conn = DriverManager.getConnection(oracleJdbcUrl, username, password)) {
        assertNotNull(conn);
        ...
    }
    ...
}

2.3. Connect to Oracle Database With tnsnames.ora Entries

We can also include tnsnames.ora entries in the JDBC URL to connect to Oracle databases:

jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=<host>)(PORT=<port>))(CONNECT_DATA=(SERVICE_NAME=<service>)))

Let's see how to connect to our “my_servicename” service using entries from the tnsnames.ora file:

@Test
public void givenOracleTnsnames_thenCreateConnectionObject() {
    String oracleJdbcUrl = "jdbc:oracle:thin:@" +
      "(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)" +
      "(HOST=myoracle.db.server)(PORT=1521))" +
      "(CONNECT_DATA=(SERVICE_NAME=my_servicename)))";
    ...
    try (Connection conn = DriverManager.getConnection(oracleJdbcUrl, username, password)) {
        assertNotNull(conn);
        ...
    }
    ...
}

3. JDBC URL Formats for MySQL

In this section, let's discuss how to write the JDBC URL to connect to MySQL databases.

To connect to a MySQL database from our Java application, let's first add the JDBC driver mysql-connector-java dependency in our pom.xml:

<dependency>
    <groupId>mysql</groupId>
    <artifactId>mysql-connector-java</artifactId>
    <version>8.0.22</version>
</dependency>

Next, let's take a look at the generic format of the connection URL supported by the MySQL JDBC driver:

protocol//[hosts][/database][?properties]

Let's see an example of connecting to the MySQL database “my_database” on the host “mysql.db.server“:

@Test
public void givenMysqlDb_thenCreateConnectionObject() {
    String jdbcUrl = "jdbc:mysql://mysql.db.server:3306/my_database?useSSL=false&serverTimezone=UTC";    
    String username = "dbUser";
    String password = "1234567";
    try (Connection conn = DriverManager.getConnection(jdbcUrl, username, password)) {
        assertNotNull(conn);
    } catch (SQLException e) {
        System.err.format("SQL State: %s\n%s", e.getSQLState(), e.getMessage());
    }
}

The JDBC URL in the example above looks straightforward. It has four building blocks:

  • protocoljdbc:mysql:
  • host mysql.db.server:3306
  • databasemy_database
  • propertiesuseSSL=false&serverTimezone=UTC

However, sometimes, we may face more complex situations, such as different types of connections or multiple MySQL hosts, and so on.

Next, we'll take a closer look at each building block.

3.1. Protocol

Except for the ordinary “jdbc:mysql:” protocol, the connector-java JDBC driver still supports protocols for some special connections:

When we talk about the load-balancing and JDBC replication, we may realize that there should be multiple MySQL hosts.

Next, let's check out the details of another part of the connection URL — hosts.

3.2. Hosts

We've seen the JDBC URL example of defining a single host in a previous section — for example, mysql.db.server:3306.

However, if we need to handle multiple hosts, we can list hosts in a comma-separated list: host1, host2,…,hostN.

We can also enclose the comma-separated host list by square brackets: [host1, host2,…,hostN].

Let's see several JDBC URL examples of connecting to multiple MySQL servers:

  • jdbc:mysql://myhost1:3306,myhost2:3307/db_name
  • jdbc:mysql://[myhost1:3306,myhost2:3307]/db_name
  • jdbc:mysql:loadbalance://myhost1:3306,myhost2:3307/db_name?user=dbUser&password=1234567&loadBalanceConnectionGroup=group_name&ha.enableJMX=true

If we look at the last example above closely, we'll see that after the database name, there are some definitions of properties and user credentials. We'll look at these next.

3.3. Properties and User Credentials

Valid global properties will be applied to all hosts. Properties are preceded by a question mark “?” and written as key=value pairs separated by the “& symbol:

jdbc:mysql://myhost1:3306/db_name?prop1=value1&prop2=value2

We can put user credentials in the properties list as well:

jdbc:mysql://myhost1:3306/db_name?user=root&password=mypass

Also, we can prefix each host with the user credentials in the format “user:password@host:

jdbc:mysql://root:mypass@myhost1:3306/db_name

Further, if our JDBC URL contains a list of hosts and all hosts use the same user credentials, we can prefix the host list:

jdbc:mysql://root:mypass[myhost1:3306,myhost2:3307]/db_name

After all, it is also possible to provide the user credentials outside the JDBC URL.

We can pass the username and password to the DriverManager.getConnection(String url, String user, String password) method when we call the method to obtain a connection.

4. JDBC URL Format for Microsoft SQL Server

Microsoft SQL Server is another popular database system. To connect an MS SQL Server database from a Java application, we need to add the mssql-jdbc dependency into our pom.xml:

<dependency>
    <groupId>com.microsoft.sqlserver</groupId>
    <artifactId>mssql-jdbc</artifactId>
    <version>8.4.1.jre11</version>
</dependency>

Next, let's look at how to build the JDBC URL to obtain a connection to MS SQL Server.

The general format of the JDBC URL for connection to the MS SQL Server database is:

jdbc:sqlserver://[serverName[\instanceName][:portNumber]][;property=value[;property=value]]

Let's have a closer look at each part of the format.

  • serverName – the address of the server we'll connect to; this could be a domain name or IP address pointing to the server
  • instanceName – the instance to connect to on serverName; it's an optional field, and the default instance will be chosen if the field isn't specified
  • portNumber – this is the port to connect to on serverName (default port is 1433)
  • properties – can contain one or more optional connection properties, which must be delimited by the semicolon, and duplicate property names are not allowed

Now, let's say we have an MS SQL Server database running on host “mssql.db.server“, the instanceName on the server is “mssql_instance“, and the name of the database we want to connect is “my_database“.

Let's try to obtain the connection to this database:

@Test
public void givenMssqlDb_thenCreateConnectionObject() {
    String jdbcUrl = "jdbc:sqlserver://mssql.db.server\\mssql_instance;databaseName=my_database";
    String username = "dbUser";
    String password = "1234567";
    try (Connection conn = DriverManager.getConnection(jdbcUrl, username, password)) {
        assertNotNull(conn);
    } catch (SQLException e) {
        System.err.format("SQL State: %s\n%s", e.getSQLState(), e.getMessage());
    }
}

5. JDBC URL Format for PostgreSQL

PostgreSQL is a popular, open-source database system. To work with PostgreSQL, the JDBC driver postgresql should be added as a dependency in our pom.xml:

<dependency>
    <groupId>org.postgresql</groupId>
    <artifactId>postgresql</artifactId>
    <version>42.2.18</version>
</dependency>

The general form of the JDBC URL to connect to PostgreSQL is:

jdbc:postgresql://host:port/database?properties

Now, let's look into each part in the above JDBC URL format.

The host parameter is the domain name or IP address of the database server.

If we want to specify an IPv6 address, the host parameter must be enclosed by square brackets, for example, jdbc:postgresql://[::1]:5740/my_database.mysql

The port parameter specifies the port number PostgreSQL is listening on. The port parameter is optional, and the default port number is 5432.

As its name implies, the database parameter defines the name of the database we want to connect to.

The properties parameter can contain a group of key=value pairs separated by the “&” symbol.

After understanding the parameters in the JDBC URL format, let's see an example of how to obtain the connection to a PostgreSQL database:

@Test
public void givenPostgreSqlDb_thenCreateConnectionObject() {
    String jdbcUrl = "jdbc:postgresql://postgresql.db.server:5430/my_database?ssl=true&loglevel=2";
    String username = "dbUser";
    String password = "1234567";
    try (Connection conn = DriverManager.getConnection(jdbcUrl, username, password)) {
        assertNotNull(conn);
    } catch (SQLException e) {
        System.err.format("SQL State: %s\n%s", e.getSQLState(), e.getMessage());
    }
}

In the example above, we connect to a PostgreSQL database with:

  • host:port – postgresql.db.server:5430
  • databasemy_database
  • properties – ssl=true&loglevel=2

6. Conclusion

This article discussed the JDBC URL formats of four widely used database systems: Oracle, MySQL, Microsoft SQL Server, and PostgreSQL.

We've also seen different examples of building the JDBC URL string to obtain connections to those databases.

As always, the full source code of the article is available over on GitHub.

The post JDBC URL Format For Different Databases first appeared on Baeldung.

        
Viewing all 3777 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>