Read Last N Lines From File in Java

1. Introduction

In this article, we’ll see how can we read the last N lines from a file using different standard Java packages and the Apache Commons IO library.

2. Sample Data

We’ll use the sample data and parameters defined below for all our examples in this tutorial.

Let’s start by creating a simple file named data.txt that we’ll use as an input file:

line 1
line 2
line 3
line 4
line 5
line 6
line 7
line 8
line 9
line 10

Furthermore, we’ll use the following sample values the number of last N lines to read, the output to verify, and the file path:

private static final String FILE_PATH = "src/test/resources/data.txt";
private static final int LAST_LINES_TO_READ = 3;
private static final String OUTPUT_TO_VERIFY = "line 8\nline 9\nline 10";

3. Using BufferedReader

Let’s explore the BufferedReader class which will allow us to read file line by line. It has the advantage of not storing the whole file in memory. We can use the Queue which is a FIFO structure. While we read the file we will start removing the first element as soon as the Queue size reaches the number of lines to read:

@Test
public void givenFile_whenUsingBufferedReader_thenExtractedLastLinesCorrect() throws IOException {
    try (BufferedReader br = Files.newBufferedReader(Paths.get(FILE_PATH))) {
        Queue<String> queue = new LinkedList<>();
        String line;
        while ((line = br.readLine()) != null){
            if (queue.size() >= LAST_LINES_TO_READ) {
                queue.remove();
            }
            queue.add(line);
        }
        assertEquals(OUTPUT_TO_VERIFY, String.join("\n", queue));
    }
}

4. Using Scanner

We can achieve the same result with a similar approach using the Scanner class:

@Test
public void givenFile_whenUsingScanner_thenExtractedLastLinesCorrect() throws IOException {
    try (Scanner scanner = new Scanner(new File(FILE_PATH))) {
        Queue<String> queue = new LinkedList<>();
        while (scanner.hasNextLine()){
            if (queue.size() >= LAST_LINES_TO_READ) {
                queue.remove();
            }
            queue.add(scanner.nextLine());
        }
        assertEquals(OUTPUT_TO_VERIFY, String.join("\n", queue));
    }
}

5. Using NIO2 Files

If we want to work with large files, we can use the Files class. Its Lines method provides a stream to read a file line by line. After that, we will use a similar Queue approach to read the required content:

@Test
public void givenLargeFile_whenUsingFilesAPI_thenExtractedLastLinesCorrect() throws IOException{
    try (Stream<String> lines = Files.lines(Paths.get(FILE_PATH))) {
        Queue<String> queue = new LinkedList<>();
        lines.forEach(line -> {
            if (queue.size() >= LAST_LINES_TO_READ) {
                queue.remove();
            }
            queue.add(line);
        });
        assertEquals(OUTPUT_TO_VERIFY, String.join("\n", queue));
    }
}

6. Using Apache Commons IO

We can use the Apache Commons IO library. We will use the FileUtils class and the ReversedLinesFileReader class.

6.1. Read File Using FileUtils Class

This class provides the readLines method using which we can read the whole file in the list. This causes the whole file content to be stored in memory. We can traverse the list and can read the required content:

@Test
public void givenFile_whenUsingFileUtils_thenExtractedLastLinesCorrect() throws IOException{
    File file = new File(FILE_PATH);
    List<String> lines = FileUtils.readLines(file, "UTF-8");
    StringBuilder stringBuilder = new StringBuilder();
    for (int i = (lines.size() - LAST_LINES_TO_READ); i < lines.size(); i++) {
        stringBuilder.append(lines.get(i)).append("\n");
    }
    assertEquals(OUTPUT_TO_VERIFY, stringBuilder.toString().trim());
}

6.2. Read File Using ReversedLinesFileReader Class

This class allows to read files in reverse order using its readLines method. This helps to directly read the required content from the last without applying any other logic or skipping the file content:

@Test
public void givenFile_whenUsingReverseFileReader_thenExtractedLastLinesCorrect() throws IOException{
    File file = new File(FILE_PATH);
    try (ReversedLinesFileReader rlfReader = new ReversedLinesFileReader(file, StandardCharsets.UTF_8)) {
        List<String> lastLines = rlfReader.readLines(LAST_LINES_TO_READ);
        StringBuilder stringBuilder = new StringBuilder();
        Collections.reverse(lastLines);
        lastLines.forEach(
          line -> stringBuilder.append(line).append("\n")
        );
        assertEquals(OUTPUT_TO_VERIFY, stringBuilder.toString().trim());
    }
}

7. Conclusion

In this article, we have looked at the different ways of reading the last N lines from a file. We should pick the approach considering whether an application can sustain more CPU usage or more memory usage.

All of the code in this article is available over on GitHub.