Creating an IMDB clone with a Java backend using SparkJava and Neo4j

We decided to create course for Java Backend and want to discuss some aspects and selected alternatives that we noticed during its creation.

The Application Development course walks you through the implementation of endpoints step by step, starting with test datasets and ending with a complete application ready to be deployed.

The app is an IMDB clone based on a dataset MovieLens recommendations, supplemented with data on films and roles from themoviedb.org.

The frontend is written in vue.js and looks pretty nice.

It calls multiple REST API endpoints to call different views and functions.

Main functions:

  • registration and authentication of the user and storage of his information

  • list of genres, movies, people sorted and filtered and related information

  • add movies to favorites and ratings, and return those lists and recommendations

The repository contains complete application code and can be used to build and run the application.

It has branches for each course module so you can track progress/differences and jump to the completed solution if something goes wrong.

The course infrastructure uses Asciidoctor.jsso we can use includes (using the tag region) directly from our code repository.

So we also get syntax highlighting, warning markers, and many other useful things out of the box.

During the interactive course, many quizzes test understanding, allow you to run tests, or check the database for successful updates.

The course also automatically integrates with Neo4j sandboxso you can run test queries as well as the application for your personal instance hosting the movie dataset.

Setting

We used the traditional Java setup by installing Java 17 and Apache Maven via sdkman.

Since Java 17 introduced inline blocks and records, we wanted to use these features.

sdk install java 17-open
sdk use java 17-open
sdk install maven

Web framework – SparkJava

You may not have heard of SparkJavawhich has been around for quite a while and is a minimalistic web framework equivalent to Java’s Express/Sinatra.

Since there will be other courses with Spring (Data Neo4j), we wanted to keep this course as simple as possible. Quarkus was also an option we thought about, but then chose SparkJava due to its minimalism.

Also the JavaScript course app used Express.

Thus, porting the code from JavaScript to Java was quite simple, with just a few changes, we had something working in a few minutes.

A minimal hello world example looks like this:

import static spark.Spark.*;

public class HelloWorld {
    public static void main(String[] args) {
        get("/hello", (req, res) -> "Hello World");
    }
}

Our entire main application, which registers routes, adds error handling, checks authorization, serves JSON formatting of public files (using GSON), and starts the server, is no more than 20 lines long.

Documentation for SparkJava very short and at the same time exhaustive, everything you need can be found quickly.

package neoflix;

import static spark.Spark.*;

import java.util.*;
import com.google.gson.Gson;
import neoflix.routes.*;
import org.neo4j.driver.*;

public class NeoflixApp {

    public static void main(String[] args) throws Exception {
        AppUtils.loadProperties();
        int port = AppUtils.getServerPort();
        port(port);

        Driver driver = AppUtils.initDriver();
        Gson gson = GsonUtils.gson();

        staticFiles.location("/public");
        String jwtSecret = AppUtils.getJwtSecret();
        before((req, res) -> AppUtils.handleAuthAndSetUser(req, jwtSecret));
        path("/api", () -> {
            path("/movies", new MovieRoutes(driver, gson));
            path("/genres", new GenreRoutes(driver, gson));
            path("/auth", new AuthRoutes(driver, gson, jwtSecret));
            path("/account", new AccountRoutes(driver, gson));
            path("/people", new PeopleRoutes(driver, gson));
        });
        exception(ValidationException.class, (exception, request, response) -> {
            response.status(422);
            var body = Map.of("message",exception.getMessage(), "details", exception.getDetails());
            response.body(gson.toJson(body));
            response.type("application/json");
        });
        System.out.printf("Server listening on http://localhost:%d/%n", port);
    }
}

Routes – AccountRoutes

Routes can be grouped by root path and then processed in a simple DSL. Here’s how to get a list of favorites in AccountRoutes

get("/favorites", (req, res) -> {
    var params = Params.parse(req, Params.MOVIE_SORT);
    String userId = AppUtils.getUserId(req);
    return favoriteService.all(userId, params);
}, gson::toJson);

We first parse some parameters from the request URL, then we extract the userId from the request attributes and call the FavoriteService to query the database.

Fixtures

As the course progresses, an implementation is gradually added to use the database.
To be able to test, run and interact with the application from the very beginning, some static fixture datasets are used to return responses from services.

The original Javascript application used JS files to store fixtures as JS objects, but we wanted a more portable option.

So we’ve converted fixtures to JSON files and then read them into List<Map> structures in services that used fixtures.

public static List<Map> loadFixtureList(final String name) {
    var fixture = new InputStreamReader(AppUtils.class.getResourceAsStream("/fixtures/" + name + ".json"));
    return GsonUtils.gson().fromJson(fixture,List.class);
}

Which can then be used in a service with this.popular = AppUtils.loadFixtureList("popular");.

[{
    "actors": [
      {"name": "Tim Robbins","tmdbId": "0000209"},
      {"name": "William Sadler","tmdbId": "0006669"},
      {"name": "Bob Gunton","tmdbId": "0348409"},
      {"name": "Morgan Freeman","tmdbId": "0000151"}
    ],
    "languages": ["English"],
    "plot": "Two imprisoned men bond over a number of years, finding solace and eventual redemption through acts of common decency.",
    "year": 1994,
    "genres": [{"name": "Drama"},{"name": "Crime"}],
    "directors": [{"name": "Frank Darabont","tmdbId": "0001104"}],
    "imdbRating": 9.3,
    "tmdbId": "0111161",
    "favorite": false,
    "title": "Shawshank Redemption, The",
    "poster": "https://image.tmdb.org/t/p/w440_and_h660_face/5KCVkau1HEl7ZzfPsKAPM0sMiKc.jpg"
}]

To make it not completely static, we used Java Streams processing to implement filtering, sorting, and pagination of fixture data.

public static List<Map> process(
                List<Map> result, Params params) {
    return params == null ? result : result.stream()
        .sorted((m1, m2) ->
            (params.order() == Params.Order.ASC ? 1 : -1) *
                ((Comparable)m1.getOrDefault(params.sort().name(),"")).compareTo(
                        m2.getOrDefault(params.sort().name(),"")
                ))
        .skip(params.skip()).limit(params.limit())
        .toList();
}

Which is then used in services, for example:

public List<Map> all(Params params, String userId) {
    // TODO: Open an Session
    // TODO: Execute a query in a new Read Transaction
    // TODO: Get a list of Movies from the Result
    // TODO: Close the session


    return AppUtils.process(popular, params);
}

Neo4j driver

Real service implementations use the official Neo4j Java driver for database queries.
We can send parameterized Cypher requests to the server, use the parameters, and process the results as part of a repeatable transaction function (read or write transaction).

Add a driver dependency org.neo4j.driver:neo4j-java-driver в файл pom.xml.

You can then create one instance of the driver for the lifetime of your application and use driver sessions as needed.

Sessions do not hold TCP connections, but use them from a pool as needed.
Within a session, you can use read and write transactions to complete your unit of work.

We have read the connection credentials from application.properties and set it as a system property for convenience, and then initialized the driver.

static Driver initDriver() {
    AuthToken auth = AuthTokens.basic(getNeo4jUsername(), getNeo4jPassword());
    Driver driver = GraphDatabase.driver(getNeo4jUri(), auth);
    driver.verifyConnectivity();
    return driver;
}

FavoriteService

The driver is then passed to each service when built and can be used from there to create sessions and interact with the database.

Here in the example FavoriteService to enumerate the featured user, you can see how we use String blocks for the Cypher statement and lambda expressions for the callback readTransaction

public List<Map> all(String userId, Params params) {
    // Open a new session
    try (var session = this.driver.session()) {

        // Retrieve a list of movies favorited by the user
        var favorites = session.readTransaction(tx -> {
            String query = """
                        MATCH (u:User {userId: $userId})-[r:HAS_FAVORITE]->(m:Movie)
                        RETURN m {
                            .*,
                            favorite: true
                        } AS movie
                        ORDER BY m.title ASC
                        SKIP $skip
                        LIMIT $limit
                    """;
            var res = tx.run(query, Values.parameters("userId", userId, "skip",
                                        params.skip(), "limit", params.limit()));
            return res.list(row -> row.get("movie").asMap());
        });
        return favorites;
    }
}

When adding a favorite movie, we use the write transaction FavoriteService.addto create a connection FAVORITE between the user and the movie.

public Map add(String userId, String movieId) {
    // Open a new Session
    try (var session = this.driver.session()) {
        // Create HAS_FAVORITE relationship within a Write Transaction
        var favorite = session.writeTransaction(tx -&gt; {
            String statement = """
                        MATCH (u:User {userId: $userId})
                        MATCH (m:Movie {tmdbId: $movieId})

                        MERGE (u)-[r:HAS_FAVORITE]-&gt;(m)
                                ON CREATE SET r.createdAt = datetime()

                        RETURN m {
                            .*,
                            favorite: true
                        } AS movie
                    """;
            var res = tx.run(statement,
                            Values.parameters("userId", userId, "movieId", movieId));
            return res.single().get("movie").asMap();
        });
        return favorite;
    // Throw an error if the user or movie could not be found
    } catch (NoSuchRecordException e) {
        throw new ValidationException("Could not create favorite movie for user",
            Map.of("movie",movieId, "user",userId));
    }
}

Method result.single() will fail if there is not exactly one result from NoSuchRecordExceptionso we don’t need to check it in the request.

Authentication

Our app also provides user management for personalization.
That’s why we must:

  • register user

  • authenticate user

  • validate the authorization token and add user information to the request

For registration and authentication, we save the user directly as a node in Neo4j and use the library bcrypt for hashing and comparing hashed passwords with input data.

Here is an example from the method authenticate class AuthService.

public Map authenticate(String email, String plainPassword) {
    // Open a new Session
    try (var session = this.driver.session()) {
        // Find the User node within a Read Transaction
        var user = session.readTransaction(tx -&gt; {
            String statement = "MATCH (u:User {email: $email}) RETURN u";
            var res = tx.run(statement, Values.parameters("email", email));
            return res.single().get("u").asMap();

        });
        // Check password
        if (!AuthUtils.verifyPassword(plainPassword, (String)user.get("password"))) {
            throw new ValidationException("Incorrect password", Map.of("password","Incorrect password"));
        }
        String sub = (String)user.get("userId");
        // compute JWT token signature
        String token = AuthUtils.sign(sub, userToClaims(user), jwtSecret);
        return userWithToken(user, token);
    } catch(NoSuchRecordException e) {
        throw new ValidationException("Incorrect email", Map.of("email","Incorrect email"));
    }
}

To pass authentication information in the form JWT tokens to the browser and back we use Java Auth0 library to create a token, and then also verify it.

This is done with a handler before in SparkJava which on successful validation saves the attribute subA containing the request attribute userId. Which routes can then access, for example for personalization or ratings.

static void handleAuthAndSetUser(Request req, String jwtSecret) {
    String token = req.headers("Authorization");
    String bearer = "Bearer ";
    if (token != null &amp;&amp; !token.isBlank() &amp;&amp; token.startsWith(bearer)) {
        token = token.substring(bearer.length());
        String userId = AuthUtils.verify(token, jwtSecret);
        req.attribute("user", userId);
    }
}
// usage in NeoflixApp
before((req, res) -&gt; AppUtils.handleAuthAndSetUser(req, jwtSecret));

Java 17 Entries

Initially, we planned to use Java 17 entries throughout the course, but then we ran into two problems.

First, the library Google Gson does not yet support (de-)serialization of records, so we would have to switch to Jackson instead (probably should have).

And the results of the Java Neo4j driver couldn’t just be converted to a record instance as we wanted.

Because we didn’t want to add a lot of unnecessary complexity to the educational code, especially. if you want to keep a one line lambda closure for the callback. That’s why we kept the API toMap()proposed for the results of the driver.

var movies = tx.run(query, params)
    .list(row -&gt; row.get("movie")
                    .computeOrDefault(v -&gt;
                        new Movie(v.get("title").asString(),v.get("tmbdId").asString()),
                            v.get("published").asLocalDate())));
var movies = tx.run(query, params).list(row -> row.get("movie").toMap());

Testing

We used JUnit 5 for testing, which was not difficult.

Because we wanted the same test to run across all branches of the repository, whether a database connection was available or not, we used the statements Assumeto skip a few tests and driver instance existence condition for some cleanup code in setup methods @BeforeClass/Before.

The course uses the command to run tests mvn test -Dtest=neoflix.TestName#testMethodwith which the course participant could check their progress and correct execution.

Some quizzes also display results that the user must complete in quizzes during the course.

Conclusion

For a real application, we could use one of the larger application frameworks as there are more needs to be met.

In addition, the reuse of simple Cypher requests for operations, as well as matching and error handling, had to be handled by the infrastructure code that we usually refactor.

But, for educational purposes, we left it in every service.

Feel free to check out the course Neo4j Java and neoflix application

If you are interested in rewriting this example from any other web framework: Spring Boot, Quarkus, Micronaut, vert.x, Play, etc., please let us know and share the repository so we can add it as a branch.

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *