How we screwed Neo4j to Helidon
The idea to experiment with integrating Neo4j with Helidon came naturally.
Neo4j is an open source graphical database management system implemented in Java. As of 2015, it is considered the most common graph DBMS. (Wikipedia, 10/21/2021)
Neo4j is written in Java and is available from software written in other languages using the Cypher query language, via the HTTP endpoint, or via the bolt protocol. Neo4j is currently the de facto standard for graph databases used in many industries.
Actually, it all started with a little conversation with Michael Simonis, one of the authors of Spring Data Neo4j 6 and maintainer of Neo4j-OGM. We asked Michael what he thinks about how Helidon and Neo4J can work together. Less than an hour later Michael sent me a link to this repository with a fully functional example for Helidon MP and Neo4j SDN.
When I started reading the code, I saw a strange dependency in the pom.xml file:
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-neo4j</artifactId>
</dependency>Spring? Come on! Why does our project have Spring?
But as I continued to explore the code, I was quite surprised to see the CDI 2.0 extension that handles driver initialization and preparation. This CDI extension ran seamlessly in Helidon as it fully supports the CDI container – awesome! I again felt the beauty and power of standards.
The next day Michael sent me another repo. This time, he included his ideas on how to use Neo4j with Helidon SE. The integration is also very simple. It was easy to use Helidon Config to bring the entire Neo4j configuration into a single configuration. And since the Neo4j driver fully supports native image, everything can be compiled into a native-image executable without any additional steps from the programmer’s point of view.
This sparked discussions between the Helidon and Neo4j teams about which kind of integration would be more appropriate for Helidon. As a result, we have created an official integration!
After some consultation with the Neo4j representative and my good friend Michael Simonis, we came to the conclusion that we only need to deliver an initialized driver to Helidon users. Metrics and Health Checks from Neo4j should be forwarded to Helidon / MicroProfile metrics and should be provided as separate modules – separate Maven dependencies.
So how do we write a Neo4j integration with Helidon? There are two flavors of Helidon – MP and SE. SE is a collection of reactive APIs implemented in pure Java. Absolutely no “magic” like a reflex or other tricks. Helidon MP essentially wraps SE and brings it to MicroProfile standards. This means that it is recommended to first implement the Neo4j integration for Helidon SE and then wrap it as CDI extensions for Helidon MP.
Let’s do that!
Disclaimer: In this article, I will demonstrate only the basic code snippets. Since Helidon is an open source project licensed under Apache 2.0, the complete code is available in the official Helidon repository.
In Helidon, we usually create a so-called support object to implement the integration. This object contains all configuration and initialization information. We follow the Builder pattern to read from the config. This means that we create an internal Builder object that should read all data from the configuration:
public Builder config(Config config) {
config.get("authentication.username").asString().ifPresent(this::username);
config.get("authentication.password").asString().ifPresent(this::password);
config.get("authentication.enabled").asBoolean().ifPresent(this::authenticationEnabled);
config.get("uri").asString().ifPresent(this::uri);
config.get("encrypted").asBoolean().ifPresent(this::encrypted);
//pool
config.get("pool.metricsEnabled").asBoolean().ifPresent(this::metricsEnabled);
config.get("pool.logLeakedSessions").asBoolean().ifPresent(this::logLeakedSessions);
config.get("pool.maxConnectionPoolSize").asInt().ifPresent(this::maxConnectionPoolSize);
config.get("pool.idleTimeBeforeConnectionTest").as(Duration.class).ifPresent(this::idleTimeBeforeConnectionTest);
config.get("pool.maxConnectionLifetime").as(Duration.class).ifPresent(this::maxConnectionLifetime);
config.get("pool.connectionAcquisitionTimeout").as(Duration.class).ifPresent(this::connectionAcquisitionTimeout);
//trust
config.get("trustsettings.trustStrategy").asString().map(TrustStrategy::valueOf).ifPresent(this::trustStrategy);
config.get("trustsettings.certificate").as(Path.class).ifPresent(this::certificate);
config.get("trustsettings.hostnameVerificationEnabled").asBoolean().ifPresent(this::hostnameVerificationEnabled);
return this;
} You can see the lines with the keys, they are actually taken from the config file. moreover, either from the SE configuration, or from the MicroProfile configuration. Each element is customizable following the Builder pattern:
...
public Builder password(String password) {
Objects.requireNonNull(password);
this.password = password;
return this;
}
... When all the fields are set, we can build the support object:
@Override
public Neo4j build() {
if (driver == null) {
driver = initDriver();
}
return new Neo4j(this);
} This way we guarantee that all values will not be null. The actual driver initialization is pretty straightforward (some methods are omitted):
private Driver initDriver() {
AuthToken authToken = AuthTokens.none();
if (authenticationEnabled) {
authToken = AuthTokens.basic(username, password);
}
org.neo4j.driver.Config.ConfigBuilder configBuilder = createBaseConfig();
configureSsl(configBuilder);
configurePoolSettings(configBuilder);
return GraphDatabase.driver(uri, authToken, configBuilder.build());
} The Support object then simply returns the driver:
public Driver driver() {
return driver;
} … and it can be used:
Neo4jMetricsSupport.builder()
.driver(neo4j.driver())
.build()
.initialize();
Driver neo4jDriver = neo4j.driver(); Helidon will take care of initializing it from the application.yaml file:
neo4j:
uri: bolt://localhost:7687
authentication:
username: neo4j
password: secret
pool:
metricsEnabled: true Actually everything – you can use it already in your code:
public List<Movie> findAll(){
try (var session = driver.session()) {
var query = ""
+ "match (m:Movie) "
+ "match (m) <- [:DIRECTED] - (d:Person) "
+ "match (m) <- [r:ACTED_IN] - (a:Person) "
+ "return m, collect(d) as directors, collect({name:a.name, roles: r.roles}) as actors";
return session.readTransaction(tx -> tx.run(query).list(r -> {
var movieNode = r.get("m").asNode();
var directors = r.get("directors").asList(v -> {
var personNode = v.asNode();
return new Person(personNode.get("born").asInt(), personNode.get("name").asString());
});
var actors = r.get("actors").asList(v -> {
return new Actor(v.get("name").asString(), v.get("roles").asList(Value::asString));
});
var m = new Movie(movieNode.get("title").asString(), movieNode.get("tagline").asString());
m.setReleased(movieNode.get("released").asInt());
m.setDirectorss(directors);
m.setActors(actors);
return m;
}));
}
} AND voila! Helidon and Neo4j can now work together.
What about MP?
We just need to wrap our integration in a CDI extension – it’s really quite simple:
public class Neo4jCdiExtension implements Extension {
private static final String NEO4J_METRIC_NAME_PREFIX = "neo4j";
void afterBeanDiscovery(@Observes AfterBeanDiscovery addEvent) {
addEvent.addBean()
.types(Driver.class)
.qualifiers(Default.Literal.INSTANCE, Any.Literal.INSTANCE)
.scope(ApplicationScoped.class)
.name(Driver.class.getName())
.beanClass(Driver.class)
.createWith(creationContext -> {
org.eclipse.microprofile.config.Config config = ConfigProvider.getConfig();
Config helidonConfig = MpConfig.toHelidonConfig(config).get(NEO4J_METRIC_NAME_PREFIX);
ConfigValue<Neo4j> configValue = helidonConfig.as(Neo4j::create);
if (configValue.isPresent()) {
return configValue.get().driver();
}
throw new Neo4jException("There is no Neo4j driver configured in configuration under key 'neo4j");
});
}
} As you can see, we can read the configuration using SE functions:
Config helidonConfig = MpConfig.toHelidonConfig(config).get(NEO4J_METRIC_NAME_PREFIX); Then we just reuse our Neo4j SE support object:
ConfigValue<Neo4j> configValue = helidonConfig.as(Neo4j::create);… and return the driver:
return configValue.get().driver();And that’s actually all – just inject the driver:
@Inject
public MovieRepository(Driver driver) {
this.driver = driver;
} The configuration will be taken from the file microprofile-config.properties:
# Neo4j settings
neo4j.uri=bolt://localhost:7687
neo4j.authentication.username=neo4j
neo4j.authentication.password: secret
neo4j.pool.metricsEnabled: true Then you can use the driver as in the SE example.
Now Helidon MP can work with Neo4j too!
But that is not all!
Metrics
As I already mentioned, for proper operation in the clouds, we also need to forward metrics and health checks from Neo4j.
Let’s do it as before – first for SE, and then wrap everything in MP.
Let’s start with metrics! Let’s make a separate module for Neo4j metrics.
As with Helidon SE, in the Neo4j support object we will follow the Builder pattern to set up metrics support. In fact, we only need the Neo4j driver, since we can get all the indicators from it:
public static class Builder implements io.helidon.common.Builder<Neo4jMetricsSupport> {
private Driver driver;
private Builder() {
}
public Neo4jMetricsSupport build() {
Objects.requireNonNull(driver, "Must set driver before building");
return new Neo4jMetricsSupport(this);
}
public Builder driver(Driver driver) {
this.driver = driver;
return this;
}
} Then we have to wrap the counters and gauges of Neo4j:
private static class Neo4JCounterWrapper implements Counter {
private final Supplier<Long> fn;
private Neo4JCounterWrapper(Supplier<Long> fn) {
this.fn = fn;
}
@Override
public void inc() {
throw new UnsupportedOperationException();
}
@Override
public void inc(long n) {
throw new UnsupportedOperationException();
}
@Override
public long getCount() {
return fn.get();
}
}
private static class Neo4JGaugeWrapper<T> implements Gauge<T> {
private final Supplier<T> supplier;
private Neo4JGaugeWrapper(Supplier<T> supplier) {
this.supplier = supplier;
}
@Override
public T getValue() {
return supplier.get();
}
} Next, you need to register these counters with the MetricsRegistry:
private void registerCounter(MetricRegistry metricRegistry,
ConnectionPoolMetrics cpm,
String poolPrefix,
String name,
Function<ConnectionPoolMetrics, Long> fn) {
String counterName = poolPrefix + name;
if (metricRegistry.getCounters().get(new MetricID(counterName)) == null) {
Metadata metadata = Metadata.builder()
.withName(counterName)
.withType(MetricType.COUNTER)
.notReusable()
.build();
Neo4JCounterWrapper wrapper = new Neo4JCounterWrapper(() -> fn.apply(cpm));
metricRegistry.register(metadata, wrapper);
}
}
private void registerGauge(MetricRegistry metricRegistry,
ConnectionPoolMetrics cpm,
String poolPrefix,
String name,
Function<ConnectionPoolMetrics, Integer> fn) {
String gaugeName = poolPrefix + name;
if (metricRegistry.getGauges().get(new MetricID(gaugeName)) == null) {
Metadata metadata = Metadata.builder()
.withName(poolPrefix + name)
.withType(MetricType.GAUGE)
.notReusable()
.build();
Neo4JGaugeWrapper<Integer> wrapper =
new Neo4JGaugeWrapper<>(() -> fn.apply(cpm));
metricRegistry.register(metadata, wrapper);
}
} And we are almost ready:
private void reinit() {
Map<String, Function<ConnectionPoolMetrics, Long>> counters = Map.ofEntries(
entry("acquired", ConnectionPoolMetrics::acquired),
entry("closed", ConnectionPoolMetrics::closed),
entry("created", ConnectionPoolMetrics::created),
entry("failedToCreate", ConnectionPoolMetrics::failedToCreate),
entry("timedOutToAcquire", ConnectionPoolMetrics::timedOutToAcquire),
entry("totalAcquisitionTime", ConnectionPoolMetrics::totalAcquisitionTime),
entry("totalConnectionTime", ConnectionPoolMetrics::totalConnectionTime),
entry("totalInUseCount", ConnectionPoolMetrics::totalInUseCount),
entry("totalInUseTime", ConnectionPoolMetrics::totalInUseTime));
Map<String, Function<ConnectionPoolMetrics, Integer>> gauges = Map.ofEntries(
entry("acquiring", ConnectionPoolMetrics::acquiring),
entry("creating", ConnectionPoolMetrics::creating),
entry("idle", ConnectionPoolMetrics::idle),
entry("inUse", ConnectionPoolMetrics::inUse)
);
for (ConnectionPoolMetrics it : lastPoolMetrics.get()) {
String poolPrefix = NEO4J_METRIC_NAME_PREFIX + "-";
counters.forEach((name, supplier) -> registerCounter(metricRegistry.get(), it, poolPrefix, name, supplier));
gauges.forEach((name, supplier) -> registerGauge(metricRegistry.get(), it, poolPrefix, name, supplier));
// we only care about the first one
metricsInitialized.set(true);
break;
}
} It’s always a good idea to update metrics in time, and for that we have a function:
private void refreshMetrics(ScheduledExecutorService executor) {
Collection<ConnectionPoolMetrics> currentPoolMetrics = driver.metrics().connectionPoolMetrics();
if (!metricsInitialized.get() && currentPoolMetrics.size() >= 1) {
lastPoolMetrics.set(currentPoolMetrics);
reinit();
if (metricsInitialized.get()) {
reinitFuture.get().cancel(false);
executor.shutdown();
}
}
} We only need to provide the Neo4j driver to provide us with the metrics information in our application.yaml file:
neo4j:
pool:
metricsEnabled: true … Or in our microprofile-config.properties :
neo4j.pool.metricsEnabled = true Now if you go to “/ health” you will also get readings from Neo4j.
For MP, we only need to wrap the metrics registration as a CDI extension. This event should happen after the driver has already been initialized:
public class Neo4jMetricsCdiExtension implements Extension {
private void addMetrics(@Observes @Priority(PLATFORM_AFTER + 101) @Initialized(ApplicationScoped.class) Object event) {
Instance<Driver> driver = CDI.current().select(Driver.class);
Neo4jMetricsSupport.builder()
.driver(driver.get())
.build()
.initialize();
}
} That’s all! Neo4j metrics are now available in Helidon MP!
And now, last but not least, a health check!
Health checks
And again, this should be a separate module to keep everything clean and beautiful.
This time we’ll start with MP:
@Readiness
@ApplicationScoped
public class Neo4jHealthCheck implements HealthCheck {
private static final String CYPHER = "RETURN 1 AS result";
private static final SessionConfig DEFAULT_SESSION_CONFIG = SessionConfig.builder()
.withDefaultAccessMode(AccessMode.WRITE)
.build();
private final Driver driver;
@Inject
//will be ignored outside of CDI
Neo4jHealthCheck(Driver driver) {
this.driver = driver;
}
public static Neo4jHealthCheck create(Driver driver) {
return new Neo4jHealthCheck(driver);
}
private static HealthCheckResponse buildStatusUp(ResultSummary resultSummary, HealthCheckResponseBuilder builder) {
ServerInfo serverInfo = resultSummary.server();
builder.withData("server", serverInfo.version() + "@" + serverInfo.address());
String databaseName = resultSummary.database().name();
if (!(databaseName == null || databaseName.trim().isEmpty())) {
builder.withData("database", databaseName.trim());
}
return builder.build();
}
@Override
public HealthCheckResponse call() {
HealthCheckResponseBuilder builder = HealthCheckResponse.named("Neo4j connection health check").up();
try {
ResultSummary resultSummary;
// Retry one time when the session has been expired
try {
resultSummary = runHealthCheckQuery();
} catch (SessionExpiredException sessionExpiredException) {
resultSummary = runHealthCheckQuery();
}
return buildStatusUp(resultSummary, builder);
} catch (Exception e) {
return builder.down().withData("reason", e.getMessage()).build();
}
}
private ResultSummary runHealthCheckQuery() {
// We use WRITE here to make sure UP is returned for a server that supports
// all possible workloads
if (driver != null) {
Session session = this.driver.session(DEFAULT_SESSION_CONFIG);
Result run = session.run(CYPHER);
return run.consume();
}
return null;
}
} Technically, for a health check, we run a simple request for Cypher, and if it works, then Neo4j is alive, that’s enough!
We only need to add the Maven dependency to our project:
<dependency>
<groupId>io.helidon.integrations.neo4j</groupId>
<artifactId>helidon-integrations-neo4j-health</artifactId>
</dependency> And again – Voila everything works!
As for SE, since it’s pure Java, we just need to initialize everything:
Neo4j neo4j = Neo4j.create(config.get("neo4j"));
Neo4jHealthCheck healthCheck = Neo4jHealthCheck.create(neo4j.driver());
Driver neo4jDriver = neo4j.driver();
HealthSupport health = HealthSupport.builder()
.addLiveness(HealthChecks.healthChecks()) // Adds a convenient set of checks
.addReadiness(healthCheck)
.build();
return Routing.builder()
.register(health) // Health at "/health"
//other services
.build();
} Now our Helidon application, both MP and SE, can work with Neo4j, read its metrics and perform health checks.
By the way, since the Neo4j driver fully supports GraalVM native-image, Helidon MP or SE applications can be compiled into it!
This article has demonstrated how to create a multi-technology integration with Helidon. As for Neo4j, we’ve already officially done it for you!
You just need to include the following dependencies in your Maven project:
<dependency>
<groupId>io.helidon.integrations.neo4j</groupId>
<artifactId>helidon-integrations-neo4j</artifactId>
</dependency>
<dependency>
<groupId>io.helidon.integrations.neo4j</groupId>
<artifactId>helidon-integrations-neo4j-metrics</artifactId>
</dependency>
<dependency>
<groupId>io.helidon.integrations.neo4j</groupId>
<artifactId>helidon-integrations-neo4j-health</artifactId>
</dependency> … And that’s all you need to get started with Helidon and Neo4j!
Conclusion
As you can see, the integration with Helidon is pretty straightforward. The standard way to do this is to first write a support Helidon SE object, follow the Builder pattern to initialize it, and then just wrap it in a CDI extension so MicroProfile can take advantage of this “magic”!
You can play with the Helidon in Neo4j examples in our official Helidon Neo4j integration repositories …
As for Neo4j, you might also be interested in CypherDsl and example with him.
Next steps
In this article, I wanted to show you not only how Neo4j works with Helidon, but also how you can write your own extensions. Since Helidon is an open source product, we invite you to contribute to it!
