JPA Entity. Load me not completely
JPA is often criticized for not being able to partially load entities, which is actually a big misconception. Spring Data JPA and Hibernate include many tools for partially loading entities.
Team Spring AIO prepared an article in which I reviewed the tools available in Spring Data JPA for partial loading of entities, and also analyzed their features and corner cases. Let's try to consider all the methods of such partial loading of entities using the example of the main methods of interaction with Hibernate in Spring applications:
Spring Data JPA
EntityManager
Criteria API
There is also a project Jakarta Datawhich is actively developing. However, at the time of writing this article, only the first stable version 1.0.0 has been released. We will not consider it until we accumulate a sufficient number of real uses.
Is this issue of interest to the community?
After yet another holy war in one of the Telegram channels about Java on the topic of how bad JPA and Hibernate in particular are, how it executes queries in an unoptimized manner and how much extra data is loaded, I decided to delve a little deeper into these issues and try to defend the above-mentioned stack by going on a trip to Stackoverflow. Let's do a search by tag [spring-data-jpa] and sort the questions by popularity. We will see that the question Spring JPA selecting specific columns is in sixth place.
In this article, we will try to answer this question as broadly as possible: we will consider not only the simplest cases with basic attributes, but also dive into the world of JPA associations.
Task
IN project the following data model will be considered:
Our task is to load several basic fields + ToOne (in our case, this is the author attribute, referring to the Post entity) of the association for each partial loading method. Suppose we want to get all articles whose title contains some text, and the search is not case-sensitive, i.e. contains with ignore case. So, we unload Post: id, slug, title; User(author): id, username.
We will check the result in the appropriate teststhe resulting queries will be visible in the console: a log that will display the SQL query generated by Hibernate.
Test data
Let's create two records Postinitializing the basic fields and linking the authors. The service in which data is created and deleted InitTestDataService.
toOne
In total, 21 partial loading methods were found for the task we set.
These methods include different approaches to writing a query:
Test class where you can see all tests with comments – ToOneTest.
Preface
It's worth clarifying a little before our experiment. Derived methods are methods that are automatically implemented by the framework based on their names, i.e. without explicitly specifying an annotation @Query. Since we don't specify the query explicitly, and therefore we don't have the ability to specify which attributes we want to load, we only have one way to specify specific attributes to load – this is projection.
The projections themselves are of two types: interface based (Interface-based Projections) and in classes (Class-based Projections). Interface-based Projections in turn can be divided into open and closed.
IN closed In projections, getters are declared explicitly:
interface NamesOnly {
String getFirstname();
String getLastname();
}In case open projections The value of interface getters can be calculated based on the SpEL expression:
interface NamesOnly {
@Value("#{target.firstname + ' ' + target.lastname}")
String getFullName();
} We will not consider them, because The documentation clearly statesthat query optimization will not be performed for them:
Spring Data cannot apply query execution optimizations in this case, because the SpEL expression could use any attribute of the aggregate root.
For download ToOne-associations we set the task of checking two options with flatten (flat) attributes, with nested (nested) class, as well as with Tuple And Map.
For ToMany it makes sense to test the work with flat attributes, but in this case the loading will be with complexity n*m. This means that our experiment is suitable for both ToOne and ToMany, but records in the case of ToMany more will be unloaded.
Almost everything we will consider for ToOne is also true for ToMany, however, Hibernate is not able to map collection attributes to DTO/Projection, due to which Hibernate can only execute HQL, and for the case with ToMany, the Cartesian product n*m will be returned. In other words, in addition to unloading and mapping the result, we will also have to collapse duplicate records. If this topic is interesting, we will definitely write an additional post about partial loading and ToMany associations. However, you can find several examples in project in the test class ToManyTest.
Everything that will be discussed below is also perfect for Embedded-case, so it makes no sense to consider it separately
Derived methods
Interface-based flat projection
For this case, we will use a separate interface as a projection. Perhaps, this approach can be attributed to the basic concept in the context of using projections.
Let's declare a method in our repository:
public interface PostRepository extends JpaRepository<Post, Long> {
<T> List<T> findAllByTitleContainsIgnoreCase(String title, Class<T> projection);
}For convenience purposes only, the method will be tailored to dynamic projectionso as not to write a separate method for each projection.
Let's create a projection class:
public interface PostWithAuthorFlat {
Long getId();
String getSlug();
String getTitle();
Long getAuthorId();
String getAuthorUsername();
}Let's test the solution by writing a test. Let's try to delve a little deeper into the projection's operation using debug. Let's set a breakpoint on our test:
The final link in obtaining data is the work of the TupleBackedMap class, the Tuple object itself will contain the information we need. To see the chain of obtaining data, we will mark the boundary by setting a breakpoint in the method of obtaining a value from TupleBackedMap (org.springframework.data.jpa.repository.query.AbstractJpaQuery.TupleConverter.TupleBackedMap):
And we observe the following chain of data acquisition:
The received proxy will first reach the method invoke class MapAccessingMethodInterceptor:
As a result, we get an object of the class Accessorwhich provides access to the very propertyName. The converted string will be passed to TupleBackedMap. Next, the obtained value will be presented to us from the object familiar to us all Tuple:
The object itself Tuple in turn gives us full access to the key and value.
The whole scheme can be described as follows:
Under the projection lies a proxy, in the proxy lies a proxy, in the proxy lies a certain target object, which is an object TupleBackedMap.
Or like this:
“On the sea in the ocean there is an island, on that island there is an oak tree, under the oak tree there is a chest buried, in the chest there is a hare, in the hare there is a duck, in the duck there is an egg” There is a needle in the egg – the death of Koshchei!
To sum up this case, we can conclude that Spring provides a converter that maps itself Tuple to the projection. This approach is perfect for solving the problem we have set, because the query contains only those columns that are specified in the projection:
Hibernate:
select
p1_0.id,
p1_0.slug,
p1_0.title,
p1_0.author_id,
a1_0.username
from
posts p1_0
left join
users a1_0
on a1_0.id=p1_0.author_id
where
upper(p1_0.title) like upper(?) escape '\'Interface-based nested interface projections
The next way to solve our problem could be a solution based on using a projection that has another projection inside itself to get data about the author of the post. This is the very nested case.
public interface PostWithAuthorNested {
Long getId();
String getSlug();
String getTitle();
UserPresentation getAuthor();
}For basic fields, only what is specified in the projection will be loaded. We will get the ones we need id, slug, title. However, for a nested object, absolutely all fields will be loaded, which of course contradicts our requirements. The problem is known and even has official response.
It is worth noting that PostWithAuthorNested it's all the same proxy around TupleBackedMapand here is the nested object itself UserPresentation is a proxy directly around the entity itself User:
Hibernate:
select
p1_0.id,
p1_0.slug,
p1_0.title,
a1_0.id,
a1_0.bio,
a1_0.email,
a1_0.image,
a1_0.password,
a1_0.token,
a1_0.username
from
posts p1_0
left join
users a1_0
on a1_0.id=p1_0.author_id
where
upper(p1_0.title) like upper(?) escape '\'To sum up: the approach does not work optimally and does not solve the problem we set.
Class-based flat projections
Let's move on and the next option for solving the problem of partial unloading of entity fields can be noted as the use of a separate projection Record-class (repository method remains the same)
public record PostWithAuthorFlatDto(Long id,
String slug,
String title,
Long authorId,
String authorUsername) {
}The test itself:
@Test
void derivedMethodClassFlatPrj() {
var posts = postRepository.findAllByTitleContainsIgnoreCase(
"spring",
PostWithAuthorFlatDto.class
);
assertEquals(1, posts.size());
var postFirst = posts.getFirst();
assertEquals(POST1_SLUG, postFirst.slug());
assertEquals(POST1_AUTHOR_NAME, postFirst.authorUsername());
}Spring Data JPA passes the result type to Hibernate, which in turn performs mapping. Spring is only responsible for the correct formation JPQL request. This option suits us completely.
Result:
Hibernate:
select
p1_0.id,
p1_0.slug,
p1_0.title,
p1_0.author_id,
a1_0.username
from
posts p1_0
left join
users a1_0
on a1_0.id=p1_0.author_id
where
upper(p1_0.title) like upper(?) escape '\'Class-based nested dto projections
We are moving at a good pace, but it would be worth considering negative, non-working scenarios to better understand the logic of working with projections. Let's create such a record class with a nested DTO projection:
public record PostWithAuthorNestedDto(Long id,
String slug,
String title,
UserPresentationDto author) {
}It seems like everything should work, but when we try to “take off” we get an error:
Cannot set field 'author' to instantiate 'io.spring.jpa.projection.PostWithAuthorNestedDto'What are we about? warned:
In this case, all the logic lies on the Hibernate side, Spring Data has no influence here.
Class-based nested entity projections
We can also specify a whole entity for record. However, this method obliges us to get all the fields from the nested entity. We will have to add another negative case to the piggy bank. Not our option, but it seems useful to know about it:
public record PostWithAuthorEntity(Long id,
String slug,
String title,
User author) {
}We get:
Hibernate:
select
p1_0.id,
p1_0.slug,
p1_0.title,
a1_0.id,
a1_0.bio,
a1_0.email,
a1_0.image,
a1_0.password,
a1_0.token,
a1_0.username
from
posts p1_0
left join
users a1_0
on a1_0.id=p1_0.author_id
where
upper(p1_0.title) like upper(?) escape '\'Query methods
Now let's look at a method where we can explicitly specify in the request which attributes we need to unload, i.e. Query-methods. In this case, we immediately solve one important problem – we clearly indicate what we want to get. In our area of responsibility, it remains only to correctly implement the mapping.
Interface-based flat projections
Spring is rich in its capabilities, so it can, for example, handle JPQL which we wrote ourselves, and then formed completely TupleBackedMap.
This method is interesting because nothing prevents us from writing a regular one JPQL query, and specify the projection as the return value:
@Query("""
select a.id as id,
a.slug as slug,
a.title as title,
a.author.id as authorId,
a.author.username as authorUsername
from Post a
where lower(a.title) like lower(concat('%', ?1, '%'))""")
List<PostWithAuthorFlat> findAllPostWithAuthorFlat(String title);The projection itself:
public interface PostWithAuthorFlat {
Long getId();
String getSlug();
String getTitle();
Long getAuthorId();
String getAuthorUsername();
}The key moment of formation JPQL-request is a requirement for specifying aliases: they must be the same as the properties in the projection, otherwise the magic of mapping will not happen. By the way, as expected, under the interface lies the same proxy with TupleBackedMap.
The result itself:
Hibernate:
select
p1_0.id,
p1_0.slug,
p1_0.title,
p1_0.author_id,
a1_0.username
from
posts p1_0
join
users a1_0
on a1_0.id=p1_0.author_id
where
lower(p1_0.title) like lower(('%'||?||'%')) escape ''Class-based flat projections
This method of partial data retrieval is basic for Hibernate. You just need to create a projection class with a constructor. Then we initialize this class directly in JPQL. In this case, we do not need aliases. In this case, a proxy is also not created, only a DTO projection is used.
@Query("""
select
new io.spring.jpa.projection.PostWithAuthorFlatDto(
a.id,
a.slug,
a.title,
a.author.id,
a.author.username
)
from Post a
where lower(a.title) like lower(concat('%', ?1, '%'))""")
List<PostWithAuthorFlatDto> findAllPostWithAuthorFlatDto(String title);Projection:
public record PostWithAuthorFlatDto(Long id,
String slug,
String title,
Long authorId,
String authorUsername) {
}Test:
@Test
void queryMethodClassFlat() {
var posts = postRepository.findAllPostWithAuthorFlatDto("spring");
assertEquals(1, posts.size());
PostWithAuthorFlatDto post = posts.getFirst();
assertEquals(POST1_SLUG, post.slug());
assertEquals(POST1_AUTHOR_NAME, post.authorUsername());
}Success:
Hibernate:
select
p1_0.id,
p1_0.slug,
p1_0.title,
p1_0.author_id,
a1_0.username
from
posts p1_0
join
users a1_0
on a1_0.id=p1_0.author_id
where
lower(p1_0.title) like lower(('%'||?||'%')) escape ''Class-based nested class projections
An important feature of the method using nested projections is that you can make as many nestings as you want. Since we are in HQL, we can initialize our DTO as we like. Including creating new DTO objects inside DTO.
Let's write such a method:
@Query("""
select
new io.spring.jpa.projection.PostWithAuthorNestedDto(
a.id,
a.slug,
a.title,
new io.spring.jpa.projection.UserPresentationDto(
a.author.id,
a.author.username
)
)
from Post a
where lower(a.title) like lower(concat('%', ?1, '%'))""")
List<PostWithAuthorNestedDto> findAllPostWithAuthorNestedDto(String title);And also a couple of projections for it: a projection for the post itself, inside which we declare a projection for the author.
public record PostWithAuthorNestedDto(Long id,
String slug,
String title,
UserPresentationDto author) {
}Projection for the author:
public record UserPresentationDto(Long id, String username) {
}Let's write a simple test:
@Test
void queryMethodClassNested() {
var posts = postRepository.findAllPostWithAuthorNestedDto("spring");
assertEquals(1, posts.size());
PostWithAuthorNestedDto post = posts.getFirst();
assertEquals(POST1_SLUG, post.slug());
assertEquals(POST1_AUTHOR_NAME, post.author().username());
}Direct request:
Hibernate:
select
p1_0.id,
p1_0.slug,
p1_0.title,
p1_0.author_id,
a1_0.username
from
posts p1_0
join
users a1_0
on a1_0.id=p1_0.author_id
where
lower(p1_0.title) like lower(('%'||?||'%')) escape ''Tuple, Object[]List<>, Map
Hibernate also allows us to return select expressions, which we can write as Object[], Tuple, Map, List. We will not dwell on each one in detail, the only difference is in the return value:
@Query("""
select
a.id as id,
a.slug as slug,
a.title as title
from Post a
where lower(a.title) like lower(concat('%', ?1, '%'))""")
List<Tuple> findAllTupleBasic(String title);@Query("""
select
a.id as id,
a.slug as slug,
a.title as title
from Post a
where lower(a.title) like lower(concat('%', ?1, '%'))""")
List<Object[]> findAllObjectWithAuthor(String title);@Query("""
select
a.id as id,
a.slug as slug,
a.title as title
from Post a
where lower(a.title) like lower(concat('%', ?1, '%'))""")
List<List<?>> findAllListWithAuthor(String title);@Query("""
select
a.id as id,
a.slug as slug,
a.title as title
from Post a
where lower(a.title) like lower(concat('%', ?1, '%'))""")
List<Map<String, Object>> findAllMapWithAuthor(String title);PS: for Derived-methods can similarly specify the same return values. Spring Data will take care of everything and will unload all fields.
Entity Manager
All methods described for @Querywork in Entity Managerexcept Interface-based projectionsince Interface-based projection is a concept of Spring itself, and Hibernate itself knows nothing about it.
Criteria API
Criteria API is used to create type-safe and flexible database queries in Java. The Criteria API itself is a “type-safe alternative to HQL”
First, we need to specify the attributes that we are going to pass to the request. We can very conveniently specify the attribute name using a constant that was generated automatically using the dependency:
dependencies {
annotationProcessor 'org.hibernate:hibernate-jpamodelgen:{version}'
}var idPath = owner.<Long>get(Post_.ID);
var slugPath = owner.<String>get(Post_.SLUG);
var titlePath = owner.<String>get(Post_.TITLE);
var authorIdPath = owner.get(Post_.AUTHOR).<Long>get(User_.ID);
var authorUsernamePath = owner.get(Post_.AUTHOR).<String>get(User_.USERNAME);The latest Hibernate documentation uses this method in all examples, which suggests that it is a “silent” recommendation.
As a result, the constants will always be regenerated and will always be up to date, as a result of which we gain in the security and reliability of the application.
DTO
So, we have declared the paths, all that remains is to write a query, throw the result into the collection, and then make sure that the approach is viable:
@Test
void criteriaDto() {
var cb = em.getCriteriaBuilder();
var query = cb.createQuery(PostWithAuthorFlatDto.class);
var owner = query.from(Post.class);
var idPath = owner.<Long>get(Post_.ID);
var slugPath = owner.<String>get(Post_.SLUG);
var titlePath = owner.<String>get(Post_.TITLE);
var authorIdPath = owner.get(Post_.AUTHOR).<Long>get(User_.ID);
var authorUsernamePath = owner.get(Post_.AUTHOR).<String>get(User_.USERNAME);
query.multiselect(idPath, slugPath, titlePath, authorIdPath, authorUsernamePath)
.where(cb.like(cb.lower(titlePath), "%spring%"));
var resultList = em.createQuery(query).getResultList();
for (PostWithAuthorFlatDto post : resultList) {
assertEquals(POST1_SLUG, post.slug());
assertEquals(POST1_AUTHOR_NAME, post.authorUsername());
}
}Projection:
public record PostWithAuthorFlatDto(Long id,
String slug,
String title,
Long authorId,
String authorUsername) {
}In this case, we return a set of elements (attributes) and map them to the DTO that we specify when creating the query
jakarta.persistence.criteria.CriteriaBuilder#createQuery(java.lang.Class)Successful success:
Hibernate:
select
p1_0.id,
p1_0.slug,
p1_0.title,
p1_0.author_id,
a1_0.username
from
posts p1_0
join
users a1_0
on a1_0.id=p1_0.author_id
where
lower(p1_0.title) like ? escape ''The only downside to working through DTO is that there is still a risk of accidentally refactoring the DTO. In this case, we will catch an error that, for example, a suitable constructor was not found. To solve this problem, let's consider the option of using Tuple.
Tuple
In the case of Tuple the object initialization occurs manually and we simply will not be able to create the object itself and, of course, we will catch the error even before the application is launched at the compilation stage.
Here is perhaps the safest way to perform partial queries:
@Test
void criteriaTuple() {
var cb = em.getCriteriaBuilder();
var query = cb.createTupleQuery();
var owner = query.from(Post.class);
var idPath = owner.<Long>get(Post_.ID);
var slugPath = owner.<String>get(Post_.SLUG);
var titlePath = owner.<String>get(Post_.TITLE);
var authorIdPath = owner.get(Post_.AUTHOR).<Long>get(User_.ID);
var authorUsernamePath = owner.get(Post_.AUTHOR).<String>get(User_.USERNAME);
query.select(cb.tuple(idPath, slugPath, titlePath, authorIdPath, authorUsernamePath))
.where(cb.like(cb.lower(titlePath), "%spring%"));
var resultList = em.createQuery(query).getResultList().stream()
.map(tuple -> new PostWithAuthorNestedDto(
tuple.get(idPath),
tuple.get(slugPath),
tuple.get(titlePath),
new UserPresentationDto(
tuple.get(authorIdPath),
tuple.get(authorUsernamePath)
)
)).toList();
for (PostWithAuthorNestedDto post : resultList) {
assertEquals(POST1_SLUG, post.slug());
assertEquals(POST1_AUTHOR_NAME, post.author().username());
}
}The projections themselves:
public record PostWithAuthorNestedDto(Long id,
String slug,
String title,
UserPresentationDto author) {
}public record UserPresentationDto(Long id, String username) {
}After forming the Path, we extract the values from Tuple and put it into our DTO (if no additional manipulations with the selection are required). Thus, we get not only a “type-safe alternative to HQL”, but also safe and controlled work with the result of our request. The concept is very similar to working with Tuple in the library QueryDSL.
As a result, we get a request that satisfies our requirements. The remaining examples for the Criteria API are discussed here here. Everything works in the most typical way in relation to the cases considered above.
You can see the results of all methods in repositories.
By the way, up to version 6 of Hibernate Criteria API-requests to entities were executed as follows:
Criteria API generated regular JPQL, Hibernate in turn analyzed it in accordance with the HQL grammar, and only then generated an SQL query.
Starting with Hibernate 6, the Criteria API is immediately converted to SQM:
You can read more about SQM here here.
Conclusions
If we write HQL/JPQL query, then we control the query and return only what we want. The only question is how to map.
If we write HQL/JPQL we can always return
TupleorMapand map from it to DTO.“Whether to use a repository derived method?” – everyone decides for themselves. In simple cases, HQL will be simple, in complex cases, the length of the method name will tend to go beyond our solar system. Within the framework of the tasks solved for this project, specifically for partial data unloading, the repository derived method looks the most unsafe. We cannot control the request, and HQL can change due to a change in DTO/Projection or the Entity itself.
When we work with
Tuplea very convenient solution is to use the hibernate-jpamodelgen library, which allows us to use auto-generated constants. In the latest Hibernate documentation, this method is used in all examples, we can say that this is a “silent” recommendation. Also, using these constants, it is easy to create jakarta.persistence.criteria.Path for the Criteria API.When we write a Query in Spring Data and use DTO, by default the expressions with DTO will be validated: both the types and the number of arguments in the constructor will be checked. And most importantly – no proxy magic.
Don't know what to return? Return
Tuple. It's very convenient.HQL + Class-Based Projection, aka DTO, aka select new class construction always works, including nested classes.
For ToMany, when unloading data as Projection/DTO/Tuple, we will have to solve the issue of merging duplicate data.
Join the Russian-speaking Spring Boot developer community on Telegram — Spring AIOto stay up to date with the latest news from the world of Spring Boot development and everything related to it.
We are waiting for everyone, join us!
