0

Recently, a JHipster 7.7.0 application started generating server side/database errors for ALL tables. The application was deployed in production, and users entered data without problem. Suppose there is an entity called 'Product', and users have added several entries:

|  ID  | Name        |
|------|-------------|
| 1001 | Corn flakes |
| 1002 | Baked beans |
| 1003 | Carrots     |
| 1004 | Apples      |
| 1005 | Watermelon  |

First thing I notice is the ID values start counting up from 1001. (This is new... in JHipster 6 ID values start at 1.)

After I made some improvements to the application code, I redeployed into production. A user attempts to add a new product, and the following error is generated in the server logs:

2022-03-24 12:54:43.775 ERROR 11277 --- [  XNIO-1 task-1] o.h.e.jdbc.batch.internal.BatchingBatch  : HHH000315: Exception executing batch [java.sql.BatchUpdateException: (conn=33) Duplicate entry '1001' for key 'PRIMARY'], SQL: insert into product (name, id) values (?, ?)
2022-03-24 12:54:43.776  WARN 11277 --- [  XNIO-1 task-1] o.h.engine.jdbc.spi.SqlExceptionHelper   : SQL Error: 1062, SQLState: 23000
2022-03-24 12:54:43.776 ERROR 11277 --- [  XNIO-1 task-1] o.h.engine.jdbc.spi.SqlExceptionHelper   : (conn=33) Duplicate entry '1001' for key 'PRIMARY'
2022-03-24 12:54:43.779 ERROR 11277 --- [  XNIO-1 task-1] o.z.problem.spring.common.AdviceTraits   : Internal Server Error

org.springframework.dao.DataIntegrityViolationException: could not execute batch; SQL [insert into product (name, id) values (?, ?)]; constraint [PRIMARY]; nested exception is org.hibernate.exception.ConstraintViolationException: could not execute batch
        at org.springframework.orm.jpa.vendor.HibernateJpaDialect.convertHibernateAccessException(HibernateJpaDialect.java:276)
        at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:233)
        at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:566)
        at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:743)
        at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:711)
        at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:654)
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:407)
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698)
        at com.mycompany.app.web.rest.ProductResource$$EnhancerBySpringCGLIB$$84c14d6d.createProduct(<generated>)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
...

Clearly, the ID counter restarted at 1001, found an entry (Corn flakes), and the database produced the error.

When the user attempts to save again, an almost identical error occurs but this time the ID value has incremented to 1002, which now conflicts with Baked beans, and the database throws another 'Duplicate entry' exception.

Repeated saves fail until the user gets past 1005 (Watermelon), and the save succeeds with 1006. (Subsequent saves into this table continue without error.)

Similar errors occur on all entity tables, and can only be resolved by repeatedly 'saving' until the ID has incremented past the last ID value already in that table.

This is a problem.

Investigation

I've been generating JHipster applications since version 6, and generally without issue.

I looked at an old JHipster 6 project and compared entities with those in new JHipster 7 projects, and noticed that there has been a shift in strategy to how an ID value is generated.

In JHipster 6 (and older, I presume) ID values are defined for each entity as follows:

@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;

Coupled with Liquibase changelog file entries for each entity (e.g. ..._added_entity_Product.xml):

<column name="id" type="bigint" autoIncrement="${autoIncrement}">
  <constraints primaryKey="true" nullable="false"/>
</column>

And the liquibase 00000000000000_initial_schema.xml file, contains (near the top):

<property name="autoIncrement" value="true"/>

In this case, the database tables created define the ID field as an AUTO_INCREMENT field, meaning the ID values and increments are managed by the database:

CREATE TABLE `product` {
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
  `name` varchar(255) DEFAULT NULL
} ENGINE=InnoDB DEFAULT CHARSET=utf8mb4

However, in JHipster 7, things have changed. For each entity, the ID property is defined as:

@Id
@GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "sequenceGenerator")
@SequenceGenerator(name = "sequenceGenerator")
@Column(name = "id")
private Long id;

Coupled with Liquibase changelog file entries (e.g. ..._added_entity_Product.xml):

<column name="id" type="bigint">
  <constraints primaryKey="true" nullable="false"/>
</column>

And the liquibase 00000000000000_initial_schema.xml file, contains (near the top):

<changeSet id="00000000000000" author="jhipster">
  <createSequence sequenceName="sequence_generator" startValue="1050" incrementBy="50"/>
</changeSet>

In this configuration, a sequence generator appears to be managing ID values and increments. Database tables no longer define the ID field as an AUTO_INCREMENT:

CREATE TABLE `product` {
  `id` bigint(20) NOT NULL,
  `name` varchar(255) DEFAULT NULL
} ENGINE=InnoDB DEFAULT CHARSET=utf8mb4

Temporary Fix

To get things working, I performed the following changes to my JHipster 7 project:

  1. Modified each domain POJO entity's ID property annotation:
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
  1. Modified each entity's liquibase changelog XML file (..._added_entity_Product, etc):
<column name="id" type="bigint" autoIncrement="${autoIncrement}">
  <constraints primaryKey="true" nullable="false"/>
</column>
  1. Modified the liquibase 00000000000000_initial_schema.xml file, by adding the following line near the top:
<property name="autoIncrement" value="true"/>

Now, my database is generated such that the each entity table ID values are defined as:

`id` bigint(20) NOT NULL AUTO_INCREMENT

This appears to be working for me, but I am not happy about this.

Questions

Ideally, I'd like to NOT change the JHipster 7 original configuration, and better understand how this ID sequence generation is managed. (I'm not sure tinkering with changelog files is the right way of fixing this, and have more faith in the JHipster team's database expertise than my own.)

  • How I can take control of this sequence generator, and instruct it pick up from whichever value is appropriate for each entity, particularly after a redeploy/update? Is it possible?

  • Can I instruct it to find the next available value in the database automatically without throwing errors and failing?

  • Why have the JHipster team made this change? Is there an advantage? Can someone point me to a technical resource so I can learn about this advantage?

Can anyone point me in the right direction, so I can learn and improve my understanding?

NOTE: In case it is not clear, the project suffering from this database issue was JHipster 7 from inception. It was NOT originally in JHipster 6 and upgraded to 7. The only reason I mention JHipster 6 is because previous projects using JHipster 6 did not experience this problem, so I was investigating what changed between versions of JHipster, particularly regarding handling of ID values.

Environment: JHipster 7.7.0 (Angular, monolithic), MariaDB 10.4, OpenJDK 16.0.2_7, OS Windows 10 Pro and openSUSE 15.2, Firefox 98.0.2 and Chrome 99.0.4844.84.

5
  • Why do you want to upgrade to JHipster 7 an application which is in production ? What are the benefits you expect versus upgrading yourself the libraries? Mar 26, 2022 at 18:34
  • @gaël-marziou Thanks so much for your comment! I apologize for my poor wording... The project I'm having problems with was JHipster 7 from inception (and was NOT upgraded). I only mention JHipster 6 because previous projects did not suffer from this issue... I'll update my question to make things more clear.
    – Mike Smith
    Mar 27, 2022 at 20:30
  • ok, then I don't understand your problem. Unless you inject some data manually or from liquibase CSV imports forcing the IDs, new rows should get new IDs from sequence and should not conflict with existing ones. Mar 28, 2022 at 8:32
  • I finally found a solution for this... and posted it here: stackoverflow.com/questions/71760804/…
    – Mike Smith
    Aug 31, 2022 at 3:58
  • 1
    It's a mistake... Thanks for pointing it out, I shall attempt to remove some. (Sorry for any confusion.... and THANKS for all your help.)
    – Mike Smith
    Sep 2, 2022 at 3:01

2 Answers 2

0

Sequences are better for performance and can be pre-allocated per batch. See https://vladmihalcea.com/mariadb-10-3-database-sequences/

This strategy was already used for other databases, as MariaDB introduced sequences in 10.3, the JHipster team decided to apply same strategy over all databases and as it was a breaking change they made it on a major release.

2
0
Solution:

@Entity
@Table(name = "jhi_todo")
public class Todo extends AbstractAuditingEntity<Long> implements Serializable {  
@Id
    @GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "sequenceGenerator")
    @SequenceGenerator(name = "sequenceGenerator002")
    private Long id;

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

  <changeSet id="00000000000000" author="jhipster">
        <createSequence sequenceName="sequence_generator002" startValue="1050" incrementBy="50"/>
    </changeSet>

    <!--
        JHipster core tables.
        The initial schema has the '00000000000001' id, so that it is over-written if we re-generate it.
    -->
    <changeSet id="00000000000001" author="jhipster">
        <createTable tableName="jhi_todo">
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.