5

I want to create two separate JDBC data-sources (aka. connection pools) that would re use connections similar to following diagram.

                .------.
                '------'
                | Data |
                | base |
                '------'
                   ^
                   |
            +------+-------+
            |              |
            |  Connection  |
       +--->|  Pool Main   |<----+
       |    |              |     |
       |    +--------------+     |
+------+----------+   +----------+--------+
|Connection Pool A|   | Connection Pool B |
+-----------------+   +-------------------+
         ^                       ^
         |                       |
 +-------+------+       +--------+-----+
 | App Module A |       | App Module B |
 +--------------+       +--------------+

Idea is to have parent connection pool to have 10 connections limit, and each child source to have limit of 8.

So even if App module A uses all 8 connections, module B won't starve and would have at least two connection.

So to reword the question: I want to represent single connection pool in such way, that one part of application can have at least N connections of total M available. Reserving this N connections for each part of application.

Is this possible with any existing library, for example HikariCP?

P.S. This is spring boot application, but this should not matter as JDBC datasources can exist outside of spring. But perhaps Spring has some tricks for this use-case.

2 Answers 2

5
+50

Based on SpringBoot and HikariCP, try the following:

Create DataSourceConfig class to create two separate HikariDataSource instances, each representing sub-connection pool A & B.

Each data source below configured with their respective max connection limits (8 for the apps) and will use the main connection pool as their underlying source with its own max connection limit (10 in this case).

@Configuration
public class DataSourceConfig {
    
    @Bean("mainDataPoolSource")
    @Primary
    public DataSource mainDataPoolSource() {
        HikariConfig config = new HikariConfig();
        config.setMaximumPoolSize(10);
        // Configure other Main Connection Pool properties such as JDBC URL, username, password, etc.
        return new HikariDataSource(config);
    }
    
    @Bean("dataPoolSourceA")
    public DataSource dataPoolSourceA(@Qualifier("mainDataPoolSource") DataSource mainDataPoolSource) {
        HikariConfig config = new HikariConfig();
        config.setMaximumPoolSize(8);
        // Configure other properties for App Module A
        config.setDataSource(mainDataPoolSource);
        return new HikariDataSource(config);
    }
    
    @Bean("dataPoolSourceB")
    public DataSource dataPoolSourceB(@Qualifier("mainDataPoolSource") DataSource mainDataPoolSource) {
        HikariConfig config = new HikariConfig();
        config.setMaximumPoolSize(8);
        // Configure other properties for App Module B
        config.setDataSource(mainDataPoolSource);
        return new HikariDataSource(config);
    }
}

To assign each module to a specific data source with Spring's dependency injection mechanism, the example below uses @Autowired and @Qualifier annotations for App Module A.

@Service
public class AppModuleAService {
    private final DataSource dataPoolSourceA;

    @Autowired
    public AppModuleAService(@Qualifier("dataPoolSourceA") DataSource dataPoolSourceA) {
        this.dataPoolSourceA = dataPoolSourceA;
    }
    // Use the dataPoolSourceA in your module A service methods
}

Repeat for App Module B

4
  • 1
    Wow! That's ingenious! Don't know why I haven't thought about it. I have locked my mind into thinking that it only accepts string as paramter. But need to provide some confgs, to return connections back to main soruce ASAP. I'll test it. Thanks. Commented May 23, 2023 at 11:04
  • turns out that both pools requests 5 connections proactively. Commented May 24, 2023 at 20:19
  • Unfortunatuly that is not completely what I wanted. with some fiddling with config I was able to make secondary pools to not capture many connections proactively, but thy stil don't release them before 30 secons pass, and this is minimam value I can set. But here is config I've used: config.setMinimumIdle(0); to not capture 5 connection at start up. And following two configs are overridden by defaults config.setIdleTimeout(1); config.setMaxLifetime(1); . There is logic inside Hikary that resets it to minimum value of 30_000. Commented May 24, 2023 at 20:21
  • and even If I alter maxLifetime after pool is started. my unit tests verify that I have to wait 30+ seconds for connection to become available to second pool. I guess I shall try other pool than Hikary. Or perhaps would have to resort to implementing DataSource manually. But first would try another connection pool. Anyways thanks for providing direction. Commented May 24, 2023 at 20:24
-1
+50

Just to add to what djmonki said. To ensure that the child pools do not proactively capture connections at startup, you could set the minimum idle size to 0 for both child pools. This allows the connections to be allocated dynamically based on demand.

By configuring the child pools to use the main connection pool as their base source, you’d be able to reserve a certain number of connections (N) for each part of the application. It ensures that even if one part of the application uses all its allocated connections, the other part won’t be starved and will have at least two connections available. Hope it helps! Open to a better iteration

@Configuration
public class DataSourceConfig {
    
    @Bean("mainDataPoolSource")
    @Primary
    public DataSource mainDataPoolSource() {
        HikariConfig config = new HikariConfig();
        config.setMaximumPoolSize(10);
        // Configure other Main Connection Pool properties such as JDBC URL, username, password, etc.
        return new HikariDataSource(config);
    }
    
    @Bean("dataPoolSourceA")
    public DataSource dataPoolSourceA(@Qualifier("mainDataPoolSource") DataSource mainDataPoolSource) {
        HikariConfig config = new HikariConfig();
        config.setMaximumPoolSize(8);
        config.setMinimumIdle(0); // Avoid capturing connections proactively
        // Configure other properties for App Module A
        config.setDataSource(mainDataPoolSource);
        return new HikariDataSource(config);
    }
    
    @Bean("dataPoolSourceB")
    public DataSource dataPoolSourceB(@Qualifier("mainDataPoolSource") DataSource mainDataPoolSource) {
        HikariConfig config = new HikariConfig();
        config.setMaximumPoolSize(8);
        config.setMinimumIdle(0); // Avoid capturing connections proactively
        // Configure other properties for App Module B
        config.setDataSource(mainDataPoolSource);
        return new HikariDataSource(config);
    }
} 
2
  • You just ripped of other answer. And it don't address all problems of that answer :-( Commented May 25, 2023 at 9:42
  • Hmm, although it did have a slight variation. config.setMinimumIdle(0); // Avoid capturing connections proactively. By setting a minimum idle property, connections are not kept open when they are not in use. But i guess back to the drawing board on this one
    – Dev-ohene
    Commented May 25, 2023 at 11:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.