What I Learned at Work this Week: Giving Hikari a Local DataSource

Photo by Burak The Weekender: https://www.pexels.com/photo/hanging-light-bulb-132340/

It was a very big moment for me at work. Those who read last week’s post know that I was able to get an Integration Test working and, since I had code that was passing a test, I figured it would soon be ready for deployment. If you’ve been in a similar situation, you probably know what happened next: I got some notes on my PR.

One of the notes was a suggestion that I further test the functionality of my code locally, using Tilt. I like Tilt, I’ve used Tilt a lot, but I have not written a Tiltfile from scratch before. So I asked for some help. After a couple of zooms, we had debugged four or five issues and I had some actionable next steps. And, of course, I immediately came across a new error.

I was stressed, but I pushed myself, saying “give this error a real honest effort before you ask for help yet again.” I have struggled with debugging for years because I’m usually so unfamiliar with my domain that I can’t tell if I’m making a mistake in the language, the library, or my local environment. But I think it’s an important part of being an engineer, so I really tried.

And this is what I figured out.

Beans in my Config file

Last week, I described the business logic of the job I’m working on: delete a bunch of old rows from a database that it doesn’t run out of space. Since I was able to test that, we’re now moving on to the structure that executes that logic. I’ve done a bunch of Java work in the past that involved Repositories and Services, but in this case I was also advised to create a Config class. This class is a series of methods that actually instantiate the pieces I had only previously defined: the Service, the Repository, a JDBI instance for my SQL, and also a DataSource, but we’ll get to that later.

I want these methods to run whenever my job runs, so I was advised to make them beans. I think it’s likely that I will never fully understand what a Bean is, but I have had it explained to me many times, so I will attempt to explain it here. In Spring (Java framework that does magic to make your application run more easily), a bean is “an object that is instantiated, assembled, and managed by a Spring IoC container.” What that means to me is that Spring is going to instantiate my beans when it starts up, even if I don’t write MyBeanClass myBean = new MyBeanClass(); . Here’s the bean that would eventually cause me problems:

@Bean("myServicePostgresDatasource")
public DataSource getMyDataSource() {
    LOG.info("DataSource bean built HikariDataSource");
    return DataSourceBuilder.create().type(HikariDataSource.class).build();
}

The first line is a Spring annotation specifying that the method we are about to define is a bean. In parentheses is the bean’s name, represented as a string. Spring documentation points out that a bean’s name should start with a lowercase letter and be in camel case. The name is useful here because it will allow me to reference this bean in other parts of my code. Since beans aren’t traditionally instantiated, we can’t specify which one we’re going to use in different situations. Instead, we can use a qualifier to say “I want this DataSource” rather than some other DataSource defined by a different bean in my project. For example:

@Bean
public Jdbi postgresJdbi(
  @Qualifier("webhookDispatchServicePostgresDatasource") DataSource dataSource
) {
...
}

Without the @Qualifier, I’d get an error at runtime because the app doesn’t know what I mean when I pass in dataSource. My mentor pointed out that this makes it extra important that we test our Spring applications locally or in dev before deploying changes, because a bad bean will compile, but the app itself won’t start.

Outside of the annotation, my bean looks like a totally normal method. It uses Spring’s DataSourceBuilder class to create a DataSource which we’ll later use to instantiate Jdbi and query our Postgres database. Java supports a few different pooling DataSource implementations, including HikariCP. And Hikari is where I started to see an error.

My error

I attempted to start up my application, but it wasn’t behaving as expected. When I looked through the logs, I saw a message that looked like this:

java.lang.IllegalArgumentException: dataSource or dataSourceClassName or jdbcUrl is required.
 at com.zaxxer.hikari.HikariConfig.validate(HikariConfig.java:1029)
 at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:109)
 at org.jdbi.v3.core.Jdbi.open(Jdbi.java:318)
 at org.jdbi.v3.core.Jdbi.withHandle(Jdbi.java:364)
...

Reading the documentation, I learned that Hikari needs a DataSource or jdbcUrl. Here’s the first example provided:

HikariConfig config = new HikariConfig();
config.setJdbcUrl("jdbc:mysql://localhost:3306/simpsons");
config.setUsername("bart");
config.setPassword("51mp50n");
config.addDataSourceProperty("cachePrepStmts", "true");
config.addDataSourceProperty("prepStmtCacheSize", "250");
config.addDataSourceProperty("prepStmtCacheSqlLimit", "2048");

HikariDataSource ds = new HikariDataSource(config);

This makes sense; I have to specifically tell Hikari details about the DB connection we’re using. But we’re not instantiating Hikari the same way this example is — we’re passing it as a class into a DataSourceBuilder. So I started looking for more examples online or examples within my codebase that looked similar to what I was doing. I eventually learned that I was missing a ConfigurationProperties annotation.

Since this was specifically happening when I ran my code locally, I went to my application-local.yaml file to see if I could learn more about the connection we were attempting to make. I found something that looks like this:

  accounts.postgres:
    username: postgres
    password: password
    minIdle: 10
    idleTimeout: 600000
    maxPoolSize: 100
    connectionTimeout: 30000
    jdbcUrl: jdbc:postgresql://888.0.0.1:123456/accounts?verifyServerCertificate=false&useSSL=false

This has a jdbcUrl, so it might be what I’m looking for. Clearly my application doesn’t understand that it should be looking here for DB info. ConfigurationProperties serves as the pointer to tell it where to look:

@Bean
@ConfigurationProperties(prefix = "service-name.accounts.postgres")
public Jdbi postgresJdbi(
  @Qualifier("webhookDispatchServicePostgresDatasource") DataSource dataSource
) {
...
}

By specifying the location in the annotation, I clarify the data that Hikari is looking for. And the error disappears!

Pattern Matching

When investigating errors, it can sometimes feel like we don’t know what to investigate. Pattern matching sometimes gets us in trouble, but if we work to understand what we’re seeing in someone else’s code, a solution that they implemented might solve our problem as well. I felt very fortunate that I was able to solve this problem without taking up anyone else’s time and I hope I’ll be able to build on that momentum in the coming weeks.

Sources

• Tilt, tilt.dev
• Bean Overview, Spring docs
• DataSourceBuilder, Spring docs
• HikariCP, Brett Wooldridge
• Spring Boot HikariCP DataSource Configuration, Lokesh Gupta, HowToDoInJava