What I Learned at Work this Week: Dealing with Maven

Where I work, writing and testing Java can be tough. Beyond the fact that Java itself isn’t always the friendliest language, I’ve sort of accepted the fact that, most of the time, all of my code will be underlined because of missing modules and any attempt to run code locally will fail.

It’s tempting to ignore these issues, find workarounds, or even not test things locally before making a PR (it’s the same pattern of every other file…why would it fail?), and I’m guilty of all of these. Fortunately, I work with people who see problems and seek to fix them rather than ignore them for as long as they can. This week, one of those people helped me understand why our Java environment appears so brittle, and what I can do to address issues as they come up.

It all starts with Maven.

Maven

Maven is a build tool. It takes care of the steps between writing and running a project in Java, including (credit Simplilearn):

• Generating source code
• Generating documentation
• Compiling source code
• Packaging the code
• Installing the code locally, or on a server

It’s easy to forget that a computer can’t read Java or any other programming code on its own. We rely on compilers and packages to change our programs from syntax we can understand into something that can be executed by a machine. Maven brings together and automates processes that were previously manual, arduous tasks. This is a big help when it comes to importing external libraries (dependencies), which we naturally expect to come from a variety of locations.

I learned that the errors I regularly saw, which called out missing packages, were tied to Maven. More specifically, the way that Maven interacts with our codebase and our IDE, IntelliJ.

Syncing States

IntelliJ is an integrated development environment (IDE) designed for use with Java. One thing that I see frequently is that IntelliJ will generate sources without my asking, usually when I’ve changed branches and accessed new files with new dependencies. But one thing I learned this week is that IntelliJ doesn’t sync with Maven by default, so not every update is included when IntelliJ generates sources.

By default, IntelliJ only syncs dependencies and output directories, but Maven would normally do more than that. For example, if someone else on my team has written a new protobuf for gRPC (a topic I covered last month), that will not generate a new source with IntelliJ. And so if I’m testing something that involves an API call with the aforementioned gRPC, I should expect an error because the API may not be functional without that update.

As you might expect, there is an option in IntelliJ that specifies a Maven sync, but it’s not always ideal because it could slow the speed of our default build down to the speed of the Maven build. If we would prefer to avoid this for any reason, or can’t figure out the IntelliJ setting, there is another option:

Generate Sources

If IntelliJ isn’t doing what we want be default, then let’s tell it what we want! We can execute Maven actions from our command line. So if IntelliJ is falling behind, we can run:

mvn compile

which will take our source code and…compile it! It’s possible that we’re seeing an issue because IntelliJ hasn’t properly compiled a newer file yet. Or we can try out another step in the build lifecycle:

mvn package

Build lifecycle is another name for the series of steps we reviewed earlier which take source code to executable data. Packaging puts our Java code into a distributable format. A common example of this format is a JAR (stands for Java Archive). When we’re deploying our project, we’d prefer to efficiently group all of our work in one place rather than sending one class or module at a time. By packaging files into JARs, we save our servers a lot of time and energy. And if we’ve got a stray module that hasn’t been packaged by IntelliJ…we might see an error!

I’ve used versions of these commands in the past and seen them work, but I never understood what I was actually doing. The good news is that they’re fairly reliable, but the bad news is that they’re extremely generic. If I were working on a personal project, it wouldn’t be much trouble to wait for my files to compile each time I wanted to try something new. But in my company’s monorepo (huge repository that controls most of our product’s functionality), it can take a very long time. And, on top of that, it doesn’t always solve the problem.

This week, I learned a few more specific commands that can better help address common issues:

mvn clean compile

If anything, this will make our whole process longer because the clean argument deletes the existing build directory and rebuilds from scratch. I also learned about these flags, which can be added to our commands:

• -am: This accepts a subsequent argument of a module or modules. Maven will then determine which other modules this passed argument depends on and will build those, saving us from having to rebuild our whole project.
• -pl: This will ignore dependencies and just build the modules we specify.
• -T 1C: The T flag allows us to specify how many threads our job should use. The more threads we use, the faster everything will run, but if we use too many threads, it could prevent our computer for executing any other tasks until the build is finished. 1C indicates one thread per CPU core and, as I understand it, this will increase our build’s speed without overloading our machine.

Maven’s quirks

The last thing I learned is that Maven isn’t the right tool for every job. It is a “top down” tool that connects our entire project rather than separating different modules. So if I try to build one new part, re-generating sources will mean intertwining the new module with other pieces of my code. It won’t simply compile and package that one part — it will redo everything, which takes time, even if we’re using flags to specify the packages we want to build.

That’s a challenge when it comes to a large monorepo, but as many of us have experienced, once you start down a path in engineering, it’s very challenging to go back and choose another. I’m fortunate that I don’t have to be the one to make that decision, and I’m also grateful that other engineers took the time to explain why these decisions were made and how we all can navigate through, rather than around our problems. Learning from those who came before us helps us make good decisions in the future.

Sources

• Maven — Introduction, Apache Maven Project
• What is Maven?, Simplilearn
• Introduction to the Build Lifecycle, Apache Maven Project
• JAR File Overview, Oracle Docs
• Maven — Build Life Cycle, Tutorialspoint