“Test-driven development” was one of the first terms I learned when I first entered bootcamp at the end of 2019. Our lessons would always be similar: we would read about a new concept, see some examples, and then be asked to demonstrate our knowledge by coding. There would be an empty class and a suite of failing tests. Once we got the tests to pass by writing our code, we could move on to the next topic.
So I learned about the concept of “TDD,” but I wasn’t really practicing it because I wasn’t actually writing the tests (that’s the hard part). It’s been almost five years since that point and, though I’ve written a few tests, I can’t say I’ve done much test-driven development. This week, I paired with a senior engineer and, with a lot of guidance, I might have truly followed the process for the first time.
The Objective
My team manages integrations with external services and vendors. This means that we’re constantly sending information back and forth between various parties through API calls, file sharing and webhooks. Webhooks are lightweight HTTP callback functions that can be used to share information between a client and a server. They’re distinct from an API because the server makes requests to get data, rather than the client.
The technical details of webhooks aren’t too important here — we just want to understand that my team manages a service that dispatches webhooks and, as part of that service, saves them to a Postgres database. However, it’s possible for the DB update to fail, and if that happens the data is lost. I picked up a ticket that asked to route failed webhooks to a Dead Letter Queue so that we could replay them later. I was able to find the line of code where I’d want to add an enqueue method, but rather than update that line, it was time to write a test.
The Infrastructure
Because I was pairing with a senior engineer, I was told exactly what classes I’d need to write my TDD tests. This is where I started paying very close attention, because I wouldn’t have been able to figure this out on my own and I wanted to make sure I knew for the future.
First, we created an interface called DeadLetterService. It looks like this:
public interface DeadLetterService {
void enqueue(WebhookRequest webhookRequest);
}You may remember when I learned the value of an interface; this is the same situation. I am going to want to test a method that, in production, would interact with an external API (an SQS queue), but when testing, I want to eliminate that variable. So I create an interface that can be implemented by the test and implemented in production. My teammate explained that this strategy should always be followed when dealing with “non-deterministic” code, or code where the output might vary depending on outside factors.
Next, we created two test doubles. Test doubles are classes that replace production classes so that we can better check our work. In this case, we need a stub, which is a double that will always return a canned response regardless of the input. Since we are testing behavior that is triggered by a failure to write to a DB, we want to stub out the existing method in our test and replace it with one that always throws an exception. It looks something like this:
public class AlwaysFailsWebhookRequestRepositoryStub implements WebhookRequestRepository {
@Override
public Optional<Long> createWebhookRequest(
Long webhookId,
Long companyId) {
throw new RuntimeException("Failure");
}The production class that implements WebhookRequestRepository has a bunch of logic that connects to postgres and adds a row. But we don’t care about that in this case, we just want an exception. Next, we need another double, this time a spy. While a stub helps us control the return value of a method, a spy helps us measure its side-effects. Our DeadLetterService interface requires an enqueue method, which we expect to use to add a failed webhook to some sort of queue (SQS in prod, local memory while testing). The return value of enqueue is void, but we care about what it does before that fact. So our spy defines the enqueue method and a couple of other methods that can give us information on the results:
public class DeadLetterServiceSpy implements DeadLetterService {
private final Queue<WebhookRequest> queue;
public DeadLetterServiceSpy() {
queue = new ArrayDeque<>();
}
@Override
public void enqueue(WebhookRequest webhookRequest) {
queue.add(webhookRequest);
}
public int spyNumberOfQueuedMessages() {
return queue.size();
}
public WebhookRequest spyPoll() {
return queue.poll();
}
}We create a fresh queue locally and then just add to it. spyNumberOfQueuedMessages will measure the queue and spyPoll will return the contents. Now we should be able to test the repository, force it to fail, and check whether the message has been enqueued.
The Test
I’ve always feared test-writing because it felt overwhelming to have to create these doubles and figure out how to eliminate non-deterministic code. But in this case, it all seemed pretty straightforward. So then maybe the tests themselves will be difficult? Here’s what we wrote:
public class WebhookHandlerServiceErrorsTest {
private WebhookHandlerService webhookHandlerService;
private DeadLetterServiceSpy deadLetterServiceSpy;
@BeforeEach
public void setup() {
deadLetterServiceSpy = new DeadLetterServiceSpy();
webhookHandlerService = new WebhookHandlerService(
new AlwaysFailsWebhookRequestRepositoryStub(),
deadLetterServiceSpy);
preconditions();
}
private void preconditions() {
assertThat(deadLetterServiceSpy.spyNumberOfQueuedMessages()).isZero();
}
@Test
public void dispatch_newWebhookRequest() {
// Given
WebhookRequest request = DataFixture.createWebhookRequest();
// When
webhookHandlerService.dispatch(request);
// Then
assertThat(deadLetterServiceSpy.spyNumberOfQueuedMessages()).isOne();
// And
assertThat(deadLetterServiceSpy.spyPoll()).isEqualTo(request);
}
}This Java is so clear that even I can read it! But I have an advantage because I’ve been looking at it for two days, so let’s review:
- We create a class and give that class two attributes:
webhookHandlerServiceanddeadLetterServiceSpy. BeforeEachtest we instantiatewebhookHandlerServiceanddeadLetterServiceSpy.webhookHandlerServiceis the class that actually runs the logic that should eventually triggerenqueue. Note that we pass ourAlwaysFailsWebhookRequestRepositoryStub, which ensures that we’ll always hit that exception. We use BeforeEach instead of a constructor because it’s safer to have fresh instances before each test.- The BeforeEach also includes the execution of a
preconditionsmethod. This checks to make sure we have a clean queue. We see the method definition, which asserts that the number of queued messages on the spy is zero. - Finally, we hit the test. Given a request (any request of the correct type), when we dispatch that request via our service, then the queue size becomes 1 and the element in the queue matches our request. Voila!
The Results
When we ran the test, it failed of course, because we hadn’t implemented any of the desired code in production. But I had a better idea of the classes involved in this process and I had clear feedback about what wasn’t working. Most importantly, starting with a failing test means we have clearly defined the desired behavior and we will know for sure whether or not we will successfully create it. I more commonly write tests for existing code, but when those pass, they could pass for any reason. Through test-driven development, we can be more methodical, more consistent, and more confident in our code.
Sources
- What is a webhook?, Red Hat
- What is a Dead-Letter Queue (DLQ)?, AWS docs
- What is Amazon Simple Queue Service, AWS docs
- Test Doubles, Martin Fowler