Advanced Test Setup Part 1 - Cucumber for Integration Tests

This is the first part of a series I want to do on an advanced test setup. I have used a setup of this form in several projects in the past.

Disclaimers:

• I will use C# in my examples, because that is the language I have used in recent projects. But a similar setup worked for me also for Java projects. And I have no reason to believe it can not be adapted for other languages / runtimes.
• This is not a tutorial on Cucumber / ReqNRoll. If you want to learn more about those tools, please refer to their respective documentation.
• "Integration test" is an overloaded term. Here, I use it to mean tests that cover an application's functionality across all layers and modules (except UI). But external dependencies are (mostly) still being replaced with a test double.

The Goal

Most software projects struggle with test automation. Unit tests do not cover enough scope, and so many teams try implementing E2E tests via the UI to cover all functionality.

As I have mentioned in previous blog articles, UI tests are brittle and slow, and don't scale well as applications grow.

An alternative is to write "integration tests" against the API / the layer under the UI. These tests complement unit tests, and do not replace E2E tests entirely. The aim is to have confidence in the system's functionality without having to resort to UI tests for everything.

Setting up Cucumber

There is a lot of examples online on how to structure Features and Step Definitions. Most I have seen build a 1:1 relation between Feature files and "their" Step Definitions.

I prefer to group Step Definitions by domain concepts, not by Feature files. This way I can reuse my Step Definitions when testing multiple Features (often for Given and Then statements).

If you are doing DDD, this structure will make a lot of sense: for example by grouping Step Definitions by Bounded Contexts and/or Aggregates. But even if you don't, you can, for example, group them by functional areas of your application.

Here is one example:

Features
| - CreateAccount.feature
| - ChangePaymentMethod.feature
| - RemoveActivePaymentMethod.feature
| - CancelSubscription.feature
| - ...

StepDefinitions
| - AccountManagement
|   | - UserAccountSteps.cs
|   | - BusinessAcountSteps.cs
| - Payments
|   | - PaymentMethodSteps.cs
|   | - InvoicesSteps.cs

Your structure will differ, of course. But the main point is to avoid coupling the Step Definitions to the Feature files. That would lead to step definitions being written for one specific Feature and therefore hard to reuse.

Reusability of Step Definitions is important, because it makes writing new tests easier, faster, and more enjoyable. (You should enjoy writing code. If it's tedious and boring, you will avoid it. Not just true for tests.)

Step Definitions

An example for what a Step Definition file could look like:

public class UserAccountSteps(ScenarioContext context) : AbstractStepDefinitions(context)
{
    private UserController UserController => GetService<UserController>(); // GetService is defined in AbstractStepDefinitions and resolves from ServiceProvider
    
    [Given(@"a user with email (.*) exists")]
    [When(@"I create a user with email (.*)")]
    public void CreateUserWithEmail(string email)
    {
        var request = new CreateUserRequest { Id = Guid.NewGuid(), Email = email, ... };
        UserController.CreateUser(request);
        
        Context.AddUser(request.Id, request.Email); // Context is ScenarioContext, .AddUser is an extension method
    }
    
    [Then(@"the user with email (.*) exists")]
    public void ThenUserWithEmailExists(string email)
    {
        var request = new GetUserByEmailRequest { Email = email };
        var user = UserController.GetUserByEmail(request);
        user.Should().NotBeNull();
        Context.Set(user); // Store user in ScenarioContext for later steps
    }
    
    [Then(@"user has name (.*)")]
    public void ThenUserHasName(string name)
    {
        var user = Context.Get<UserDto>(); // Retrieve user from ScenarioContext
        user.Name.Should().Be(name);
    }
}

The code is simplified for clarity. I omitted details like how the extension methods for the ScenarioContext look like, or how the controller was fetched from the ServiceProvider. Also, I tend to have different DTOs for different use cases, not just one User DTO.

Here are some points to note:

• There is one Step Definition with both a Given and a When attribute. This way the same definition can be used for both set up and test action.
• There are two Then Step Definitions. One loads the user by mail and stores it in the ScenarioContext for subsequent assertions.
• Any newly created entity are registered by ID in the ScenarioContext. This way other step definitions can load the entity by either ID or a more readable identifier (like E-Mail, unique names, etc.).

All the points above aim to make the code easily reusable in Feature files.

Good code design is not reserved for production code only! By designing your test code well, you reap the same benefits: better readability, maintainability, and reusability.

Feature Files

Just a few words on feature files, because I see one particular mistake quite often. Take a look at the following example:

Feature: Add product to cart
    Description omitted

Background:
    Given a user with email "mymail@test.com" and password "12345" exists
    Given I am logged into the system by entering username "mymail@test.com" and password "12345"
    Given I navigate to the product catalog via the main menu
    Given an article "Product A" exists in the catalog

Scenario: Add product to cart
    When I enter "Product A" in the search field
    And I click on the first product in the search result
    And I add the product to the cart
    Then the cart contains 1 item

This is not a feature description. This is a test script, that is clearly aimed at navigating a UI.

Feature files like these break easily for the same reason UI tests break: they are brittle and depend on many details of the UI. If the UI changes, the test breaks, even if the underlying functionality is still working.

The same goes for feature files that describe API calls in too many details. Aim to keep the steps at a higher level of abstraction, describing the behavior of the system, not the implementation details.

Here is a better version:

Feature: Add product to cart
    Description omitted

Background:
    Given a user with email "mymail@test.com"
    Given an article "Product A"

Scenario: Add product to cart
    When I add "Product A" to the cart
    Then the cart contains 1 item

Granted: this is an oversimplified example. But it illustrates a point: Step Definitions should describe behavior, not implementation details.

When writing a Feature File description, ask yourself:

• If I did not have a GUI, would the same description still make sense? What if I switched from a REST API to a CLI client? Does the description need to change?
• Would a non-technical stakeholder understand what the Feature is about? Does it contain technical jargon?

Another tip: the language should flow naturally. In the example Step Definition above, I added created and fetched data to the ScenarioContext. That way, I can write Feature files using expressions like "And the user" or "* has a name", without having to repeat the full description or ID of the entity.

Given the user with email "mymail@test.com" has registered on 1.1.2025
And the user with email "mymail@test.com" has admin rights
And the user with email...

vs

Given a user with "mymail@test.com"
* has registered on 1.1.2025
And has admin rights
* with ...

The second example feels much more natural, right?

Fixture

The last part I want to mention here is the setup of the software under test. This much more depends on the specifics of your application. But I want to point out a couple of general principles.

In my test setup, the software under test runs in the same process as the test code. Before the first test is run, the application is started up, and all dependencies are registered. (For some you might need test doubles.)

I like to run my database in a Docker container, so I can easily wipe it between tests. I prefer not to use in-memory databases, because they often behave differently than the production database.

You might also want to spin up other containers, like RabbitMQ for messaging, etc.

If at all possible, I like to clear the database between tests, so each test starts with a fresh DB fixture. Sometimes that is difficult. On one recent project we had to code a bunch of SQL scripts to disable temporal tables in SQL Server. But if you can, clear the database between tests. It saves a lot of headaches.

Another common practice is to set up a standard seeded fixture in the database. This avoids the pain of having to create all relevant data in each Feature file's Background section.

One of the limitations of Cucumber is that there is no way to share Backgrounds. I would love to see an additional .fixture file type that defines grouped steps to be reused in Feature Files. IDEs could then help collapsing / expanding a mention of this step group.

I'm not sure if there is a good reason this mechanic does not exist?

Finally, I like to have a separate logging configuration for my tests. If you, for example, use Serilog or slf4j or similar logging frameworks, you can add a configuration file for your test project/package/module, and load this when spinning up the application. Log tests to a file with the "Debug" level enabled. Also, have this log published as an artifact in your CI/CD pipeline.

Integration tests can become flaky (especially if you use a shared fixture in your tests). And being able to look at the logs when a test fails is invaluable. (If you ever had to debug a flaky test that never fails when you run it with a debugger, you know why.)

Conclusion

I like to use Cucumber for automated tests. Not necessarily only for the kind of integration-level tests I have described here.

Over several projects and years of working with different libraries of the Cucumber family, I fell on my nose plenty of times. By now I have a system that works very well for me. In this post I have described the main points of my setup.

Feel free to use the code above as a starting point for your own projects. If you have questions or suggestions, please feel free to reach out to me (you can find the contact information in the footer at the end of the page).

And if you need help setting up a test automation strategy for your project, I would be happy to assist you.

In the next posts we will look at some other "advanced" testing techniques I employ in my projects. Until then:

Happy Testing!