Test Anti-Patterns: The Composite Test

Composite Test

A composite test is one that is actually testing multiple units in a single test body. It has the problem that it’s difficult to tell what parts of the test are accidental vs. intentional. This leads to breaking changes because future developers are as likely to modify the test as they are the production code.

Imagine a bank transaction cache that has three defects:

  • get with no accounts returned nothing instead of all transactions. It should return all transactions.
  • Filtering by more than one account led to duplicate transactions in the result. Each transaction should only appear once.
  • Filtering by more than one account led to a result that is not sorted from newer to older.

Consider the following test written to cover these three issues:

    #[test]
    fn by_account_does_not_duplicate_transactions() {
        let env = TestEnv::default();
        let transactions = [
            build(
                Bank::SendMoney(Data {
                    amount: 42,
                    from: account1,
                    to: account2,
                }),
            ),
            build(
                Bank::SendMoney(Data {
                    amount: 24,
                    from: account2,
                    to: account1
                }),
            ),
        ];
        env.store_transactions(transations);

        let result = env
            .service
            .get(Some(Filter {
                accounts: vec![
                    account1,
                    account2,
                ]
            }));

        assert_eq!(result.total, 2);
        assert_eq!(
            &*result
                .transactions
                .into_iter()
                .rev()
                .map(|t| Transaction::try_from(t).unwrap())
                .collect::<Vec<_>>(),
            &transactions,
        );
    }

Remember the attributes of good unit tests

  1. The test must fail reliably for the reason intended.
  2. The test must never fail for any other reason.
  3. There must be no other test that fails for this reason.

Our good unit tests guidelines ask us to write tests that fail for one reason–not multiple. They also ask us to write tests that clearly communicate what is being tested. The test name above tells us about the duplication failure, but it does not communicate that filtering failure or that ordering is an issue. Those requirements appear accidential to the design of the test. The test fails to function as a specification.

Could we get around this problem by "aliasing" the test body multiple times?

#[test]
fn all_transactions_are_returned_without_a_filter() {
    really_test_the_thing();
}

#[test]
fn transactions_are_returned_in_order() {
    really_test_the_thing();
}

fn really_test_the_thing() {
    // Real code goes here
}

If we try this approach. We do satisfy the goal of specifying the test behavior, but now any failure in the test body will cause all of our specifications to fail.

We can account the issue by following the practice of TDD which is to start with a failing test, one unit at a time. I almost always see this issue arise in test maintenance when the engineer wrote the test after the fact as opposed to practicing TDD.

Let’s begin.

1. No Account Filter

// Example in pseudo-code
#[test]
fn when_no_account_filter_is_given_then_transactions_for_all_accounts_are_returned() {
    // Given
    let env = TestEnv::default();
    let accounts = vec![porky, petunia, daffy, bugs];
    let transactions = env.generate_some_transactions_for_each_account(accounts);
    env.store_transactions(transactions);

   // When
   let result = env.cache.get(None);

   // Then
   let accounts = result.get_distinct_accounts_sorted_by_name();
   assert_eq!(accounts.sort(), expected_accounts);
}

This test communicates what’s failing. It does not rely on == a list of transactions and instead isolates the behavior under test–namely that no filters by account are applied.

2. Both Transaction Sides are Returned

If we are filtering by account, we need to make sure transations are returned whether the account is on the from or to side of the payment.

// Example in pseudo-code
#[test]
fn when_account_filter_is_specified_then_transations_to_or_from_account_are_returned() {
    // Given
    let env = TestEnv::default();
    let accounts = vec![porky, petunia, daffy, bugs];

    let transactions = [
        build(
            Bank::SendMoney(Data {
                amount: 42,
                from: bugs,
                to: daffy,
            }),
        ),
        build(
            Bank::SendMoney(Data {
                amount: 24,
                from: porky,
                to: bugs
            }),
        ),
        build(
            Bank::SendMoney(Data {
                amount: 100,
                from: porky,
                to: daffy
            })
        )
    ];
    env.store_transactions(transactions);

   // When
   let result = env.cache.get(
        Some(Filter {
            accounts: vec![
                bugs,
            ]
    }));

   // Then
   result.assert_all(|transaction| transaction.from == bugs || transaction.to == bugs);
}

3. Duplicated Transactions

One of the defects the composite test was intended to cover was that when specifying multiple accounts, transactions were duplicated in the results. Let’s test that now.

// Example in pseudo-code
#[test]
fn when_filtered_by_multiple_accounts_then_transactions_are_not_duplicated() {
    // Given
    let env = TestEnv::default();
    let accounts = vec![porky, petunia, daffy, bugs];
    let transactions = env.generate_some_transactions_for_each_account(accounts);
    env.store_transactions(transactions);

   // When
   let expected_accounts = vec![bugs, daffy];
   let filter = Filter {
       accounts: expected_accounts
   }
   let result = env.get_transactions(Some(filter));

   // Then
   let ids = result.assert_transaction_ids_are_unique();
}

4. Preserve Sorting

Sorting is different enough behavior that it deserves its own focus. I would have expected a single test on sorting for the unfiltered case. Since sorting is usually applied first this would ordinarily be enough. However, given that there was a defect specifically around sorting when filtering on multiple accounts, adding a second test case is warranted–the existence of the defect proving the need.

I’m going to extend my test helper to allow for specifying dates.

// Example in pseudo-code
#[test]
fn when_filtered_by_multiple_accounts_then_sorting_is_preserved() {
    // Given
    let env = TestEnv::default();
    let accounts = vec![porky, petunia, daffy, bugs];
    let dates = vec![NewYearsEve, ChrismasDay, ChristmasEve, NewYearsDay]);

    let transactions = [
        build(
            Bank::SendMoney(Data {
                amount: 42,
                from: bugs,
                to: daffy,
                date: ChristmasDay
            }),
        ),
        build(
            Bank::SendMoney(Data {
                amount: 24,
                from: porky,
                to: bugs,
                date: ChristmasEve
            }),
        ),
        build(
            Bank::SendMoney(Data {
                amount: 100,
                from: porky,
                to: daffy,
                date: NewYearsDay
            })
        ),
        build(
            Bank::SendMoney(Data {
                amount: 150,
                from: porky,
                to: daffy,
                date: NewYearsEve
            })
        ),

    ];
    env.store_transactions(transactions);


   // When
   let filter = Filter {
       vec![bugs, daffy],
   }
   let result = env.cache.get(Some(filter));

   // Then
   let dates = env.project_transaction_dates_without_reordering();
   // use named date constants here so that the relationship between the dates is understood at a glance.
   assert_eq!(dates, vec![NewYearsDay, NewYearsEve, ChristmasDay, ChrismasEve]);
}

Final Notes

As I worked through these examples, I started to see things that can be added to the test context and the production code to make intent clearer and even easier to test. That’s also a benefit of beginning work with the failing test: the act of writing a test that tells a story helps you understand what code you need to tell it. I often find that code I wrote to support my test ends up in my production implementation since it simplifies and/or clarifies working with the test domain.

Leave a Reply

%d bloggers like this: