What is the Testing Pyramid? Does it Work?

@ Software Testing

Introduction

Recently reviewing some testing concepts, I encountered the testing pyramid from Mike Cohn’s book “Succeeding with Agile: Software Development Using Scrum.” I haven’t read this book yet, but this term is something anyone who has researched testing would come across. I’d like to revisit it and share my understanding of this topic.

Types of Tests

  • Unit Testing: Testing the smallest functional units of code, like a function without dependencies.
  • Integration Testing: Testing whether modules function correctly together, such as the DB and backend services.
  • End-to-End Testing: Testing actual user workflow.

Unit tests verify individual components, integration tests check component interactions, and end-to-end tests validate the entire system.

Testing Should be Like a Pyramid

The testing pyramid is a concept about the ideal distribution of tests, emphasizing that, from a cost perspective, a large number of quick, simple, and independent unit tests should form the base. As the scope of testing increases, the cost progressively rises, and tests become more fragile and harder to maintain. Broader tests fail due to even the slightest error, so let’s quickly identify issues through small-scope tests in the early stages of problems.

My Real-World Testing Experiences

Useful software emerges from struggles and trial-and-error; in a reality where requirements are uncertain, testing can become a stumbling block. Thus, only software that survives will have the necessity to maintain tests. The most common scenario is a program that tends to stabilize but has never had automated testing or documentation. There’s very low confidence in the code, leading to situations where any change to A breaks B or the burden of testing becomes too heavy before starting to consider automated testing.

Should we start filling in unit tests based on the pyramid model? It seems unnecessary… Firstly, it can’t be filled in all at once, and secondly, it’s not cost-effective. A lot of logic is one-time and involves trade-offs.

Instead, the most fragile and hardest to maintain E2E tests might actually be more helpful because they are black-box tests (requiring no knowledge of internal program logic), capable of covering expected behaviors, testing many functions at once, and providing confidence that the program works.

The testing pyramid is a concept that is good and makes sense, but in actual situations, it is not so useful.

The World is Changing and So is Testing

Testing tools and environments continue to evolve, and there are different concepts that can solve problems. For example, why is testing considered “slow”? Why is it seen as a “stumbling block”?

  • In the concept of TDD, it requires tests (automated documentation) to be clearly defined before writing code. The role of tests should not be just a barrier but a guide to prevent going down the wrong path.
  • With the help of AI providing clear context, testing can be build in a snap.
  • Enhancements in testing tools and hardware allow tests to run faster and in real-time.

Further Reading