Insights on E2E Testing and BDD After 4 Months at New Job

Introduction

Documenting the questions and answers, personal experiences, key points, and techniques related to testing over the past four months, as well as enhancing my team’s confidence and understanding of BDD.

During my recent time at a new job, in addition to front-end development, I also took on testing, writing nearly a hundred E2E tests of various sizes. I plan to solidify my knowledge by writing this article while reviewing documentation, and also to record it for my team members to quickly get up to speed with the newly established Cypress E2E testing project. Ultimately, I hope to experiment with introducing the BDD process into the team’s development workflow.

First Encounter with BDD and E2E Testing

My exposure to BDD came from an old project that attempted to implement this development process. However, as personnel changed, the old project was gradually abandoned. The first task upon joining the team was to establish a new E2E testing project to stabilize the legacy code and explore the possibility of introducing the BDD process to “enhance the efficiency of product development” and “increase confidence in product modifications.”

The first few months were focused on “familiarizing myself with the existing product code and behavior.” During this process, I reviewed documentation while learning and writing tests. In the old project, I used Cypress🔗 and Cucumber🔗.

E2E? Cypress? BDD? Cucumber? These terms were all unfamiliar to me at the time. I just wanted to quickly set up the project and gradually fill in the theoretical knowledge. Now, if I were to summarize them in one sentence, I would describe them as follows:

End-To-End Testing: Validating product functionality and reliability by simulating real user interactions within the application.
Cypress: A JavaScript framework primarily used for E2E testing, designed for writing automated test code to simulate user actions.
Behaviour-Driven Development (BDD): A software development model focused on the expected behavior of the system, achieved through means such as encouraging collaboration between business and development teams, organizing business-readable specification documents, and automating the verification of whether behaviors meet expectations by linking specification documents with test code.
Cucumber: A BDD tool used to convert business-readable specification documents (Gherkin syntax) into executable test code.

Team Collaboration

Since both business and development teams are encouraged to participate in the BDD process, and BDD considers testing an integral part of software development, understanding some basic principles of good testing is beneficial for all parties! If you agree with defining product development based on product behavior, the following methods and principles may be helpful:

Encourage Collaboration Between Business and Development Teams

The most common method is the “Three Amigos” meeting, which gathers three different perspectives to discuss (the key is that they are different perspectives, not just numbers):

💹 Business personnel - What problems are we trying to solve?
👨🏻‍💻 Developers - How do we construct a solution to solve the problem?
🧪 Testers - What potential scenarios could occur?

When the business team defines the product requirements, they can use the following template to articulate them: As a (type of user), I need to use (type of feature) so that I can achieve (type of benefit). Here is an example from Cucumber School’s BDD tutorial🔗:

As a (sales assistant) I need to use (refund feature) So that I can achieve (satisfy the user’s legal rights)

Although the developer in the example does not have much to say, simply participating in this meeting allows relevant personnel to gain a deeper understanding of the requirements’ context. Typically, during detailed discussions of practical details, developers can better see technical limitations and work with the team to discuss feasible solutions.

1. 💹: Any product can be refunded within 14 days as long as the user has the receipt.
2. 👨🏻‍💻: Can you provide a real case?
3. 💹: User A purchased a product but later didn't like the color, so she requested a refund from the sales assistant.
4. 🧪: So what do we expect to achieve? What can be tested?
5. 💹: We need to increase the product quantity because the user has returned the sold product.
6. 🧪: Okay, anything else?
7. 💹: I think that's it?
8. 🧪: What if the user doesn't have a receipt? Should the sales assistant reject the request outright?
9. 💹: Because the manager can handle any refunds as long as there is any proof of purchase... perhaps User A can use a bank statement to prove her purchase record. But the sales assistant doesn't have the authority, so he should communicate with his superior.
10. 🧪: So there are two rules: one for users with receipts and another for users without receipts, and refunds without receipts require superior authorization, right?
11. 💹: Yes, I think that's it.
12. 👨🏻‍💻: Shouldn't the user get a refund?
13. 🧪: And... does the system need to notify the warehouse to retrieve the refunded product?
14. 💹: Good idea! I think we'll wrap up this story discussion here.

After the meeting, special attention must be paid to writing the product behavior document based on the discussion content.

When a requirement document is created solely from a single perspective, such as by the business team alone, many deviations may occur. The resulting document does not represent the consensus and understanding of everyone, and these deviations can lead to difficulties in development or automated testing. Developers, in order to automate these scenarios, may end up writing documents that lead to special syntax (like Gherkin) becoming “something used by the development team,” which may misinterpret the true intent and produce incorrect documents.

The best practice is for both business and development sides to maintain a consensus, defining “living documents” through behavior descriptions and jointly reviewing and maintaining them. The documents can be written according to team preference rules, ideally so that anyone in the team can read, understand, and automate tests.

Document Writing

The 3A Principle of Good Testing

The structure of any test borrows from the 3A principle mentioned by Filip Hric🔗, which consists of three simple steps:

Arrange - Act - Assert

Arrange: Prepare the preconditions for the test - what has happened
Act: Execute the test - what is supposed to happen
Assert: Verify the test results - what will happen

Describe Product Behavior in a Consistent Language

To avoid communication deviations, a standardized language is usually introduced to describe product behavior, such as Gherkin syntax. The advantage of this syntax is that both the business team and the development team can describe product behavior in the same language, and tools can convert these statements into executable test living documents.

Given reaching the login page
When entering username and password
Then successfully logged in

Using Gherkin syntax as an example, it may seem that there are many keywords to learn, but in fact, all keywords still follow the 3A testing rules, revolving around the three keywords Given, When, and Then. You can refer to the official Gherkin documentation🔗 or this super condensed cheat sheet🔗. Additionally, Cucumber’s starting point is to write business-readable living documents, and you can ensure that every team member can quickly get up to speed and communicate effectively by using localized Gherkin syntax🔗.

Using BDD does not mean you must use Cucumber’s Gherkin syntax, but it is a very effective tool for practicing BDD.

Excessive Practical Details

Focus on the goals of the software, not the mechanisms Express specifications starting from the problem, not the solution

There are many interaction details related to the UI in operating programs, such as “clicking a specific link” or “finding a specific element.” These descriptions reflect user expectations rather than being fragile and variable practical details. Therefore, when writing documents, these details should be avoided, focusing instead on user expected behavior. Here are some actual cases:

@Declarative (Good)
Scenario: Verify Login
Given an unauthenticated user reaches the login page
When entering login information and logging in
Then display a successful login message

@Imperative (Bad)
Scenario: Verify Login
Given an unauthenticated user reaches the login page
When entering the username
When entering the password
When clicking the login button
Then display a successful login message

Avoiding the description of practical details in documents is not laziness! Also, do not feel that “since we’re here, we should write everything clearly.” If there are any changes to the login UI or data in the future, the document will also need to be modified. Let the practical details be written in the tests; the document should only describe the intent.

Documents focus on intent rather than practical details
Intent is less frequently changed compared to practical details and can better accommodate practical changes
Documents that remove operational details will be easier to read (the prerequisite knowledge requirement or reading cost of the document should be minimized as much as possible)

This issue often arises when you are writing scenario documents while also proposing solutions or implementing test functionalities. It is essential to remember that the essence of the document is to describe product behavior rather than practical details, ensuring the document’s readability and maintainability.

Conclusion

The software development model is a relatively abstract and broad topic, and you are always welcome to contact me for discussions. This article summarizes some important concepts and team discussions based on my recent experiences and data collection, hoping to help all product teams develop products more efficiently. I expect to continue updating with more information as needed.

What Counts as BDD?

“What counts as BDD?” This question repeatedly appears in my mind. Is it enough to use Cucumber? Or is it because the tests use Given-When-Then statements for classification? Or does it count only if cross-domain discussions are held to write documents together? Ultimately, I have consolidated explanations for different people:

Developers
1. Developers start writing tests for the program
2. Developers find that as the number of tests increases, their confidence in the program also rises
3. Developers discover that writing tests before development helps them focus on the necessary code
4. Developers find that tests can serve as documentation to help understand the behavior of old code
5. Developers accept that tests can serve as development documentation
6. Developers realize that TDD is actually about defining behavior rather than testing
7. Behaviour is about the interactions between components of the system and so the use of mocking is fundamental to advanced TDD.
8. *My personal summary is: BDD is an advanced version of TDD for team collaboration
Non-Dev People
1. Encourage collaboration between business teams and development teams
2. Organize business-readable specification documents
3. Automate the verification of whether the behavior meets expectations by linking specification documents with test code