QA documentation: what and when needed

Considering the complexity of modern IT products, it is challenging to imagine a situation where the development process takes place without any testing.

In most cases, we must deal with tight deadlines, varying team abilities, and changing product specifications.

In these situations, having accurate QA documentation is crucial. We have gathered and organized information about creating testing documentation, which we are happy to share. This guide will be helpful for newcomers in the field of QA or managers who aim to understand when and what test documentation is needed to organize quality assurance processes.

QA documentation refers to a set of documents that record the testing process. It assists in verifying that the software satisfies the intended quality criteria and that any problems are promptly detected and resolved. QA documentation includes various documents such as test plans, test cases, test results, defect and bug reports, and other relevant information.

QA testing documentation is crucial in the testing process.  It also helps assess the team's efficiency, comparing planned progress in the test plan with the actual progress.

Different types of QA documentation serve a specific purpose in the testing process. Test plans are comprehensive documents that outline the testing strategy, scope, and objectives. Test cases provide step-by-step instructions on how to test a specific software feature or functionality. Bug reports document issues. You can create most documents in any format. The company or team establishes the standard of documentation. Let's dive deep into QA documentation.

QA documentation encompasses various documents created and maintained throughout the software testing and quality assurance processes. Each type of documentation has a purpose and enhances the efficiency of QA tasks. Usually, these are the following types of test documents:

• Test Strategy 

• High-Level test plan

• Low-Level test plan

• Test Case

• Checklist

• Test Suite

• Test Coverage

• Test Run

• Bug report

To organize, maintain and effectively reuse those documents, it’s recommended to use specialized resources (own or third-party):

• Test management system 

• Task (bug) management system

Let's delve into the details.

A Test Strategy is a high-level document that guides the testing team and stakeholders on testing activities throughout the software development lifecycle. The Test Strategy ensures the application meets the requirements and delivers a high-quality user experience.

So, it is a list of necessary testing types, desired or available tools, quality criteria, and the criteria for starting and ending testing. The Test Strategy is typically created during the early stages of the project.

Amid the debate over who should determine the testing strategy, one thing is sure: involving a project manager (product owner) can ensure higher quality. Therefore, consider the test strategy before creating a more formal and comprehensive test plan.

A test plan is a step-by-step guide that outlines how to test something, like software or a system. It includes details on what needs to be tested, how it will be tested, and the resources required. The test plan helps ensure that everything is tested thoroughly and helps identify any issues or problems before the final product is released. Typically, high-level and specific-level test plans are used on projects. Having both types of test plans for the project is recommended. You can familiarize yourself with our articles on how to write a test plan in software testing and a typical software test plan for an in-depth understanding. 

A high-level test plan is created based on the testing strategy, but it provides a more detailed and project-specific approach. It expands upon the information in the testing strategy and includes specific details, deadlines, and resources required for the project. The high-level test plan outlines the objectives of the testing effort, identifies the features or functionalities that need to be tested, and defines the testing techniques and approaches to be used. By creating a high-level test plan, the team can effectively plan and execute testing activities, ensuring thorough testing coverage and early identification of issues or problems before the final product is released.

This document contains all test cases arranged logically to align with user behavior. For example, the first logical set of cases would involve creating a user account, followed by cases that include actions and interactions with the application. Testing teams commonly create separate Regression, Smoke, and API testing plans.

Main functional test plan (Smoke test plan)

Preconditions: Application is opened.

This table provides an example of a smoke test plan. The test plan includes positive test cases for various functionalities such as User Registration, User Login, Create New Post, Edit Profile, and Delete Account. Each test case outlines the necessary test steps and the expected results for each step, ensuring the successful execution and validation of the corresponding functionality.

The test case is a QA documentation example. It is a detailed description of a specific scenario or condition that defines the steps to be executed, the data to be used, and the expected results for testing a particular aspect of an application or system.

Test cases are crucial in software testing, as they help ensure the software functions correctly and meets the specified requirements.

Here is an example of a test case from the previous test plan.  It also may include test data, preconditions, and postconditions that verify requirements.

Writing and using test cases have a significant impact on business success. It results in happier clients, increased customer loyalty, reduced customer service expenses, and improved product reliability. As a result, the company's reputation and brand image are greatly enhanced. You can also read our article on how to write a test case in software testing.

The checklist can be a part of the Test Plan or a separate document. It is a specific list of items to be verified. It helps in planning particular test suites or some application parts checking. A checklist is created in a flexible format when there is a need for quick testing planning. A checklist serves as a tool to ensure all required steps or items are completed or considered.

It's essential to regularly update the status of a checklist to reflect the progress and ensure that we complete all necessary steps. Due to its simplicity in implementation, a checklist can be used for quick testing and as a reminder for essential test activities.

Here’s an example of the checklist created from the test suite above:

A test suite typically includes specific instructions or objectives for each set of test cases, along with details about the system configuration employed during the testing process. Test cases may consist of prerequisite states or steps and descriptions of the following tests. A test suite is a collection or container of many related test cases. Grouping your test cases into adequately named suites makes it easier to manage them. You can group tests into suites according to several criteria:

• by type of test (e.g., integration tests vs. unit tests);

• by execution time (e.g., slow vs. medium vs. fast running tests);

• finally, by modules or areas in the application.

What a test suite looks like in practice will depend on the type of testing. A test suite can be as simple as a Word document folder for manual, scripted testing.

The Test Coverage measures the extent to which the system has undergone testing. It provides visibility into which tests cover system areas and which areas still require test documentation. Matrices, tables, or other visualizations can represent it to depict the coverage status. To calculate test coverage, you can use the following simple steps:

• Step 1: Count all the lines of code in your program.

• Step 2: Find out how many lines of code are covered by your current test cases.

Now, divide the number of covered lines (Y) by the total lines of code (X) and multiply the result by 100 to get your test coverage percentage. For instance:

If you have 500 lines of code in a system component and your current test cases cover 50 lines, your test coverage is 10%.

Test coverage helps us keep track of the features we need to test and the tests we have.

During a test run, testers execute test scenarios or cases, record the results, and identify and document errors or discrepancies. It allows the development team to address the identified issues and improve the quality of the software product before its release.

Here is an example of a Test report based on a test case example.

The status of a whole test case or its separate steps can vary depending on the context and purpose, but some common statuses include:

• Passed

• Failed

• Blocked

• In progress

• Skipped

• Retest

• Not run

Test run results are usually summarized in a test run report, which shows the statistics that allow project management to evaluate the quality level of the whole product or some parts before releasing it.

A Bug report is a technical document that provides a complete description of a bug, including information about the actual error (short description, severity, priority, etc.) and the conditions under which it occurred. In addition, the bug report should contain consistent terminology that describes the elements of the user interface and events that trigger the bug.

Let's have a look at an example of a bug report. (Read more about how to create a bug report)

Name (short description): The text field has a "NUMLOCK" value in the "Blood Product Number" control after pressing the "NUMLOCK" key

Status: Ready for QA

Project: Desktop

Affects versions: 1.0.5

Fix versions: 1.0.5

Priority: Medium

Assignee: Deepanshu Vashist

Reporter: Denis Terpylo

Description

Steps to reproduce:

1. Login
2. Create a new patient in DCR mode
3. Open RESUS on CDS Panel
4. Switch to Intake tab
5. Set “Whole Blood (WB)“ into “IV Fluid“ filed
6. Click on the “Blood Product Number“ field
7. Press the “NUMLOCK“ button on the keyboard
8. Take a look at the “Blood Product Number“ field value

Actual result: The text field has a "NUMLOCK" value in "Blood Product Number" field after pressing the "NUMLOCK" key

Expected result: The field value is clear

By reporting even seemingly minor bugs, you become a vital guardian against the emergence of interconnected complications. Imagine it as tending to a leaking pipe in your home; addressing it can save your abode from being submerged under unforeseen floods.

A test management system efficiently creates, organizes, and manages test cases,test runs, and test results. Tools like TestRail, Redmine, and Xray enhance testing processes. As the number of test cases grows, maintenance becomes crucial. These systems provide structured test cases with fields for steps, expected results, and data. Integration with Jira enables collaboration by linking failed tests to bug reports.

This integration streamlines issue tracking and resolution, ensuring prompt fixes. Specifying the Jira ticket number in the failed test case allows easy status checking and retesting. A test management system organizes and controls test cases tracks execution, and integrates with project management tools for efficient bug tracking and retesting.

A task (bug) management system is a software tool or platform designed to facilitate the organization, search, and resolution of tasks and bugs within a project or organization. It provides a centralized and structured approach to managing tasks and bugs from creation to release. Prevalent task (bug) management systems include Jira, Azure, and Bugzilla.

We can use these systems to provide various features and customization options for project management and bug-tracking needs.

Here's an example table for recording bug information on the functionality of the shopping cart:

A bug report can include a large amount of information. Retaining the title (briefly describing the bug), the environment, preconditions, steps to reproduce, expected and actual results, and attachments (additional files) is often essential. QA can also determine the bug's impact on the system by assigning a severity level. It is crucial to mark this severity level, especially if the bug is a blocker for other functions. In the Comments section of a bug report, QA and developer can discuss bug-related questions (how to reproduce, fix and retest the bug) with the involvement of other specialists (tech leads, PMs, etc.)

QA documentation is vital to successful software development by understanding its importance and utilizing appropriate documentation throughout testing.  QA documentation templates are valuable resources for standardizing the creation of test plans, test cases, and bug reports. Using templates saves time and ensures consistency in documenting the testing process. Talk to us to discover how QA documentation can benefit your software development.