Negative Testing In Software Testing: A Comprehensive Guide

Negative testing has its unique niche among software testing techniques. If we look at the definition, we will see that negative testing in software testing is a process that aims to ensure that an application or system can adequately respond to inappropriate or unexpected input data and user actions. This method involves including incorrect data in the testing QA process to verify the correctness of the program's responses. 

The essence of negative testing is to investigate the application's responses to input beyond normal expectations. The goal of this technique is not only to detect errors but also to ensure that the application remains functional and reliable even under the most unusual conditions. 

This guide will focus on the basic principles and practices of how to do negative testing. We will provide valuable guidelines and tips to help you improve your test procedures. You will also find test cases that give a deeper understanding of this technique, and the starting point can serve as a basis for creating your test scenarios. 

When to use negative testing in software testing 

Negative testing is often referred to as failure testing. It requires maximum creativity, as its intended purpose is to check how errors are displayed and what the user sees. This testing aims to identify potential flaws and determine the conditions under which the application may fail. 

Negative scenarios in software testing include:  

• The user clicked the OK button but did not enter data.  
• The data entered exceeds the allowable number of characters.  
• The name contains numeric values.  
• There is a special character in the name that the application does not anticipate it.  
•  The input contains invalid words.  

You can see how the software behaves in the above negative test cases. 

Goals negative software testing 

Negative testing is a method to analyze the system response to incorrect input data or non-standard user behavior. It differs from positive testing, which checks whether the application functions as expected, but from the point of view of incorrectly entered data. Negative testing aims to identify how an application handles erroneous or unexpected scenarios, leading to a deeper understanding of its resilience. 

The main objectives of negative testing in QA are:   

• Bug detection. Can identify bugs that positive testing did not detect. For example, positive testing verifies that the system handles data processing in the correct format. Negative testing reveals that when processing data in the wrong format, the system will warn the user and not perform the action.   
• Security Improvement. Can identify security bottlenecks that attackers use to gain unauthorized access to the system. Negative testing verifies the software correctly processes data potentially used for hacking the system, like empty or knowingly false data. You can ensure that the software remains stable and processes data accurately even when it receives too much data or data in the wrong format.   
• Increasing software quality. The quality of software increases as it identifies bugs and defects that could lead to an unsatisfactory user experience. Negative testing aims to verify that the software correctly handles data that may cause a user error, such as entering an incorrect password.   

Thus, negative testing complements positive testing by providing a more complete picture of the application's reliability and stability in various usage scenarios. 

Negative software testing and positive software testing 

In QA and testing services, there are primarily two fundamental approaches: 

Positive testing works correctly in the order defined in the criteria. If positive testing detects an error, the test fails. 

Negative testing is software testing that checks the application for unexpected input data and conditions. 

On the other hand, this same type of testing deals with what the system should not do. Negative testing goes beyond the requirements. It focuses on unexpected scenarios, so it is important to think outside the box.  

Combining positive and negative testing approaches is generally considered good practice. This strategy provides a higher coverage of the applications under test than using only one of the specified automated testing methodologies. 

Difference between positive and negative software testing 

The main difference between positive and negative testing is that in the latter case, the issuance of an exception is not an unexpected event. Negative testing expects errors, indicating the application correctly handles incorrect user behavior. 

In contrast to negative testing, positive testing is software testing that ensures the application works correctly under normal circumstances. This table provides an overview of the main differences between the two: 

Of course, these two types of testing are opposite, but they can be additional methods. 

What is boundary value analysis?  

Boundary value analysis is the process of testing between extreme points or boundaries of input values. The extreme values (e.g. Start-End, Lower-Upper, Maximum-Minimum, Just Inside-Just Outside) are called boundary values and the testing is called "boundary value analysis". The basic idea of this approach is as follows - we need to select the values of input variables at their:  

• Minimum;  
• Just above the minimum;  
• Nominal value;  
• Just below the maximum;  
• Maximum.  

What is "equivalence division"?  

Domain inputs are divided into different equivalence classes. This method takes all possible tests and puts them into classes. During testing, one test value is selected from each class. If you are testing an input field where you can enter numbers from 1 to 1000, there is no point in writing thousands of tests for all valid input numbers. Tests can be divided into classes according to the three sets of input data.

• Class for input data for all valid input data.  
• Class for input data with all values beyond the lower bound.    
• Class for input data with any value greater than 1000. 

Methods of negative testing in software testing 

Like any other type of testing, negative testing can traditionally be done on both the frontend and backend. Here are some valuable techniques and methods:  

• Exploratory testing. A well-known technique that helps software testers learn more about the application/system they are testing. In negative testing, testers know the requirements and functionality of the product, so their imagination and creativity are the only limitations.   
• Phase testing. Software testers enter random data into application/system fields and submit this data to see the product's behavior. These results find patterns in problems that cause bugs and failures.     
• Boundary value testing. This is an analysis method rather than a more practical technique. The analysis results determine the minimum and maximum number of characters each input field in the application/system under test allows. 
• The software tester can effectively test and verify the bounds beyond the allowed numbers using these results.  

In certain situations, these methods and techniques can be useful for finding bugs and possible failures.  

• Input Field Testing. Scenarios such as entering the wrong number of characters, special characters, emoticons, etc., are most common in negative testing. The famous SQL injection is one of the most popular examples of input field testing. In addition, inputting forbidden characters (e.g., Cyrillic letters) is an excellent example of negative testing of input fields.   
• Conformity between frontend and backend. The front end's behavior should match the backend's behavior. For example, an input field in the front end allows a maximum input length of 30 characters, but the user enters 31 or more characters. In this case, the front end should not allow this and should display an error. On the other hand, if the server side accepts more than 31 characters and sends them to the database, a crash may occur.      
• Web Session Testing. The web session should have some timeout rules. Negative software testing checks what happens in case of a slower internet connection. 

Let's continue to consider the negative testing in QA. 

How to run negative testing: Techniques negative software testing 

The following are the techniques used in negative testing.  

• Analysis of boundary values: The method involves writing test cases in a standard form for testers: checking values that are out of bounds, for example, for a field with values from 1 to 100.   
• Divisions by equivalence classes: A method of testing functionality by grouping test values into several "equivalence classes".   
• Error guessing: During the procedure, the tester sets special conditions that produce error messages. If possible, the tester tries to identify and fix the problem without crashing the application.   
• Checklists: A basic yet critical, method in QA that documents the conditions under which testing is performed. Usually used in teamwork.   
• Antipatterns: If there are design patterns, there can be antipatterns. If a pattern is the best way to solve a problem, then an antipattern is a solution that doesn't work. Antipatterns are the perfect source of negative tests.   
• Exploratory testing: A method that increases the tester's skills and understanding of the application. Used in parallel with other tests. Makes the "big picture" of the application clearer - in what conditions the application works, in what conditions it doesn't.   
• Small automation: It catches memory errors, gross errors in code, and other unexpected problems on the product. A few hundred or thousand runs are run several times to see what happens to the application.   
• Fuzzing testing: Inputting random data that can cause unexpected crashes, system crashes, and other errors. This method has no "expected results" (unlike other negative test cases). Roughly speaking, it's just observing what happens when some random data is fed.   

State transition testing: A bug-finding technique built on the concept that the software must be in one particular state at a point in time. The application should be in its normal state until a problem appears. 

Typical negative testing scenarios are appropriate 

Negative testing is aimed at identifying possible application failures in various situations. Below are some possible examples of such situations:  

• Filling in mandatory fields. Some applications and web pages contain fields marked as mandatory. To test the application's behavior, create a test that leaves the mandatory fields blank and analyses the application's response. For example, the application may display a message box asking the user to fill in the required field. Your test should then interpret the application's behavior as correct when dealing with invalid data. 
• Correspondence between data types and field types. Typically, dialogue boxes and forms contain controls that can accept data of a certain type (for example, numeric, dates, text, and so on). To verify this functionality, create a test that enters incorrect data into a control. For instance, a test that enters a letter in the UpDown edit field or the invalid value “30.13.2010” in the date field.  
• Allowable character count. Some web pages and applications contain fields that allow a limited number of characters. For example, the value of the Username field for applications and web pages must be less than 50 characters. To test the application's behavior, create a test that enters more characters than allowed in the field.  
• Allowable data limits and constraints. Applications input fields that accept data within a certain range. For example, this might be an edit field in which you enter an integer between 10 and 50, or an edit field that accepts text of a certain length. To evaluate the behavior of the application, develop a negative test that uses a value below the minimum allowed or above the maximum allowed for a given field. 

Another example negative test case is entering data that exceeds the data type limits. For example, an integer value can typically contain values between 2,147,483,648 and 2,147,483,647 (the size limited by the number of bytes in memory).  

To test the application's behaviour, you can create a negative test that enters a value that is out of bounds. For example, the test might enter a large number (100,000,000,000,000,000) in an integer field.

• Sensible Data. Some applications and web pages contain fields that have reasonable limits. For example, entering 200 or a negative number as the "Your age:" value is prohibited. To test the application's reaction, create a negative test that enters invalid data into the specified field.   
• Web Session Testing. Some web browsers require a login before opening the first web page. To verify that these browsers work correctly, create a test that attempts to do so without logging in to open web pages.

All of these negative testing scenarios help to identify and resolve problems, improving the reliability and functionality of the software. 

Examples of negative test cases 

Test cases are the starting point to begin testing. Documenting negative test scenarios is just as important as documenting other test scenarios (positive, regression, smoke tests, etc.). In this section, we will provide some specific examples of negative testing scenarios covering both the interface and server side. 

Negative testing scenarios for the front-end interface 

• Create a user with an email address that already exists on the system/db.  
• Log in with an incorrect password.   
• Verify that the "First Name" and "Last Name" fields accept special characters, numbers, and emoticons.  
• Verify that the "Phone Number" field accepts letters.  
• Check if the user can fulfill an order using an expired credit card. 

Server-negative testing scenarios  

• Log in without providing a user token.  
• Log in with an expired token.  
• Log in as user #1 using user #2's token.  
• Complete an order from a user who has not saved any payment method. 

These are just a couple of examples used in negative testing by our experts. 

Steps to Conduct Negative Testing

Following how to create a software test plan, here are the main steps to take when conducting negative testing: 

Conducting negative testing requires careful planning and attention to detail. ‍ 

Benefits of software negative testing 

Let's take a look at the key benefits of negative testing by the Luxe Quality team: 

Software negative testing provides a deep understanding of the limitations and potential weaknesses of an application, resulting in more reliable, secure, and higher quality software solutions. 

Conclusion  

As you can see, software negative testing improves the testing coverage of your application. Using negative and positive testing approaches together allows you to test your application with any possible inputs (both valid and invalid) and can help you make your application more stable and reliable. We trust that this article contains fresh and useful information that you can implement in your testing process.