Manual and automated testing of chatbot implementation for Lernix Assistant

Lernix Assistant is actively integrating chatbots into the website of a network of public schools, which simplifies access to important dates, schedules, and other relevant information, making the educational process more organized and accessible to students and their parents. 

Before our specialist arrived, testing was carried out manually by PM, which resulted in a large number of errors and development delays.  

Our QA engineer was responsible for developing testing documentation, functional test cases for chatbots and automating them on the project. In the process of performing these tasks, several nuances came to light that we would like to share. 

Our QA engineer employed a variety of tools to ensure the quality and efficiency of testing: 

• Automated testing: Used WebdriverIO with JavaScript for functional and regression testing, TestRail provided centralized test case management. 
• Continuous integration and deployment: Jenkins automated CI/CD processes, integrated with GitHub for version control.  
• Performance testing: Apache JMeter for testing under high user loads. 
• Security and compatibility testing: Performed security testing using OWASP ZAP for vulnerability analysis and utilized BrowserStack for cross-device and cross-browser compatibility testing. 

The project involved integrating sophisticated chatbots designed to streamline access to educational content and administrative functions on the private school network's website. These chatbots are capable of handling complex interactions such as scheduling, accessing course material, and managing administrative queries.  

To ensure comprehensive functionality, advanced testing protocols were established that included both simple and complex user interactions, such as retrieving specific data, navigating through menus, and processing student inquiries. Each functionality was meticulously tested through a series of positive and negative scenarios to confirm the chatbots' reliability and effectiveness, ensuring their performance adhered to the high standards required for educational applications. 

• 300+ test cases automated significantly enhanced product stability and reliability through extensive automation 
• 250+ bug reports created enabled timely identification and resolution of errors, improving software quality 
• 70% of test cases automated significantly reduced ongoing testing time and ensured extensive coverage of major functionalities 
• Comprehensive security testing included vulnerability assessments to improve resilience to cyber threats and ensure data protection compliance 
• Localization testing ensured chatbot functionality across multiple languages