Our client encountered persistent difficulties in validating the visual aspects of their user interface (UI). Using their traditional automation tools, they struggled to verify complex UI components (elements that are missing in DOMs, canvas elements, graphical images having details and charts). They also struggled with automating specific parts of the dynamic content application and verification of layout consistency across various screen resolutions. Additionally, manual visual testing was prone to human error, as verifying minor UI changes can be very difficult, and the process was very time-consuming as well. These limitations led to a cascade of issues, including frequent regressions, a gradual decline in user experience quality, and disruptions in release schedules, resulting in significant downtime. The reliance on manual testing exacerbated these challenges. Human error, time-consuming verification processes, lack of consistency in test cycles, difficulties in issue reproduction, limited scope, and high investment in testing tools hindered efficient testing. The client sought a comprehensive solution that will address these challenges.

The customer is one of the largest investment management firms providing solutions to institutions, financial professionals, and millions of individuals worldwide.

• Human error: When comparing images, there is a high chance that the manual tester will miss minor changes.
• Dynamic content in application: Dynamic content in applications presents a big challenge during visual verification. It's difficult to verify the content and layout of the screen while ignoring the dynamic elements.
• Layout verification across resolutions: Layout verification is a critical aspect of visual testing, but it encounters challenges due to varying resolutions and device sizes.
• Verification on elements which are missing in DOM: Traditional UI automation struggles with elements like canvas elements, which are undetectable in the DOM. This results in difficulty testing applications consisting of these types of elements.
• Info Graphics data verification: Infographics may be present in the DOM, but traditional automation cannot verify the data of the infographic or whether the correct image is displayed in the UI. This can cause discrepancies in the UI and decrease user trust.
• Time intensive verification: The inherently time-consuming nature of manual visual testing poses a formidable obstacle, with the verification of individual modules demanding substantial time investments, ultimately impeding the overall testing process and delaying the delivery of software updates.

Framework Design:

To address these challenges, we designed a framework (Gem-Eye) that utilized:

• Advanced image processing algorithms, including template matching and object detection.
• Intelligent logic to handle dynamic elements by defining exclusion zones or employing dynamic identification algorithms.
• Established tolerance thresholds to validate layout consistency, considering pixel differences or layout shifts.

Implementation:

Gem-Eye seamlessly integrated with existing testing tools, such as Selenium, to facilitate streamlined visual verification within established workflows. The framework included:

• Development of robust script logic to capture pertinent screenshots and initiate visual comparisons automatically.
• Integration of automated reporting mechanisms to promptly identify and highlight visual discrepancies for efficient debugging and resolution.

To address the challenges and deliver comprehensive visual testing solutions, our framework offers the following key features:

• Comprehensive Defect Detection: Automatically identifies visual defects by comparing baseline images, ensuring thorough validation of visual elements.
• Dynamic Content Handling: Capable of ignoring or comparing dynamic content against multiple baselines, enabling comprehensive testing of personalized applications.
• Layout Verification: Ensures consistent layout across various resolutions, enhancing user experience and brand consistency.
• Verification of elements without interacting with DOM: Allows you to write automated tests for elements like canvas objects (missing from DOM) and verify data within infographics (present in DOM but data not exposed), ensuring comprehensive visual UI testing and user trust.
• Seamless Integration with Existing Automation Projects: Easily incorporate Gem-Eye into existing projects for seamless visual verification within established automation frameworks.
• Verbose Reporting Support: Detailed reporting highlights differences in screens, enabling efficient debugging and communication within the team.
• Custom Reporting Integration: Seamlessly integrates with custom reporting tools like Serenity and Jewel, enabling tailored visualization of test results to meet specific team needs.
• Support for Cucumber BDD: Very easy to incorporate project and write the automation for visual UI testing because user will have the flexibility to write test cases in Behavior-Driven Development (BDD) format without the need for additional coding.

Gem-Eye leveraged advanced computer vision techniques to deliver several benefits:

• Precision and Reliability: By meticulously analyzing every pixel, Gem-Eye detected even the most subtle visual imperfections, surpassing human testers in terms of accuracy and thoroughness. This enhanced the overall quality assurance process.
• Easy detection of visual defects: Gem-Eye provided detailed visual differentials, enabling efficient debugging and communication within the team. These comprehensive reporting features empowered testers to identify and address visual defects swiftly, fostering collaboration and expedited issue resolution.
• Certification for layout across multiple devices: Gem-Eye simplifies the process of achieving consistent layouts across a wide range of devices. Its features streamline layout verification, ensuring your application meets certification requirements for various screen sizes and resolutions. This reduces the risk of layout inconsistencies and streamlines the path to device certification.
• Efficiency in Regression Testing: Gem-Eye significantly improved regression testing efficiency by automating visual verification and delivering robust reporting. Testers could execute tests faster, detect visual regressions early in the development cycle, and ensure continuous improvement in application quality, leading to accelerated release cycles and enhanced user satisfaction.
• Fast Framework onboarding, automation and execution: Gem-Eye's seamless integration minimizes disruption to existing workflows and scales to meet evolving testing needs, ultimately enhancing efficiency. With support for Behavior Driven Development (BDD), users can quickly begin automation by incorporating steps directly into their current UI tests. Furthermore, Gem-Eye integrates effortlessly within CI/CD pipelines just like any other framework, streamlining the entire process. This combination empowers users to leverage powerful automation without a significant investment in time or effort.