Dolby amplifies testing success with ServiceNow automation

ServiceNow’s twice yearly upgrades and quarterly patches required long and rigorous testing to ensure ongoing functionality of complex, constantly evolving processes was complete. These frequent changes meant testing was complicated and required ongoing updates by a team of business analysts, application developers and maintenance specialists. These teams spent over nine weeks completing manual regression testing for each upgrade — a process that proved to be an ineffective use of their time. With ServiceNow’s frequent release schedule, a 9-week testing cycle made it difficult to achieve complete test coverage across the many permutations and combinations of test data and different scenarios across multiple applications and region-specific versions.

Scaling test automation quickly for 90% coverage

The infrastructure team worked with the existing TCoE at Dolby to move forward with Tosca’s scriptless test automation, leveraging documented best practices. They were able to engage a handful of senior consultants from Tricentis in conjunction with a project manager to quickly and significantly scale test automation for ServiceNow through the course of multiple planned sprints. The scope would cover ServiceNow, as well as its integrations with Peoplecare HR and AskIT – Dolby’s internal knowledge and ticketing portal.

The focus of the initial project was to automate as much as possible while ensuring test cases developed were reusable for future upgrades and releases. A risk-based test approach was used to assign priority for which tests would be automated in each sprint. Priority was based on frequency and “damage class” identified by the team. The result was 90%+ automated test coverage for all key areas of the application.

A secondary focus was creating the ability to execute test cases in an offline, unattended mode. This allows test execution to happen during non-working hours and frees up business analysts and application developers to focus on higher priority tasks during the day.

To date, the team has automated 200+ test cases to support future releases of ServiceNow. 95% of those tests are successfully executed in unattended mode on the Tricentis Distributed Execution Server (DEX).

“We defined success metrics before we started the project. If automation can bring down regression testing time to a few hours and minimize defect leakage to production, then the project would be considered successful.”

• All key areas of ServiceNow are automated and have 90%+ test coverage
• Automation was achieved using a slim resource model
• Risk-based testing approach assigns priority to test creation, ensuring the most business-critical functions are tested first
• 95% of testing is executed offline in an unattended mode during non-working hours – allowing the team to focus on high priority tasks during the workday
• Test automation helps streamline ongoing maintenance of ServiceNow
• Highly reusable test cases are available for all future upgrades and releases
• 7 different ServiceNow applications and portals automated
• Test Data Management ensures test data is available for a wide range of test scenarios, extending test coverage
• Test cases are stored using Tricentis qTest for easy organization, management, sharing, and tracking