One of the main challenges that the QA team faced was to cover an endless support matrix of mobile applications across different application versions, mobile operating systems and vendors and to verify that all the data is presented correctly across our products. Third party applications are independent, constantly changing, require updates and there is no control over them. Integration of third party apps requires constant data testing, especially post-app upgrades. Due to unpredictable changes, our software was constantly breaking, and hot fixes were unavoidable, not because of bugs in the system but due to third party application version upgrades. This problem was solved by building a team responsible for creating test data for each supported application version, later automating the process and finally adding it to our CI/CD pipeline. In this presentation, I will share how we resolved this issue · Identifying the applications that required upgrade · Inserting unique test data · Logging the data to create a reference file · Validating the new data versus the reference · Automating the steps · Expanding the test references · Adding the process to gated check-ins and to nightly builds Today, for every check-in that a developer commits to, over 10,000 application versions are verified before the commit is approved. The solution greatly improved our quality and reduced hot fixes, from a hot fix every release to no hot fixes at all.
November 27 @ 14:00
14:00 — 14:45 (45′)