I followed Automation Guild 2017 (automationguild.com) online conference organized by Joe Colantonio. 33 practical presentations for test automation developers were recorded and made available for watching later on. Tracks were Frameworks, Best practices, Automation techniques, DevOps in Test, and Mobile, API & Unit testing.
The test automation field is changing due to 1) systems under test are becoming technically more complex and demanding, and 2) because the software industry itself is evolving.
Firstly, systems to be tested today may have demanding user experience requirements, highly dynamic implementation and responsive design for multiple devices at global scale. Efficient parallel cross-platform test execution, maybe as a service in cloud, is one relevant approach for test automation strategy. Connection to the physical world (IoT, VR) requires new and innovative ways to automate and e.g. include hardware into test setups. Pokemon Go was mentioned as an example of a test automation challenge.
Secondly, as SW development is evolving through agile and continuous integration to continuous delivery and DevOps, test automation literally needs to be at the core of the process: developed at the same pace with the product itself, coverage must be high and feedback reliable. Test automation should be available as a (self-)service for the automated process and all teams. John Smart presented Serenity BDD as a proactive approach for test automation to help the whole team. Seb Rose referred Cucumber not only as a testing tool, but as a collaboration tool.
Prior to the conference, Nikolay Advolodkin made a small poll (35 responded) about realities in functional test automation. Flaky tests, i.e. unreliability, was the most often reported problem. To his question "How many tests can you run per day that will give the correct result with 95% accuracy?" 20 out of 35 responded less than 50 tests.
Many presentations were about developing a good test automation framework. Despite the technology in question, it is about good design and maintainability. Test automation must be considered as a professional SW development and all good practices like version control, code reviews, design patterns like DRY and constant refactoring should be applied for automation development as well.
A significant part of the presentations were about integrating test automation development as part of the modern DevOps and continuous delivery/deployment pipeline. Testing must not only follow new approaches, but more and more be essentially on top of things to enable fast and reliable development and successful automated deployment.
One good mindset for test automation developers is to follow different tools embracing their approaches, whether it relates to TDD, BDD, ATDD etc. The design philosophy and patterns behind may be very well utilizable in context of other frameworks. Nikolay Advolodkin’s presentation about maintenance challenges of keyword driven test automation and a Page Objects design pattern as a solution is a good example. His 3 actionable tips for test automation developers are: 1) start using Page Objects, 2) use TDD and 3) Stop being WET (i.e. follow DRY pattern), to develop code being more resilient to change.
Dave Haeffner presented a concrete “rubric” to grade Selenium code by pointing out typical design flaws such as web technology related details at test case level. Greg Paskal presented "ROI robbers" of test automation: 1) Over abstraction, 2) Object recognition issues, 3) Synchronization troubles, 4) Test data strategies and 5) Scalability related issues.
Due to a number of technologies utilized and integrated (microservices trend) a key requirement for test automation frameworks and tools evaluation is expandability. APIs, plugins, libraries or similar means are needed to integrate with a set of planning, development, testing, deployment and operations tools. Open source field nowadays is mature and can be taken as an approach to build test automation. Robot Framework fulfills this criteria. Pekka Klärck introduced Robot Framework, how to extend it and an ecosystem around it. There are an active community and plenty of tools and libraries available to extend the framework for web, mobile, database or API testing for example. Pabot is a solution for parallel test execution.
Cloud services, virtual environments and containerization are changing the way test environments are managed and a bunch of related technologies must be introduced to the test automation engineer’s toolbox. Automated visual testing is one trend to be considered as part of testing strategy. Automation in general, not just automating tests, is another field of rising interest and need.
About the author:
Tero Saarimaa is a Senior Test Automation Developer at Knowit.
