4. Automation tests
4.3. Implementation and Execution
Behavior-Driven Development helped us to be effective with our automation tests on any project. That’s why we successfully implement this approach now in each of our project.
Implementation workflow of automated tests in company “Starky’s Club”:
• Use case is being written in “.feature” file using Gherkin language.
43
Figure 4.1: Implementation of “user login” use case using Gherkin language
• Quality Engineers takes use case from a “.feature” file and using Selenium framework (framework for UI automation testing) builds the steps in separate “.py” file, which will simulate the user behaviour in browser.
Figure 4.2: Steps in “.py” files, which simulates the user behaviour in UI
• Those files are then review and saved in GitHub (described in Section 1.6)
• After that any team member could manually execute this test on a cloud server using Jenkins (described in Section 1.6) and Docker (described in Section 1.6), which builds up virtual testing environment in cloud (Amazon Web Services). After the execution, Jenkins could provide email or chat notifications with a result, videos, screenshots. Test could be also scheduled, for example, to be executed daily.
44
Figure 4.3: Jenkins dashboard screenshot of the same login tests
4.4. Results
Our own framework for user interface automated tests gives us more quality time, that we could spent on testing new features, while regression testing is provided by the automation. Gherkin syntax gave us possibility to document test cases and new features without having any separate documentation for it – use cases are self-explanatory in a “.feature” files.
Also, using Behaviour-Driven Development gives Quality Engineers an opportunity to work closely with business, customers, Product Owners and also to speak with them on the same level – without going to any technicality.
Automation framework also drastically improved our 2 KPIs:
• Automated_Tests_KPI – measure the percentage of test cases automated against the total number of test cases. Number is now up to 72%.
45
Tab 4.1. Automated_Test_KPI calculation history in “Starky’s Club”
Date Total TC # Automated TC # Automated_Tests_KPI
2018-06-01 156 18 11,5%
2018-07-01 170 23 13,5%
2018-08-01 196 26 13,3%
2018-09-01 224 40 17,9%
2018-10-01 258 54 20,9%
2018-11-01 271 62 22,9%
2018-12-01 285 80 28,1%
2019-01-01 312 94 30,1%
2019-02-01 349 97 27,8%
2019-03-01 360 126 35,0%
2019-04-01 385 148 38,4%
2019-05-01 403 176 43,7%
2019-06-01 431 191 44,3%
2019-07-01 479 203 42,4%
2019-08-01 512 233 45,5%
2019-09-01 547 282 51,6%
2019-10-01 589 308 52,3%
2019-11-01 634 351 55,4%
2019-12-01 689 402 58,3%
2020-01-01 736 461 62,6%
2020-02-01 769 524 68,1%
2020-03-01 815 586 71,9%
• Manual_time_KPI – time spent on manual testing in a 2 week iteration (in a number of days). That number is now at 3.2 and with that number we are really happy, as we see it right on a balance, between being overwhelmed with manual testing and not being in touch with a product.
46
Tab 4.2. Manual_time_KPI calculation history in “Starky’s Club”
Date Manual_time_KPI
2018-06-01 5,4
2018-07-01 4,8
2018-08-01 5,5
2018-09-01 5,8
2018-10-01 5,6
2018-11-01 6,0
2018-12-01 7,0
2019-01-01 6,5
2019-02-01 6,8
2019-03-01 6,5
2019-04-01 6,2
2019-05-01 6,0
2019-06-01 5,2
2019-07-01 5,5
2019-08-01 5,1
2019-09-01 4,7
2019-10-01 4,5
2019-11-01 4,2
2019-12-01 3,8
2020-01-01 3,2
2020-02-01 3,5
2020-03-01 3,2
Thanks to the implementation of automated tests and process improvements amount of code in company was also increasing significantly, as we can see in Figure 4.4. We can also see a downfall during 04.2019-05.2019 due to month of refactoring (code optimisation).
Figure 4.4: Evolution of code base size in company “Starky’s Club”
47
Conclusions
This Master thesis is focused on improving the development process life cycle, project management and testing activities in software company by using TMMi approach, Agile transition and automation testing. Particularly those activities were done in a company “Starky’s Club”.
My main goal was to analyse software testing department of the company “Starky’s Club” and increase its efficiency. Section 2 was dedicated to the detailed analysis of the company itself – their organizational structure, strategy and concept were described; also, SWOT analysis was provided.
Different Software Development Lice Cycle were also explained in Section 1.1 to provide better understanding to the problem. High-level overview of TMMi approach was provided in Section 1.5, which was later used in practical part of this Master thesis.
According to the KPIs, that were established in Section 3.2 and evaluated later in Sections 3.6 and 4.4, my main goal was achieved. Suggested implementations helped to improve all of the established KPIs significantly, helped to resolved and find issues in software more efficiently and in a timely manner. Code base was also constantly raising, which is a sign, that there aren’t any signifact bottlenecks in a company process. Employees are happy to work in a such workplace and that attract new customers.
This Master thesis could help small and medium size software companies not only to improve performance of their testing department, but also make a transition towards Agile methodologies. Overall reasons for software development company to apply Agile methodologies were states in Section 1.1; practical implementation of Scrum was described in Section 3.4; implementation of Kanban was stated in Section 3.5.
In addition to main goal, framework for automated testing was established and implemented in Sections 4.1-4.3. For that Behavior-Driven Development approach were taken and software tools like PyCharm, GitHub, Jenkins and Docker were used. For implementation of automation testing were used Python and Gherkin syntax. Automation testing helped Quality Engineers to improve test coverage, time spent on regression testing was decreased together with time spent on manual testing. That gave testing department more time to provide better test analysis and to dig deeper into products.
Owners of company “Starky’s Club” are satisfied with results of implementations, that were done during this Master thesis and would like to continue moving in the same way in a future.
48
List of references
AGILE BUSINESS CONSORTIOUM, 2017. What is DSDM. [online]. Available it:
https://www.agilebusiness.org/what-is-dsdm
AGILE TESTING VALUES, 2020. Official website [online]. Available at:
https://softwaretestingfundamentals.com/agile-testing/
ANDERSON, D. A., LINDEN-REED, J., 2010. Getting Started with Kanban fot Software Development.
Cary: DZone. ISBN 1934238759.
ANDERSON, D. A., 2010. Kanban: Successful Evolutionary Change for Your Technology Business. Blue Hole Press. ISBN 978-0-9845214-0-1.
BUCHALCEVOVA A., 2018. Zlepšování procesů při budování informačních systémů. Praha: Oeconomica.
ISBN 978-80-245-2235-7. [cit. 14.11.2019]
BURROWS, M., 2014. Kanban From The Inside. Seatlle, WA: Blue Hole Press. ISBN 978-0-9853051-9-2.
BRUCKNER, T., BUCHALCEVOVÁ, A., CHLAPEK, D., STANOVSKÁ, I., VOŘÍŠEK, J., ŘEPA, V., 2012. Tvorba informačních systémů: principy, metodiky, architektury. Praha: Grada Publishing. ISBN 978-80-247-4153-6.p
BRECHNER, E., 2015. Agile Project Management with Kanban. Microsoft Press. ISBN 978-0735698956.
CASTILLO, F., 2016. Managin Information Technology. Springer International Publishing. ISBN 978-3-319-38890-8.
COCKBURN, A., 2008. Using both incremental and iterative development. STSC CrossTalk (USAF Software Technology Support Center), str. 27-30. Available at:
http://www.se.rit.edu/~swen-256/resources/UsingBothIncrementalandIterativeDevelopment-AlistairCockburn.pdf
CRISP, A., 2017. Crisp: Kanban. [online]. Available at: https://crisp.se/gratis-material-och-duider/kanban DEEMER, P., BENEFIELD, G., LARMAN, C., 2010. The Scrum primer. [online]. Available at:
https://scrumprimer.org/scrumprimer20.pdf
DOUCEK, P., 2010. Informační management. 1st. vyd. Praha: PROFESSIONAL PUBLISHING. ISBN 978-80-7431-010-2.
DOUCEK, P., MARYŠKA, M., NEDOMOVÁ, L., 2013. Informační management v informační společnosti. 1. vyd. Praha: PROFESSIONAL PUBLISHING. ISBN 978-80-7431-097-3.
DOUCEK P., Řízení projektů informačních systémů. 2. vyd. Praha: PROFESSIONAL PUBLISHING. ISBN 80-86946-17-7.
HAMMARBERG, M., SUNDEN, J., 2014. Kanban in Action, Shelter Island, NY. ISBN 978-1-617291-05-0.
HIGHSMITH, J., 2002. Agile Software Development Ecosystems. Addison-Wesley. ISBN 0-201-76043-6.
49
ISTQB Glossary, 2018. Official website [online]. Available at: https://glossary.istqb.org/search/
JANES A., SUCCI G., 2012. Proceedings of the ACM international symposioum on New ideas, new
paradigms, and reflections on programming and software, ACM. ISBN 978-1-4503-1562-3. [cit. 29.10.2018]
LOCK D., 2007. Project Management (9th ed.). Gower Publishing, Ltd., 2007. ISBN 0-566-08772-3.
MANIFESTO FOR AGILE SOFTWARE DEVELOPMENT, 2001. Official website [online]. Available at:
http://agilemanifesto.org/
PRINCIPLES BEHIND THE AGILE MANIFESTO, 2001. Official website [online]. Available at:
http://agilemanifesto.org/principles.html
MEASEY P., WOLF R., BERRIDGE C., 2015. Agile Foundations: Principles, Practices and Frameworks [online]. Available at:
https://ebookcentral.proquest.com/lib/vsep/detail.action?docID=1759633&query=Agile MINDTOOLS, 2018. SWOT analysis [online]. [cit. 28.10.2018]. Available at:
https://www.mindtools.com/pages/article/newTMC_05.htm
MYERS G. J., SANDLER, C., BADGETT, T. 2011. The art of testing. 3rd ed. HOBOKEN, N.: John Wiley.
2012. ISBN 978-1118031964.
OCTABA H. PIETINI, M. 2008. Software Process Improvement for Small and Medium Enterprises:
Techniques and case studies. Information Science Reference. New York: Hershey. ISBN 978-1-59904-906-9.
PETERSON D., 2009. What is Kanban? [online]. Available at: https://kanbanblog.com/explained/.
PHILIPS J., 2003. PMP Project Management Professional Study Guide. McGraw-Hill, 2003. ISBN 0-07-223-062.
PROJECT MANAGEMENT INSTITUTE, 2017. A Guide to the Project Management Body of Knowledge (the PMBOK Guide). Project Management Institute, 2017. ISBN 978-1-62825-184-5
PRIYARANJAN K., 2018, Agile Framework: A Kickstart To Scrum. [online]. Available at: https://www.c-sharpcorner.com/article/agile-framework-a-kickstart-to-scrum/
SCRUM INC., 2015. The Scrum Guide. [online]. Available at: http://www.scrumfuides.org/scrum-guide.html. [cit. 01.03.2020]
STEVENS M., 2002. Project Management Pathways. Association for Project Management. APM Publish Limited, ISBN 1-9034-94-01-X. [cit. 28.02.2020]
SUBRAMANIAM V., HUNT A., 2011, Practices of Agile Developer: Working in the Real World, Raleigh, North California: The Pragmatic Bookshelf, ISBN 0-9745140-8-X. [cit. 28.02.2020]
SVATÁ V., 2007. Projektové řízení v podmínkách ERP systémů. Praha: Oeconomica. ISBN 978-80-245-1183-2. [cit. 03.02.2020]
TAGUE, N. R., 2005. Plan-Do-Study-Act cycle. ASQ Quality Press. ISBN 978-0873896399.
50
TMMI FOUNDATION, 2018, Test Maturity Model Integration (TMMi), Release 1.2 [online]. [cit 19.04.2020] Available at: https://tmmi.org/tm6/wp-content/uploads/2018/11/TMMi-Framework-R1-2.pdf TMMI FOUNDATION, 2018, TMMi in the Agile world, Version 1.4 [online]. [cit. 14.03.2020] Available at:
https://www.tmmi.org/tm6/wp-content/uploads/2020/01/TMMi-in-the-Agile-world-V1.4.pdf UKEssays, 2018. Waterfall Methodology in Software Development. [online]. Available from:
https://www.ukessays.com/essays/computer-science/waterfall-methodology-in-software-development.php?vref=1
WITZEL M., 2003. Fifty key figures in management. Routledge. ISBN 0-415-36977-0.
51
Annexes
Annexes will represent extensive information, which was made by author.
Annex A: SWOT analysis of company “Starky’s Club”.
Strengths Weaknesses
- Synergic teams - Modern technologies
- Well-know partners throughout industry - Ability to deliver products fast and in
agile way
- Compatative rates - Quality of the products -
- Small numbers of employees - Relatively new company - Lack of IT security certificates - Small amount of senior developers - Marketing
Opportunities Threats
- Fast growing market
- Ability to work with partners abroad - Collaboration with clients on the future
projects
- Highly competitive market - Economical recession - New technologies - Changing of trends