Since a last decade or two, therehave been tremendous changes in the software development methodologies. Varietyof tools besides agile development, extreme programming, and test drivendevelopment have changed the philosophy and are still modifying how thesoftware development takes place. Needless to say, there is a constant need forchange in developing and owning a software development. During the course ofsoftware development project, however, there is a need of continuousmaintenance and growth of the software product as it can be hit by any kind ofunexpected and unplanned changes. One of the biggest challengesthat is faced during software development phase is to handle traceability; abi-directional trace between the requirements and components implementations aswell as between requirements and acceptance tests. In such, manual maintainingof the traceability can be cumbersome and so for providing better andsustainable results, the need for an automated build acceptance test surfaces.
Build Verification Test or theBuild Acceptance Test is a set of tests that helps the testing team to go astep back for a step ahead. These tests are run on Builds- each one, and newlycreated one to ensure that a given module is ready for testing. These tests arethe core functionality test cases that ensure that the application is stableand can be tested thoroughly. Typically, the entire process is automated. Thetest helps subsequent stages and team side-step half baked work and builds thatare too early or un-ready for testing. When these tests give a green signal,they lead the build to verification stages and bug-fixing tables.
In simpler words, BVTs are run ondaily builds and if the BVT fails, the build is rejected and a new build isreleased after the bugs are fixed. The build tests address core functionalitytest cases, application stability and are thorough in their focus. This canentail main functions of the module, data-and-format alignment, and addition offiles or improvements, new and improved files and culminate into overallproject efficiency by making room for discovery and correction of defectsbefore delivery. Keep in mind that the success ofBVT depends on which test cases are included in BVT. For that to happen, onehas to consider the following points:- Include only critical test cases in BVT- Write effective test cases scripts- Include right file formats- Include useful information for yes/no diagnosis- Peek into and flags around failure causes,choice and degree of automation, insertion of break-build penalties, presenceof all new modified files etc.- All the test cases that are included should haveknown expected results- Ensure that all included critical functionalitytest cases are sufficient for application test coverageONLY CRITICAL TEST CASESThe most important point toconsider is the choice of test cases, given its regression-test flavor.
There isa lot of thrust on inclusion and exclusion hence, picking only critical testcases and including test coverage focus is a good approach to start with. This canfacilitate new critical case with consistency of different configuration check.This helps to avoid build failure that is prompted from modules and test casesare unstable or new. It is worth knowing that teststake time, resources and server muscles and so it is pertinent to run thosetests first where propensity of spotting failures is more or the improved timeto fault detection comes in. this is helpful to shave away necessary time,cost, resources and completeness of a build along with ease for testing teams. Criticalityand application test coverage should be balanced well in the selection oftests. Also, important functions of the module, clarity on expected results,verification-strength or implementable-readiness for application functions isvital to be considered.
TESTS SHOULD BE STABLEThese tests should devoteadequate time on making cases and going for stable test cases. It is less probableto weed out critical bugs or software-breakers if test cases are not formulatedwell. That is the reason why a build has to be stable for strong and gainfulexecution of test cases in the ensuing testing process. Likewise, carefuldesign along with the coverage of basic functionality is immensely important. EXAMPLEPresently, Agile is one of themost commonly used methodology in software development life cycle, with itspre-defined short iteration cycles for 2-3 weeks that delivers quality product.
QA teams work with developers to achieve the goal of BVT. Under BVT, thedevelopers set up or create the VMs/Machines for testing, set up theenvironment build deployment, verify build and inform the QA team to furtherstart testing. If this entire process isautomated, the BVT process will easily save around 3-4 man days on eachiteration besides being reliable and without manual errors. To ensure thesuccess, following steps will be followed:1. Set up Bamboo or Jenkins in a way that thenightly test builds would start at the end of every sprint or manually, whichpulls the latest code from the GIT/SVN, executes unit test cases, generatesbuild and uploads the build on Artifactory. Once the build is uploaded; it willstart the automated BVT plan. 2.
The VMs or machines can be generatedautomatically using Amazon Web Service, Dockers or VMWare ESXi Server whichhave their own required software already installed. APIs can be called usingPython, Java or even shell script, whichever is applicable.3. After the VMs are available, ensure that the webapplication server and database software has been installed successfully. Toautomate this option, Ansible can be considered a good option. 4.
With the latest build from Artifactory, deployit on Web Application server with the help of Ansible. Once it is deployedsuccessfully, check if it can access a web page. 5. After the build is deployed, start executing thepre-defined test cases which encapsulate the core functionality of the product.Automate those test cases as per the requirement using any language or toolsuch as Java, Python, shell script and selenium etc. 6. Add the test step results into Jira directlyfrom the Test script to avoid manual dependencies.
7. Send the final status email to the manual QAteam that the build is stable/not stable for further testing. This is one of the examples toautomate build verification test process in Agile. This is quite understandable thatthe automated acceptance tests play the key role in software developmentprocess. Automated BVT is very effective for daily as well as long-term buildsand helps saves time, cost, and resources of the company. Given the nature ofthe build acceptance tests, their size and scope, assortment of crucialfunctionality areas would be apt that would refine the coverage of the thesetests for the overall goal of empowering the actual testing that follows.