Not trying to teach you the ABCD of test automation frameworks…

If you search in Google, you will get numerous sites/documents with lot of information on ‘what is a test automation framework’.

Also during your trainings (if you are trained in Software Testing or Testing Automation), you might have learned a lot about the same topic.

Or you might have read in books/articles about this topics…

So I think I need not tell you all that again.

Let’s move one level up…

Have you ever thought, “Okay fine, now I understand what is a framework and moreover what is a test automation framework. But, why should I/we use it ?”

That’s where I’m going to help you. Please read my other posts to know how I’m trying to explain it from my perspective.

In Test Automation, not all the failed cases are defects

Usually when an automated test suite is executed, there can be many failed cases. May be a few of them are genuine defects and rest are failed due to the issues with the automated test scripts. So it’s not always advisable to integrate the automation tools with the Defect Management tools (before your framework/script acquire a minimum level of maturity) and log a defect for each failed test case in your suite automatically. You may be exposing the issues with your test scripts unnecessarily. If you are experienced in automation, you must have experienced this situation, like failing test cases due to the issues in the scripts. Issues with object recognition, object synchronization mechanism are the most common reasons for automated test case failures. There are many other reasons as well, but these two are the most common reasons.

Imagine if your automation tool/framework logged a defect automatically after a test case is failed due to a poor object synchronization mechanism you have implemented. How will you justify that defect ? I’m not trying to demotivate people who are striving for excellent frameworks with best in class approaches for defect management as well as part of their automation solution. I’m just trying to shed some light on the practical issues I have experienced in the past.

What is a Test Automation Framework and how it’s going to help you ?…

Lets start with examples instead of explaining the theory. See this pseudo code  in an automated test case for an application for entering details in the ‘Enter Address’ screen

{

 AddressLine1.SetText("line1")

 AddressLine2.SetText("line2")

 City.SetText("city")

 Zip.SetText("00000")

 Country.SetText("country")

 SubmitButton.Click

}

Say you need to enter different addresses for different test cases and you have recorded/coded the script for each test case. So you will have to repeat these steps in each of your test cases. To reduce that complexity and effort, what we can do is, create a function and keep this code in that function and pass only the actual address details as parameters to that function for each of those test cases. This is a very simple step towards getting into a framework, called Modular Framework – converting your reusable lines codes into small functions/modules and call them whenever needed.

Function EnterAddress(var Line1,var Line2,var City,var Zip,var Country)
{

 AddressLine1.SetText(Line1)

 AddressLine2.SetText(Line2)

 City.SetText(City)

 Zip.SetText(Zip)

 Country.SetText(Country)

 SubmitButton.Click

}

'And call this function in you test cases with different values passed
for different test cases

Call EnterAddress ("lineOne","lineTwo","MyCity","99999","MyCountry")

Say, now you have been asked to report/log the steps while performing actions on each of the objects in your applications. In the same scenario, if we are implementing reporting/logging, lets see how big the script will become.

{

 AddressLine1.SetText("line1")

 Log ("line1 is entered in the Address Line1 textbox")

 AddressLine2.SetText("line2")

 Log ("line2 is entered in the Address Line2 textbox")

 City.SetText("city")

 Log ("city is entered in the City texbox")

 Zip.SetText("00000")

 Log ("00000 is entered in the Zip texbox")

 Country.SetText("country")

 Log ("country is entered in the Country textbox")

 SubmitButton.Click

 Log(Submit Button is clicked")

}

Think, if you didn’t have a Modular Framework, how big the test suite would become after this ? So with a Modular Framework, the script would look like this –

Function EnterAddress(var Line1,var Line2,var City,var Zip,var Country)
{

 AddressLine1.SetText(Line1)
   Log ( Line1 + " is entered in the Address Line1 textbox")
 AddressLine2.SetText(Line2)
   Log (Line2 + "is entered in the Address Line2 textbox")
 City.SetText(City)
   Log (City + "is entered in the City texbox")
 Zip.SetText(Zip)
   Log (Zip + "is entered in the Zip texbox")
 Country.SetText(Country)
   Log (Country + "is entered in the Country textbox")
 SubmitButton.Click
   Log (Submit Button is clicked")
}

'And call this function in your test cases with different values passed
for different test cases

Call EnterAddress ("lineOne","lineTwo","MyCity","99999","MyCountry")

So with a Modular Framework you could save a lot of time and effort on this. Still your coding effort is more to accomplish the Reporting requirement from your client.

Okay, the next requirement is to check whether the object (text box/button) is present before entering the value or clicking the button. So the code will look like this. (To reduce the complexity of reading, I took one text box for demonstrating the example)

{

If AddressLine1 is present then
    AddressLine1.SetText("line1")
    Log ("line1 is entered in the Address Line1 textbox")
Else
    Log ("Address Line1 textbox is not present and value is not entered")

.....

}

See, with this your test case/suite would become even bigger, if you have to implement this requirement without a framework. You have to write this code for each line of your script. I think, now you have got a fair idea why we need a framework. Lets continue…

Still, you have to repeat the same approach for all the reusable modules in your test suite. Lets go one step further, lets create functions/methods at the object level rather than the reusable functionality level. Like create a method to handle a TextBox, Button etc., to check the existence, entering the value, clicking the button, log the message etc.. ,

Function EnterValueInTextBox(obj TextBoxObject, var value)
{

If TextBoxObject is present then
    TextBoxObject.SetText(value)
    Log (value+ " is entered in the " + TextBoxObjectName +" textbox")
Else
    Log (TextBoxObject + " textbox is not present and " + value + " is not entered")
}

Function ClickButtonObject(obj ButtonObject)
{

If ButtonObjectis present then
    ButtonObject.Click
    Log (ButtonObject + " button is clicked")
Else
    Log (ButtonObject+ " button is not present and not clicked")

}

'And call these functions in your test cases with different values or even objects passed
for different test cases

Call EnterValueInTextBox (AddressLine1, line1)
Call EnterValueInTextBox (AddressLine2, line2)
.
.
.

ClickButtonObject(SubmitButton)

You can use ‘EnterValueInTextBox’ function to enter any value in any text box. Just call this function with the correct text box object and the value to be entered. Same is the case with ‘ClickButtonObject’ method as well.

Now you frame your test cases with this function calls in the order of the actions/steps to be performed using the name of the functions and the values passed, if required. Like  ‘LOGIN’, ‘GO_TO_ADDRESS_PAGE’, ‘ENTER_ADDRESS’ etc.

And that’s how you create a Keyword Driven Framework.

A framework with modules (methods/functions) created for each of the below features are called a Hybrid Framework.

– Object types (HTML or third party controls)

– Test case design via keywords and via scripting

– Simplified test data management

– Customized Reports (Test results)

– Recovery Mechanisms (Recover when the application crashes/test case fails unexpectedly)

– Data driven testing (Executing the same test scenario with multiple sets of test data)

 

 

To be continued…

It’s Automated… Not Automatic…

Many people believe that test automation is Automatic. Just do some magic, click a button and everything is done. I would like to tell those people that, Test automation is Automated it’s not Automatic….

Once the test cases are automated (either record and playback or scripted) , the test execution will be automatic. Yes, you have to automate it to be automatic…

It’s takes a lot of time, effort and patience to automate the test cases. And not all the test cases are automatable. There are a lot of scenarios in real world which cannot be automated.

So be cautious while giving commitments to your client what you are going to automate.

 

Don’t automate tests for the sake of it…

Yes, a lot of people still do it.

If you do so, a lot of money, valuable time, effort will be wasted without any purpose… Moreover you will be giving false commitments to the management/investors.

Please don’t do it…

Just because a test case is automatable doesn’t necessary mean it should be automated. If you are planning to automate a test case, just check how much value it’s going to add.

In most of the cases, there may be a lot of test cases in your test suite are easily automatable, but they are rarely used/low priority test cases. Spending valuable time and effort for automating those tests will not add any value.

Do a scientific evaluation while selecting your test cases for automation. Select the best suited test cases and automate them.

Always think about the cost of automation before jumping into it. Automating a test suite is costly, but you get the return if you run your suite regularly for a large number of cycles.