In my last tutorial we tested Test Android apk file with Robotium. Simple Android Calculator’s apk was used with some specific input values. Now in this tutorial we are extending testing to Data Driven Testing approach under Robotium. Many automate testers may have worked on Data Driven Testing Framework for web sites or Desktop applications.
There are many Data Driven Frameworks available, but to save time I have tried to develop me own data driven framework. It may not mature enough but we can make it more intelligent by extending it.
1. Data Driven Testing Architecture
I am going to define my test scenarios into a csv (comma separated version) file and my script file will
* Read input from TestData.csv file
* Enter each ‘First Value’ column data in first edit field
* Enter each ‘Second Value’ column data in second edit field
* Click on Multiply field each time
* Verify Resultant value
To create a data driven development we will go through following steps
* Create Test Data
* Push Test Data into Emulator
* Import jar file
* Create Test Script (read data and assign values to AUT(application under test)
* Execute Test scenarios
* View Test Results
We will apply Data Driven testing approach on Android Simple Calculator. In which we enter two values and on clicking Multiply button it will show their resultant value. We will use Test Android apk file with Robotium project for Data Driven Testing implementation.
2. Create Test Data
We can create any combination of First and Second Value to test our Simple Calculator on Android. First Value represents the values for First Editfield and Second Value represents the Second Editfield in Android Simple Calculator. Execute column will allow the script to execute specific scenarios only.
3. Push Test Data into Emulator
Robotium will run our test script into emulator, for proper implementation of Data Driven Testing we need to make Test Data file accessible for emulator.
Load emulator, from eclipse Load DDMS interface. From Project Explorer tab navigate to the AUT package (our Android Simple Calculators package is com.calculator) and expand it to files category. It will be empty, we will push our Test Data file into files directory and then our script will be able to read TestData contents with emulator.
From top right section click on push icon and browse the TestData file from your computer.
It will show the pushed test data file into files directory like below
4. Import JXL jar file
Our TestData file is in CSV format so we need to import necessary jar file to read contents, there may be multiple API available on Internet, I will be using JXL, it is available on following link.
After successfully download import that jxl.jar file into project build path.
5. Create Test Script
From Test Android apk file with Robotium project open the TestApk.java file and replace the existing code with one (same file name) in shared project code.
6. Run Test Script
Now its time to run our test script, From eclipse right click on project, select Run As and then click on Android JUnit Test. Wait for some time it will show the progress in console bar like below
System will load Android Simple Calculator app into emulator and start to read TestData file and enter First and Second Values into calculator and asserts resultant values. I have put (System.out.println) code to print results, you need to load LogCat interface and results will be visible there in logCat window like below,
After completing execution eclipse will show successful script execution like below.
7. View Results
After successful execution test script will create and write results into txt file and that file will be visible into Project Explorer of DDMS.
Click on pull icon top right to put that file and view it. Results of all test scenarios will be visible into TestResultData file.
Note: This is the basic Data Driven Testing framework around Robotium. There are many things which can be implemented in better way. I have tried to develop it in simplest way. Its up to the test engineer to use it in his/her own way. We can make it as much intelligent as we need.
Download sample project code with TestData file
Can u please Explain the 3. Push Test Data into Emulator portion more explicitly. I am getting the com.Calculator in the Devices section but I am not getting the package com.Calculator in File Explorer.
In the file explorer I am able to see only
1.data
2. mnt
3. System
please help me out.
Hi,
From file explorer expand the "data" directory there will be another "data" directory, expand it also, and you must get your desired package name, expand that package name, you will see the hierarchy like in above tutorial screen shot.
let me know if you still facing any issue.
Hi Naveed,
Thanks for your post related Data Driven Testing. It was excellent information. I was looking forum past two months on daily, I felt very happy by seeing your post and it was very helpful. Kindly share your ideas to world…I hope many people are watching this blog…..
Hi,
Thanks for the excellent code, its working fine in emulator. But i want to run it in a device. I have problem in copying the .csv file, where exactly in device should i copy it?
I am not able to access the "data " folder for device through DDMS so can you please tell me where should i copy it….
DDMS should show that data folder, I need to see why it is not working in your case. Write more details, then I could better talk on it.
Hi Naveed,
data input part is working fine for me but still am not getting any output file. Please let me know where do I get the result file.
Thanks in Advance….
thanks a lot
Hi Naveed,
Is there no way to not having to push the data filt to the emulator or the device? Can we not have the file stored located within the project on the machine that is running the tests?
Thanks
Manoj
Manoj,
During working on this project, I searched some way out for it through code, but could not at that time.
There should be some way, will look into it, and let you know.
Hi,
I couldn't see files folder in my application or even for calculator application as well. I am seeing only lib folder under /data/data/com.calculator.
Is there any way to create it?
Hi Naveed,
What if a test fails in the middle of a iteration?? will it Run the next iteration fromthe start???
Boopathi,
You can put your file at root of /data/data/com.calculator. It should also work in that case. let me know if still not working.
RobotiumTester,
If there is only one test in your TestSuit and that fails it will start in next iteration from start.
In other case you can make multiple test cases.
Hi Naveed,
is this procedure works on real device and not on emulator?
Thanks in advance.
Hi,
That should work on both Device & Emulator.
Regards,
hi, i don't know java much, can any one help me to understand the Test java script file..and if want to modify the testresult.txt file by showing the complete result how can we modify the java script file. like if we wants to add more test cases how can we do it???
please help me out..
Hi,
Thanks for writing an excellent starter tutorial. However I the project code is no longer available on the link http://robotium.googlecode.com/files/AndroidCalculatorTestApk.zip. Could you please provide an alternative location?
Thanks,
Amar.
Check this,
http://robotium.googlecode.com/files/AndroidCalculator.apk
Hi NS
Is this AndroidCalculator.apk has ur code for data driven test. I m trying to implement data driven test for my test automation. the above links at point 4. are not working. could you please send me the code how to implement the data driven test for andriod application.
what issue you are facing at point 4 ? I just saw source code link is broken :(, Let me sort it out, I'll update links
Thanks,
Hi,
All broken link including TestProject's source Code are update, all links are working now.
Please let me know if there still something missing.
How to run every test as an individual test as we do in testNG by inserting @dataprovider and @test(provider="")
By objective to measure the time taken to perform one test in data driven framework …
Please help me out..
I would be very thankful
Hi Naved,
I am facing a few issues:
1. You have a Utility.java class in your code. You created an object myUtility for this class. But this object is not being used anywhere. So how will the writeResults() method execute when I will run the TestAPK class, as the writeResults method is in the Utility class?
2. I modified the code a bit at places like file location, input fields etc (small changes that are necessary suiting my application). But still the code is returning NullPointer Exeception. Dont know why.
3. I placed my testdata.csv file under the AUT folder. What will be the exact path for the file? (something like /sdcard/mnt/data/data/com.application/TestData.csv? or something else?)
4. Should I place my testresults.txt file at the same location or it will be created at its own?
Can you please help? I am stuck in this framework thing for a long time now.
Thanks
Varun
@Meenu Bansal sorry for late reply, you can do this using JUnit by 'test' annotation in 'TestCase' architecture
@Varun
Here are your answers,
1. Please see the code in more details
2. If you did everything right, it should work, can you send me the code with new changes?
3. Yap it is the path, but it may vary depending upon the AndroidOS implementation
4. It will be purely up to you 🙂
Thanks for your prompt response Naved.
I would need your email-id for sending you the code.
luckynaveed@gmail.com
I hope you got my mail Naved.
Naved
On your application, the code worked fine. Two issues I am facing now:
1. When I change the runmode from Yes to No, a failure comes. That is, on runmode No, some problem is coming. Try it once.
2. Data is not writing to the TestResult.txt file.
It is working that's good.
1. What you exactly did on it, can you come up with more details on it ?
2. I was working fine, not sure what you did 🙂
1. I just changed the "Execute" field from yes to No for one of the rows.
2. I did no change to the file.
What about the code i sent u? U ve seen it?
Varun, Its been long time, I've written this code, let me see it this weekend, I'll see it for you.
You can ping me during weekends if see me online, to talk on it.
Thanks Naved. Please see to the code I mailed you. If some itsy bitsy changes can make it read/write excel, I l be very thankful to you.
Hi Naveed,
I am getting the following stacktrace while running the test.
When using the Java Excel API: java.lang.NoClassDefFoundError: jxl.Workbook
When using the POI API : java.lang.NoClassDefFoundError: org.apache.poi.xssf.usermodel.XSSFWorkbook
Can you please help me in getting the problem.
Thanks
Anonymous
Hi Naveed,
In this tutorial you have suggested to store the .csv file in the the device under test, but if I want to save the file in my system only, then in which location I can store?
I tried storing my .xls file in the "src" folder, inside the package where my test class is located, but no success.
All time is says
"java.io.FileNotFoundException: //.xls (No such file or directory)"
Can you please help me with this.
Thanks
Mohit
Mohit,
When we run testProject, it will be installed on device and run through device.
In the start I tried to do the same but could not find any way through code to access PC file system through Android Device, there may be the way to do it,
the way to do this is
1. run test project and save results.csv on device file system
2. call script below after test execution
3. write some script(ssh, shell or something) to access file on device using adb and save it on your file system
Thanks Naveed for reply, I will try this and if I found any success I will definitely share that.
Thanks
Mohit
Hi Naveed,
I am new to Robotium. Tried your code but unable to run successfully due to some error and i.e.
java.lang.NullPointerException
at com.calculator.TestDataMain.getTestData(TestDataMain.java:106)
at com.calculator.TestDataMain.setUp(TestDataMain.java:67)
at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169)
at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:154)
at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:520)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1447)
Thanks,
Priya
It means code is not able to find TestData file, you changed something in code?
Can you come up with more details?
Hi naveed,
I didn't miss any code. Still want to send you the code. Let me know your mailid so that I will send you the code.
Thank,
Priya
you can attach here as well, anyways expertsguys@gmail.com
Hi Priya,
I have came across same situation. I saved my excel file with ".xls" extension instead of ".xlsx" as jxl.jar does recognize this extension. You can try this, hope it will resolve your issue.
Thanks
Ashfaq
Hi Asfaq,
Thanks for your info.I will try as per your suggestion.
Thanks,
Priya
Hi Naveed,
Now It is working fine. As you mentioned it was due to file change.
But I am not getting any output file in DDMS. Please let me know how I will get the result file.
Thanks,
Priya
Priya, there is nothing to do anything about log, if everything is working fine, it will print that as log.
I am un able push the file on to the device….
get error like "[2013-05-03 11:50:12 – ddms] null
java.lang.NullPointerException
at com.android.ddmlib.Client.read(Client.java:661)
at com.android.ddmlib.MonitorThread.processClientActivity(MonitorThread.java:311)
at com.android.ddmlib.MonitorThread.run(MonitorThread.java:263)
[2013-05-03 11:50:12 – ddms] null
java.lang.NullPointerException
at com.android.ddmlib.Client.read(Client.java:661)
at com.android.ddmlib.MonitorThread.processClientActivity(MonitorThread.java:311)
at com.android.ddmlib.MonitorThread.run(MonitorThread.java:263)
[2013-05-03 15:35:55 – ddms] transfer error: Permission denied
[2013-05-03 15:35:55] Failed to push selection: Permission denied
[2013-05-03 16:27:49] Failed to push the item(s).
[2013-05-03 16:27:49] null
[2013-05-03 17:03:16 – ddms] transfer error: Permission denied
[2013-05-03 17:03:16] Failed to push selection: Permission denied
"
Seems like it devices is not permitting. Which device you are using?
Hi,
I am able to run test on the device,just i am trying with data driven testing by adding the xls sheet,so i am getting the above error…device is Samsung grand..
Thanks
Sana
Hi all,
I am able to run test on the device,just i am trying with data driven testing by adding the xls sheet,so i am getting the below error…device is Samsung grand….
I am un able push the file on to the device….
get error like "[2013-05-03 11:50:12 – ddms] null
java.lang.NullPointerException
at com.android.ddmlib.Client.read(Client.java:661)
at com.android.ddmlib.MonitorThread.processClientActivity(MonitorThread.java:311)
at com.android.ddmlib.MonitorThread.run(MonitorThread.java:263)
[2013-05-03 11:50:12 – ddms] null
java.lang.NullPointerException
at com.android.ddmlib.Client.read(Client.java:661)
at com.android.ddmlib.MonitorThread.processClientActivity(MonitorThread.java:311)
at com.android.ddmlib.MonitorThread.run(MonitorThread.java:263)
[2013-05-03 15:35:55 – ddms] transfer error: Permission denied
[2013-05-03 15:35:55] Failed to push selection: Permission denied
[2013-05-03 16:27:49] Failed to push the item(s).
[2013-05-03 16:27:49] null
[2013-05-03 17:03:16 – ddms] transfer error: Permission denied
[2013-05-03 17:03:16] Failed to push selection: Permission denied
"
Thanks
Sana
Are you able to push file through command like
push
It should work
Hi,
Can u pls tell hw to push through cmd….
Thanks,
sana
http://developer.android.com/tools/help/adb.html
-Naveed
Hello all,
Pls any body say hw to do data driven testing on device itself….
Thanks,
bhanupriya
Hi All,
Can anyone post data driven testing on device ……..
Thank you,
bhanu sana
Yap, I've done. what is issue in your case ?
Hi Naveed,
Can please resolve the following issue on "Test run failed: Instrumentation run failed due to 'java.lang.VerifyError'" .
I got it while running the script with testdata.csv
Thanu bhanu priya
interesting, i've never seen this error, can you please send me complete trace and also let me know if you have made some changes in code.
-Naveed
Hi Naveed,
Below is code for email apk on device,Micromax A27 android 4.0.4,
public void testRecorded() throws Exception {
try {
solo.waitForActivity("Welcome");
solo.waitForActivity("AccountSetupBasics");
solo.sleep(1000);
solo.clearEditText(0);
solo.enterText(0, "automation3.qa@gmail.com");
solo.clearEditText(1);
solo.enterText(1, "cloud@123");
solo.clickOnButton("Next");
//Options page
solo.waitForActivity("AccountSetupOptions");
solo.sleep(3000);
assertTrue("Wait for button (text: Next) failed.",
solo.waitForText("Next", 1, 5000));
solo.clickOnButton("Next");
//CLick on next after entering the AccountSetupNames
solo.waitForActivity("AccountSetupNames");
solo.clearEditText(1);
solo.enterText(1, "Bhanu");
solo.sleep(3000);
solo.clickOnButton("Next");
//Click on compose option of main page
solo.waitForActivity("AccountSetupBasics");
solo.waitForActivity("Welcome");
solo.waitForActivity("EmailActivity");
solo.sleep(5000);
for(int i =1; i<rowCount; i++)
{
//solo.clickOnMenuItem("MessageCompose");
solo.clickOnScreen(solo.toScreenX(0.052f), solo.toScreenY(0.998f));
solo.waitForActivity("MessageCompose");
//Enter To Information
solo.clearEditText(0);
solo.enterText(0, vToAddress[i].toString());
solo.clearEditText(1);
solo.enterText(1, vSubject[i].toString());
solo.clearEditText(2);
solo.enterText(2, vBody[i].toString());
//Click on Send button
//solo.clickOnMenuItem("EmailActivity");
solo.sleep(3000);
solo.clickOnScreen(solo.toScreenX(0.929f), solo.toScreenY(0.082f));
solo.waitForActivity("EmailActivity");
}
} catch (AssertionFailedError e) {
solo.fail(
"com.attachments.android.test.Test.testRecorded_scr_fail",
e);
throw e;
} catch (Exception e) {
solo.fail(
"com.attachments.android.test.Test.testRecorded_scr_fail",
e);
throw e;
}
}
Thank u,
bhanu priya
Can you please send logcat, and exception details from console?
-Naveed
Hi naveed,
Below is the console
[2013-05-08 18:02:30 – MyAndroidAppTest] ——————————
[2013-05-08 18:02:30 – MyAndroidAppTest] Android Launch!
[2013-05-08 18:02:30 – MyAndroidAppTest] adb is running normally.
[2013-05-08 18:02:30 – MyAndroidAppTest] Performing android.test.InstrumentationTestRunner JUnit launch
[2013-05-08 18:02:30 – MyAndroidAppTest] Automatic Target Mode: Unable to detect device compatibility. Please select a target device.
[2013-05-08 18:02:35 – MyAndroidAppTest] Application already deployed. No need to reinstall.
[2013-05-08 18:02:35 – MyAndroidAppTest] Project dependency found, installing: MyAndroidApp
[2013-05-08 18:02:35 – MyAndroidApp] Application already deployed. No need to reinstall.
[2013-05-08 18:02:35 – MyAndroidAppTest] Launching instrumentation android.test.InstrumentationTestRunner on 0123456789ABCDEF
[2013-05-08 18:02:36 – MyAndroidAppTest] Test run failed: Instrumentation run failed due to 'java.lang.VerifyError'
thank u…
1. please go into JUnit tab and send me complete exception stack, its not showing full info
2. also send me logcat
Hi Naveed,
Actually I am using Android sdk inbuild eclipse and I am not geting Junit exception since run is not complete….I am geting only console display as I posted previously…
you can add JUnit in eclipse by tapping "+" icon from bottom left(in my case)
Hi,
I am having JUnit and i running in JUnit but Intrumentation itself i getting error so could get exception only console is dispalyed…
"MyAndroidAppTest] Test run failed: Instrumentation run failed due to 'java.lang.VerifyError'"
can you please send me your project? I'll better help me to come up with some solution.
Please zip your complete project and email me.
luckynaveed@gmail.com
Hi Naveed I followed all the steps of yours.
And testdata.csv I have copied into my device directly by copy and paste.
Finally I am getting below errors:
java.lang.NullPointerException
at TestAndroidCalculator.getTestData(TestAndroidCalculator.java:94)
at TestAndroidCalculator.setUp(TestAndroidCalculator.java:56)
at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:192)
at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:177)
at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:555)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1608)
Hi, I am running this code on the device and i get null pointer exception when trying to read the input file, did anyone resolve this issue. Please let know. that would be a huge help.