1

I've gone through various tutorials and Stack Overflow posts and understand that Selenium can output XML test results in a way that Hudson can read/report them in HTML format.

What I don't understand is the syntax to use in Python, to get the results to look something like: Testcase_LoginPage.VerifyButton1Present fail

Testcase_LoginPage.VerifyButton2Present pass

Currently, when I drill down the results in Hudson, they won't be formatted in a useful way as I described above, and also it will report that it only ran ONE test, even though it ran multiple assert Tests:

Traceback (most recent call last): File "D:\Temp\1TestingApps\Selenium\Scripts\SampleScripts\SamCodeSample\test\SOreports.py", line 22, in tearDown self.assertEqual([], self.verificationErrors) AssertionError: Lists differ: [] != ['Sign Up button issue2']

Second list contains 1 additional elements. First extra element 0: Sign Up button issue2

  • []
  • ['Sign Up button issue2']

Ran 1 test in 13.610s

FAILED (errors=1)

Generating XML reports...

Code is below. Thanks in advance for the help!

from selenium import selenium import unittest, xmlrunner, os, re

class Demo(unittest.TestCase):

def setUp(self):
    self.verificationErrors = []
    self.selenium = selenium("localhost", 4444, "*chrome", "https://workflowy.com/")
    self.selenium.start()

def test_hh(self):
    sel = self.selenium
    sel.open("/accounts/register/")
    try: self.assertEqual("Sign Up FAIL", "Sign Up FAIL","Sign Up button issue1")
    except AssertionError, e: self.verificationErrors.append(str(e))
    try: self.assertEqual("Sign Up FAIL", "Sign Up FAIL1","Sign Up button issue2")
    except AssertionError, e: self.verificationErrors.append(str(e))

def tearDown(self):
    self.selenium.stop()
    self.assertEqual([], self.verificationErrors)
if __name__ == "__main__":
#have to format the code this way as SO is complaining about 'bad indent'
    unittest.main(testRunner=xmlrunner.XMLTestRunner(output='test-reports'))

2 Answers 2

2

You have only one test defined, so it can only report one test. One test is a test method, not an assert statement. You can have several asserts in one single test, as you may need to assert several results in order to confirm a successful test result.

So first step to your desired output would be to put your second assert into a second test method, then you should see two test results.

2
  • Hi, thanks, but I can't see how that would work in cases like: 1) If you wanted to check a sign-in page before and after the sign-in action. By placing the tests outside the sign-in method, the tests would either only execute before or after the sign-in 2) a Data Driven test where it needs to run a test on that page in loop and then do a verify after each run. Unittest just runs the tests in order it sees them, and then it's done, right?
    – HRVHackers
    Oct 5, 2011 at 17:24
  • Don't get me wrong, you can have several asserts in one test. But that will still be one test. In Unittest terms, one Test is one Test method. This test can and will contain several assertion checks, but there will only be one result of this test. To get to your cases: 1) You have two tests here, the first test will assert that the sign-in page looks right before sign in, the second test will perform a sign in, and then check that the page looks right after sign-in. 2) Here you can split into tests that work with the same type of data, i.e. tests with correct data, with corrupt data etc.
    – pushy
    Oct 6, 2011 at 6:26
0

I've finally figured out how to make the verifications and assertions get reported in a useful format for my needs. The problem is that the default structure of the tests when simply exporting a Selenium IDE recorded script into a Python RC file lacks a lot of detail I needed.

What I changed: - Placed the Selenium start and stop methods in the Setup and tearDown classes which prevented Selenium from restarting the browser with each newly defined verification/assertion method
- Added error descriptions that include the testcase name via inspect.stack()


import inspect, unittest, xmlrunner
from selenium import selenium

class TESTVerifications(unittest.TestCase):
@classmethod
def setUpClass(self):
    self.selenium = selenium("localhost", 4444, "*iexplore", "https://workflowy.com/")
    self.selenium.start() 
    self.selenium.set_timeout("60000")
    print("setUpClass")      
    self.selenium.window_maximize()
    self.selenium.open("/")


def setUp(self):
    self.verificationErrors = []

def test_verification1_error(self):
    try: self.assertEqual("This application is designed", "This application is designedZZZZ",(inspect.stack()[0][3]) +" text missing 'This application is designed'")
    except AssertionError, e: self.verificationErrors.append(str(e))
def test_verification2_error_two_times(self): 
    sel = self.selenium
    ##No such element exception
    try: self.assertEqual("First failure", "First failureZZZZ",(inspect.stack()[0][3]) +" First failure'")
    except AssertionError, e: self.verificationErrors.append(str(e))
    try: self.assertEqual("Second Failure", "Second FailureZZZZ",(inspect.stack()[0][3]) +" Second failure'")
    except AssertionError, e: self.verificationErrors.append(str(e))

def tearDown(self):
    #self.selenium.stop()
    self.assertEqual([], self.verificationErrors,"Results: " + str(self.verificationErrors))
@classmethod    
def tearDownClass(self):

    self.selenium.stop()
    print("tearDownClass")

if __name__ == "__main__":
#    unittest.main()
unittest.main(testRunner=xmlrunner.XMLTestRunner(output='test-reports'))

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.