131

I have a problem; I am using the Selenium (Firefox) web driver to open a webpage, click a few links, etc., and then capture a screenshot.

My script runs fine from the CLI, but when run via a cron job it is not getting past the first find_element() test. I need to add some debug, or something to help me figure out why it is failing.

Basically, I have to click a 'log in' anchor before going to the login page. The construct of the element is:

<a class="lnk" rel="nofollow" href="/login.jsp?destination=/secure/Dash.jspa">log in</a>

I am using the find_element By LINK_TEXT method:

login = driver.find_element(By.LINK_TEXT, "log in").click()

A) How do I check that the link is actually being picked up by Python? Should I use try/catch block?

B) Is there a better/more reliable way to locate the DOM element than by LINK_TEXT? E.g., in jQuery, you can use a more specific selector, $('a.lnk:contains(log in)').do_something();


I have solved the main problem and it was just finger trouble. I was calling the script with incorrect parameters - a simple mistake.

I'd still like some pointers on how to check whether an element exists. Also, an example/explanation of implicit / explicit Waits instead of using a crappy time.sleep() call.

2
  • You can also find elements by CSS locator or xpath. Tends to be less brittle than by text contents.
    – Silas Ray
    Commented Mar 5, 2012 at 15:15
  • I believe that both - using content text and XPATH/CSS - are fragile to minor changes in design rather than application logic. A better way would be to find element by either id, class_name or name. Commented May 3, 2017 at 13:30

14 Answers 14

183

For a):

from selenium.common.exceptions import NoSuchElementException
def check_exists_by_xpath(xpath):
    try:
        webdriver.find_element_by_xpath(xpath)
    except NoSuchElementException:
        return False
    return True

For b): Moreover, you can take the XPath expression as a standard throughout all your scripts and create functions as above mentions for universal use.

I recommend to use CSS selectors. I recommend not to mix/use "by id", "by name", etc. and use one single approach instead.

6
  • You should also pass object: browser = webdriver.Firefox() to your custom function
    – user2558887
    Commented Jun 27, 2014 at 14:08
  • In general, I have rarely come across html pages which conform to the idea of uniform matching. Is there a way to work around mix-matches of the locator methods on such pages? Commented May 3, 2017 at 13:34
  • @KshitijSaraogi Not sure completely understand your question. If you are staying that it's impossible to have only 'css selectors' in your webdriver tests - it's actually quite possible. I've been doing just that for the last 6 years. Commented May 5, 2017 at 20:05
  • @AlexOkrushko What are your reasons for recommending css selectors over xpath?
    – BoZenKhaa
    Commented Mar 13, 2018 at 14:19
  • 1
    @BoZenKhaa, at least 2 reason: familiarity of css selectors to the devs, which contributes to the readability of the code; xpath had some terrible performance in IE. Commented Mar 16, 2018 at 10:20
125

You can grab a list of elements instead of a single element. An empty list in python is falsey. Example:

if driver.find_elements(By.CSS_SELECTOR, '#element'):
    print "Element exists!"

You can also use By.ID and By.NAME, but that just turns your id or name into a css selector anyway. Source

14
  • 2
    Is this faster than the above method of Exceptions handling?
    – Gal Bracha
    Commented Feb 17, 2017 at 10:41
  • 2
    @GalBracha Good question, I'm assuming it would probably depend on the version and platform, so if the speed difference is that important to you here, I would recommend testing it on your target platform for the best results. However, there is a point to be made here about pre-optimizations and the code readability might be more important than the negligable performance improvement in this case Commented Feb 17, 2017 at 14:00
  • 9
    The find-element (singular) form throws an exception when the element is not found. The find-elements (plural with 's') will return an empty list. In my humble opinion all the other examples that are using try-catch to see if an element exist are bad programming style. Try-catch should only be used to catch unexpected situations. If you think the element may not exist your code should be written to handle that situation. Commented Mar 8, 2021 at 14:37
  • 1
    @BillWorthington agree 100%. I'm not a Python dev so I don't know what's normal around here, but I wouldn't write things that way in any other languages Commented Mar 8, 2021 at 14:52
  • 1
    @BrianLeishman I was trying to point out your example is the RIGHT way to do it and the OTHER examples were not good programming style. Commented Mar 9, 2021 at 15:38
70

A) Yes. The easiest way to check if an element exists is to simply call find_element inside a try/catch.

B) Yes, I always try to identify elements without using their text for two reasons:

  1. the text is more likely to change and;
  2. if it is important to you, you won't be able to run your tests against localized builds.

The solution is either:

  1. You can use XPath to find a parent or ancestor element that has an ID or some other unique identifier and then find its child/descendant that matches or;
  2. you could request an ID or name or some other unique identifier for the link itself.

For the follow-up questions, using try/catch is how you can tell if an element exists or not and good examples of waits can be found here: http://seleniumhq.org/docs/04_webdriver_advanced.html

5
  • One use I have for finding text is to see whether a successful/unsuccessful flash message is displayed. Not sure if there's a better way to do it?
    – Rob Grant
    Commented Oct 6, 2015 at 14:34
  • Those are absolutely excellent reasons to avoid selecting by text
    – kevlarr
    Commented Jan 11, 2018 at 16:30
  • 1
    This was helpful, note that you can now use is_displayed(): selenium.dev/documentation/webdriver/elements/information
    – Raoul
    Commented Apr 27, 2022 at 9:10
  • The link is (effectively) broken. It redirects to a generic page. Commented Nov 11, 2022 at 7:02
  • @Raoul: Wouldn't that run into the problem of a thrown exception? Commented Nov 11, 2022 at 7:56
29

A solution without try&catch and without new imports:

if len(driver.find_elements_by_id('blah')) > 0: # Pay attention: find_element*s*
    driver.find_element_by_id('blah').click # Pay attention: find_element
7
  • 5
    You're finding it twice, it would certainly be more efficient to store the collection, and if it's truthy, click the first, e.g. e = driver.find_elements_by_id('blah') if e: e[0].click Commented Mar 21, 2019 at 20:22
  • 2
    @BrianLeishman I agree yours way it's more efficient, but I think mine more easy to read and understand Commented Mar 23, 2019 at 10:14
  • 3
    Good solution, but I agree with, Brian, regarding you're finding twice.
    – ivanleoncz
    Commented Jun 11, 2019 at 21:08
  • 3
    From a comment: "find_element_by_* and find_elements_by_* are removed in Selenium 4.3.0. Use find_element instead.". Commented Nov 11, 2022 at 7:24
  • 1
    And from Bill Worthington's comment: "The find-element (singular) form throws an exception when the element is not found." Commented Nov 11, 2022 at 7:49
7

The same as Brian, but add to this answer from tstempko:

I tried and it works quickly:

driver.implicitly_wait(0)

if driver.find_element_by_id("show_reflist"):
    driver.find_element_by_id("show_reflist").find_element_by_tag_name("img").click()

After this, I restore my default value:

driver.implicitly_wait(30)
4
  • yes! if you know the page is already loaded, you dont want to wait around for 30 seconds to get your "doesnt exist return". this returns immediately, just what I needed.
    – welch
    Commented May 11, 2018 at 6:32
  • 1
    Wouldn't that throw an exception if something doesn't exist (the whole point of this question)? And thus not really work? Commented Nov 10, 2022 at 4:09
  • OK, the OP has left the building: "Last seen more than 1 year ago". Perhaps somebody else can chime in? Commented Nov 11, 2022 at 7:19
  • From Bill Worthington's comment: "The find-element (singular) form throws an exception when the element is not found.". Will driver.implicitly_wait(0) prevent the exception from being thrown? Commented Nov 11, 2022 at 7:47
6

You could also do it more concisely using

driver.find_element_by_id("some_id").size != 0
3
  • 1
    This will return an error. Check Lopi Dani for the reason and the correct way to do it.
    – Sameh
    Commented Oct 4, 2020 at 12:53
  • 1
    Please note that size will return you the height and width of element.
    – Dev
    Commented Aug 30, 2021 at 16:37
  • Lopi Dani's answer. Commented Nov 10, 2022 at 4:12
2

When you know the element could not be there, the implicit wait could be a problem. I've created a simple context manager to avoid those waiting times

class PauseExplicitWait(object):
    """
    Example usage:

    with PauseExplicitWait(driver, 0):
        driver.find_element(By.ID, 'element-that-might-not-be-there')
    """
    def __init__(self, driver, new_wait=0):
        self.driver = driver
        self.original_wait = driver.timeouts.implicit_wait
        self.new_wait = new_wait
      
    def __enter__(self):
        self.driver.implicitly_wait(self.new_wait)
  
    def __exit__(self, exc_type, exc_value, exc_tb):
        self.driver.implicitly_wait(self.original_wait)
  
2

driver.find_element_by_id("some_id").size() is a class method.

We need:

driver.find_element_by_id("some_id").size which is a dictionary, so:

if driver.find_element_by_id("some_id").size['width'] != 0:
   print 'button exist'
3
  • this is not always right, the width can be zero if element exist also. el = driver.find_element_by_class_name('gNO89b') el.size this is a valid element in google search page but the width is 0
    – codeslord
    Commented Oct 6, 2020 at 16:23
  • I don't think this will work. find_element_by_id() probably throws an exception if the element does not exist (the whole point of this question). Commented Nov 11, 2022 at 7:23
  • From Bill Worthington's comment: "The find-element (singular) form throws an exception when the element is not found." Commented Nov 11, 2022 at 7:49
1

You can find elements by available methods and check response array length if the length of an array equal the 0 element not exist.

element_exist = False if len(driver.find_elements_by_css_selector('div.eiCW-')) > 0 else True
1
  • From a comment: "find_element_by_* and find_elements_by_* are removed in Selenium 4.3.0. Use find_element instead.". Though that would throw an exception if the element doesn't exist. Commented Nov 11, 2022 at 7:25
1

With the latest Selenium, you can now use you can now use .is_displayed():

https://www.selenium.dev/documentation/webdriver/elements/information/

4
  • 1
    This answer helped me. I have written same answer with example and part A) and B) to give some pointers for both part of the questions because Raoul's suggested edit queue is full.
    – TendaiM
    Commented Aug 10, 2022 at 7:58
  • Can you elaborate? What is the "latest Selenium"? What version? When was it released? What version did you test it with? On what system? What is the first version that introduced this feature? Please respond by editing (changing) your question/answer, not here in comments (*************** without *************** "Edit:", "Update:", or similar - the question/answer should appear as if it was written today). Commented Nov 11, 2022 at 1:40
  • Wouldn't that throw an exception if the element doesn't exist (before is_displayed() gets a chance to be called)? Commented Nov 11, 2022 at 7:30
  • From Bill Worthington's comment: "The find-element (singular) form throws an exception when the element is not found." Commented Nov 11, 2022 at 7:50
1

You could use is_displayed() like below:

res = driver.find_element_by_id("some_id").is_displayed()
assert res, 'element not displayed!'
2
  • 1
    Wouldn't that throw an exception if it doesn't exist (the whole point of this question)? Commented Nov 10, 2022 at 4:15
  • From Bill Worthington's comment: "The find-element (singular) form throws an exception when the element is not found." Commented Nov 11, 2022 at 7:50
0

You can check by find_elements. If the result is null, that element does not exist.

if driver.find_elements(By.SOMETHING, "#someselector") == []:
    continue   # That element does not exist
1
  • From a comment: "find_element_by_* and find_elements_by_* are removed in Selenium 4.3.0. Use find_element instead.". Though that would throw an exception if the element doesn't exist. Commented Nov 11, 2022 at 7:27
0

A) Use .is_displayed() as explained at Information about web elements:

# After navigating to the URL,
# get the Boolean value for if this element is displayed
is_button_visible = driver.find_element(By.CSS_SELECTOR, "[name='login']").is_displayed()

Continue your Visual Studio Code logic using "is_button_visible" for branching.

B) Refer to Sam Woods's answer

2
  • 1
    Wouldn't that throw an exception if the element doesn't exist (before is_displayed() gets a chance to be called) - the whole point of this question? Commented Nov 11, 2022 at 7:31
  • From Bill Worthington's comment: "The find-element (singular) form throws an exception when the element is not found." Commented Nov 11, 2022 at 7:51
-1
el = WebDriverWait(driver, timeout=3).until(lambda d: d.find_element(By.TAG_NAME,"p"))

doc

1
  • 3
    Please provide a short explanation with answers to describe how and why it works. Sharing a link to the documentation is good, but the answer should be able to stand on its own.
    – wihlke
    Commented Nov 19, 2022 at 11:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.