1. Basics of Finding Elements on a Page
Using Methods to Find Elements
When you start using Selenium in real projects, one of the first tasks will be finding elements on the page. You can compare this to looking for a hotel elevator button: you need to know where it’s located to press it and get to the right floor.
Selenium provides several methods to locate elements. Depending on your preference, you can use
find_element_by_id, find_element_by_name, find_element_by_xpath,
find_element_by_css_selector, and so on. Let's refresh what you already know and write a simple example:
from selenium import webdriver
# Creating a browser instance (using Chrome here)
driver = webdriver.Chrome()
# Opening the page
driver.get('http://example.com')
# Finding an element by ID
element_by_id = driver.find_element_by_id('main')
# Finding an element by name
element_by_name = driver.find_element_by_name('username')
# Finding an element by XPath
element_by_xpath = driver.find_element_by_xpath('//div[@class="content"]')
# Finding an element by CSS selector
element_by_css = driver.find_element_by_css_selector('div.content')
Differences Between find_element and find_elements
When you have multiple elements you want to capture (like all buttons with the btn class),
you'll need the find_elements method. The difference between find_element and
find_elements is that the former returns a single element, while the latter returns a list of elements.
# Finding a single element
single_element = driver.find_element_by_class_name('btn')
# Finding multiple elements
multiple_elements = driver.find_elements_by_class_name('btn')
# Looping through the found elements
for element in multiple_elements:
print(element.text)
As you can see, find_elements is perfect for working with collections of elements that might appear multiple times on the page.
Every find_element_xxx method has its counterpart find_elements_xxx, which returns a list of elements.
2. Interactive Interaction with Elements
Interacting with Buttons, Links, and Input Fields
Now that you’ve refreshed your knowledge of how to find elements, it’s time to remember how to interact with them! It’s kind of like playing with LEGO: you need to connect the pieces properly to build what you want.
Interactions with elements can be done using methods provided by Selenium. Let’s take another look at how this is done:
# Clicking a button
button = driver.find_element_by_id('submit')
button.click()
# Typing text into an input field
input_field = driver.find_element_by_name('q')
input_field.send_keys('Python selenium tutorial')
# Getting the text of an element
header = driver.find_element_by_tag_name('h1')
print(header.text)
Handling Events and Working with Dynamic Elements
When working with dynamic elements, like buttons that only appear after a specific action,
it’s important to account for delays and waits. Selenium provides ways to wait for such elements to load.
One of them is using the WebDriverWait class in combination with
expected_conditions.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
# Waiting for a button to appear for up to 10 seconds
button = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.ID, 'submit'))
)
button.click()
Note. To avoid typing expected_conditions every time, we aliased it as EC during import.
Using waits makes your script more reliable and prevents it from breaking due to timing delays or slow page loading. And in practice, that happens very often - almost always :)
3. What Can Go Wrong?
Everything. Everything can go wrong. You’re all set to interact with elements, and suddenly—bam—something goes wrong. Let's discuss some common errors you might face along the way.
First, NoSuchElementException — this is an error message you’ll see quite often, especially if the element you’re looking for
doesn’t exist on the page. It’s triggered when you try to find an element that’s not there. To prevent such situations,
you can use the try-except method to handle exceptions.
from selenium.common.exceptions import NoSuchElementException
try:
element = driver.find_element_by_id('non_existent_id')
except NoSuchElementException:
print("Oops, seems like the element wasn’t found!")
Issues with dynamic elements and loading times are also possible. In this case, wait for the elements to load
using WebDriverWait, as we discussed earlier.
GO TO FULL VERSION