1. Logging
Today we’re diving into a topic that might save you a ton of nerves and precious time—logging and error handling. Even the most skilled developers aren’t immune to errors, and your job is to make those errors predictable and manageable. Imagine you’re an archaeologist on a dig, and your task is to protect your expedition from unexpected collapses and tricky traps—logging and error handling are your reliable compass and shield in this risky business!
When we talk about logging in programming, we mean the process of recording information about how the program runs. These are processes happening inside your code that might not be obvious at first glance. By writing logs, you leave yourself breadcrumbs, like Hansel and Gretel leaving breadcrumbs in the woods to find their way back home.
Setting Up and Configuring Logging
Let’s start with the basic tools in Python. For logging, we’ll use the built-in logging module. It’s your Swiss Army knife for recording information about the program’s execution.
import logging
# Setting up logging
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
# Example usage
logging.info("Script has started")
logging.error("This is an error")
What are the different logging levels?
- DEBUG: Most detailed info, usually helpful only when diagnosing problems.
- INFO: Confirms that things are working as expected.
- WARNING: Something unexpected happened that might cause issues, but the program still runs.
- ERROR: Serious issues that have caused a failure in a function.
You can set the logging level to filter info. For example, if you only want to see errors and warnings, set the level to logging.WARNING.
Example of Logging in Selenium
Let’s see how logging can be used in a Selenium project:
from selenium import webdriver
# Setting up logging
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
try:
logging.info("Initializing Chrome driver")
driver = webdriver.Chrome()
logging.info("Opening Python.org page")
driver.get("https://www.python.org")
logging.info("Script completed successfully")
except Exception as e:
logging.error(f"An error occurred: {e}")
finally:
driver.quit()
This example demonstrates how to integrate logging into your Selenium script to track its execution. We use try, except, and finally for error handling—more on that below.
2. Error Handling in Selenium
In programming, just like in life, errors are inevitable. However, we can control how we react to them. Using try, except, else, and finally allows you to gracefully catch and handle errors.
Reviewing Exception Handling
Let’s take a look at how exception handling works:
try:
# Code that might raise an exception
result = 10 / 0
except ZeroDivisionError as e:
# Code that will run if a ZeroDivisionError exception is raised
logging.error("Division by zero is not possible!")
else:
# Code that will run if no exception was raised
logging.info("Operation completed successfully")
finally:
# Code that will always run
logging.info("Operations finished")
Your code in the try block will be executed. If an error like ZeroDivisionError occurs, the except block will execute, and the program won’t crash. The else block is used to execute code if no exception was raised. Finally, the finally block will always execute regardless of whether there was an error.
Error Handling in Selenium
Selenium provides a set of built-in exceptions for handling different situations. Here are some useful ones and how to use them.
-
NoSuchElementException—happens if the element isn’t found. Use it to check for the availability of elements.Pythontry: element = driver.find_element(By.ID, "element_id") except NoSuchElementException: logging.warning("Element with ID 'element_id' not found.") -
TimeoutException—happens if an expected element doesn’t show up on the page within the set time. Use it to manage waits.Pythonfrom selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC try: element = WebDriverWait(driver, 10).until( EC.presence_of_element_located((By.ID, "element_id")) ) except TimeoutException: logging.error("Waiting for element with ID 'element_id' failed.") -
ElementClickInterceptedException—happens if an element is blocked by another element, and clicking isn’t possible. Useful for handling popup-blocked elements.Pythonfrom selenium.common.exceptions import ElementClickInterceptedException try: driver.find_element(By.ID, "button_id").click() except ElementClickInterceptedException: logging.warning("Element is blocked by another element and cannot be clicked.")
Retries and Recovery Logic
To make your script more resilient, you can implement a retry mechanism when errors occur. For example, if a site is temporarily unavailable or an element isn’t found, the script can try the action several times before failing.
import time
def retry_find_element(driver, by, value, retries=3):
attempt = 0
while attempt < retries:
try:
element = driver.find_element(by, value)
logging.info(f"Element found: {value}")
return element
except NoSuchElementException:
attempt += 1
logging.warning(f"Attempt {attempt} failed, retrying in 2 seconds...")
time.sleep(2)
logging.error(f"Element not found after {retries} attempts.")
return None
Tips for Script Resilience and Reliability
- Use explicit waits: Set up explicit waits for elements that don’t appear immediately. This reduces errors due to loading delays.
- Implement try-except blocks at every stage: This helps log errors and avoids breaking the script over minor issues.
- Retry critical steps: If an error is likely to be temporary, set up retries with a few seconds’ interval.
- Log every step: Logging each key phase helps analyze the script’s execution and catch subtle bugs.
- Set up notifications for critical errors: For example, send an email or chat message if a CRITICAL level error is logged.
3. Logging That Works for You
Now that we’ve learned how to log and handle errors, let’s take it one step further to see how to make the most of your logs.
Choosing Log Location and Format
Use a file to store logs if you want to keep them for analysis. It’s like keeping a journal you can revisit to analyze and avoid future mistakes. Let’s configure logging to write to a file:
logging.basicConfig(filename='app.log', filemode='w', level=logging.INFO)
# Now all logs will be saved to app.log
logging.info("Started writing logs to file.")
Log Analysis
Analyze your logs to find recurring errors or performance problems. Maybe your script only runs into trouble on specific days or times, and you’d only notice this through careful log analysis. This analysis is your personal detective work that helps you understand how your application behaves.
GO TO FULL VERSION