Detect all names and get their link with Selenium Python

Question

I want to make a search system when we enter a word in a variable, it search between all links’ names of this page (all the games) a little like a « control-F » and display the results (names + links) using Selenium (Python).

I don’t know how to make a system like that! If you can help it’s good!

Have a Nice code!

Answer 1

You are attempting to locate specific elements on a page and then sorting through them for a key search term. Selenium can identify elements on a page through a number of methods, see here for a guide. Once you have located all the elements you can filter them for the search term of interest.

Finding ALL the elements of interest:

I would utilise the XPATH of your elements to find them on the page and make a list that you can then search through based on your keyword. In your case all they are identifiable by this xpath:

//div[@class="blog-content"]//a

Extract the required information:

Once you have the list of elements, you will need to iterate over them to extract the href tag (the game's url) and innerHTML text (the name of the game).

I have used list comprehension in the example below to do this, which creates a dictionary {url:name, ...} you can filter your specific items from.

Example Code:

from selenium import webdriver  
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by      import By
from webdriver_manager.firefox import GeckoDriverManager

website_url = 'https://steamunlocked.net/all-games-2/'
game_xpaths = '//div[@class="blog-content"]//a'

driver = webdriver.Firefox(service=Service(GeckoDriverManager().install()))
driver.get(website_url)

game_elements = driver.find_elements(By.XPATH, game_xpaths)
games = {g.get_attribute('href'):g.get_attribute('innerHTML') for g in game_elements}
games

"""
Outputs:
{'https://steamunlocked.net/red-tether-free-download/': '—Red—Tether–> Free Download (v1.006)',
 'https://steamunlocked.net/hack-g-u-last-recode-free-download/': '.hack//G.U. Last Recode Free Download (v1.01)',
 'https://steamunlocked.net/n-verlore-verstand-free-download/': '‘n Verlore Verstand Free Download',
 'https://steamunlocked.net/0-n-0-w-free-download/': '0°N 0°W Free Download',
 'https://steamunlocked.net/007-legends-free-download/': '007 Legends Free Download', ...
"""

Finding SPECIFIC items (i.e. CTRL+F)

To identify and filter only specific items from your dictionary for the occurrence of the word/string you are interested in.

def search(myDict, search_term):
    return [[v,k] for k,v in myDict.items() if search_term.lower() in v.lower()]

>>> search(games, 'Ninja')
[['10 Second Ninja Free Download','https://steamunlocked.net/10-second-ninja-free-download/'],
 ['10 Second Ninja X Free Download','https://steamunlocked.net/10-second-ninja-x-free-download/']]