'Scraping video url from multiple pages of a website

I am doing scraping using selenium. i want to scrape all the video urls that are present in the 626 products on the 25 pages. but while extracting the url it gives me the href link of the thumbnail of image file source.

import selenium
import pandas as pd 
from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException, ElementNotVisibleException,StaleElementReferenceException

#First we will connect to webdriver
driver=webdriver.Chrome(r'/Users/ankit/chromedriver')
#Open the webpage with webdriver
driver.get('https://www.getapp.com/hr-employee-management-software/human-resources/')

URL2 = []  # for product pages
URL = []  # for storing all the pages
for i in range(1, 27):
    URL.append(f"https://www.getapp.com/hr-employee-management-software/human-resources/page-{i}/")

# visiting all the pages and scraping the products/Read More About... Links
for p in URL:
    driver.get(p)
    for i in driver.find_elements_by_xpath(
        '//a[@data-testid="listing-item_text-link_read-more-about-product"]'
    ):
        URL2.append(i.get_attribute("href"))

# extracting and storing the video url of the product
video_url=[]

for i in URL2:
    driver.get(i)
    try:
        i=driver.find_element_by_xpath("/html/body/div[1]/div[2]/div[2]/section[1]/div/div[1]/div/div[1]/div[2]/div/div[2]/div/div/div[2]/img")
        video_url.append(i.get_attribute("src"))
    except NoSuchElementException:
        video_url.append('--')


Solution 1:[1]

Actually the image src attribute having the embed video name, using that unique name you can form the video url.

eg. src="https://i.ytimg.com/vi/YtktfxBFgJU/hqdefault.jpg"

Here the unique code YtktfxBFgJU can be added in suffix of a default url like below: https://www.youtube.com/embed/YtktfxBFgJU you will get video url.

Check this method may be helpful

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Vignesh