Category "web-scraping"

How to wait for translated text to appear in DeepL.com

After entering original text in DeepL, it takes 1-2 sec for translated text to appear. I'm unable to get translated text with TimeoutException. I've tried to Sy

Getting the same HREF link instead of different HREF links with Selenium

This is the first time I'm making a website scraper and I'm relatively new to programming in general. So I'm trying to get the HREF links for all the subpages o

Price won't show up in the html parsing

I'm trying to get the price of this object into a variable and print it out/put it in a CSV. This is the section of html that I am trying to parse: <span cla

Scraping the rating of some reviews as pictures

I am trying to scrape the rating of some movie reviews but the rating is not a number, but it is one of 10 different images that range from showing empty stars

(Python) Fill the column by webscraping the data on the website. Getting an error: UnicodeError: label empty or too long

I have a dataset that looks like this: ID Link 1 'https://wwwexamplecom/hello/details-5565558html' 2 'https://wwwexamplecom/hello/details-5489292html' 3 'https

Webscraping sale prices from a grocery store- Am I on the right track or is there a simpler way?

I am new to all of this, and this is my first real coding project so forgive me if the answer is obvious :) I am trying to extract sale items from [my grocery s

How do I continue a for loop when no data is present which causes an index error?

I am learning python and I'm guessing something very obvious is at fault here. I have a web scraping program that, I suspect, upon finding an empty data field t

Trouble scrapping sports table - python

I'm having a lot of trouble scrapping results off a sporting table in Python. I am new to scrapping and have tried everything I can find online. The website is

How to get table and it's element with Python/Selenium

I'm trying to get all the price in the table at this URL: https://www.skyscanner.it/trasporti/voli/bud/rome/?adults=1&adultsv2=1&cabinclass=economy&

if class not found return none in beatifulSoup

I'm trying to get None if the class is not found in web scraping. For example, in some cases stage-codes.html#10_99 doesn't exist in HTML. for st in soup.find_a

Web scraping from html code of a database using python

I am new to python and am learning things slowly. I have earlier performed API calls from databases to extract infromation. However, I was dealing with a partic

Extracting information from website with BeautifulSoup and Python

I'm attempting to extract information from this website. I can't get the text in the three fields marked in the image (in green, blue, and red rectangles) no ma

Element is not getting selected in Selenium Python Firefox Web Scrapping

I have problem with element selection checkbox with one of the list item in below code snippet. I'm able to select 1st 2nd & 4th Element in below code, howe

How to web scrape the text under <i class>?

I'm trying to get the text "PDF file" under <i class="fa fa-file-pdf-o">. I'm using BeautifulSoup and tried the following, but it didn't work: from bs4 im

How to web scrape the text under <i class>?

I'm trying to get the text "PDF file" under <i class="fa fa-file-pdf-o">. I'm using BeautifulSoup and tried the following, but it didn't work: from bs4 im

getting NoSuchWindowException while scrapping twitter usernames using Selenium

I have been trying to scrape twitter usernames by going inside the followers page but the issue is if I leave my pc there after some time I get this exception a

How will I get the content ie the title of a tag while web scrapping with puppeteer?

I am following this tutorial. https://www.digitalocean.com/community/tutorials/how-to-scrape-a-website-using-node-js-and-puppeteer I am trying to learn how to u

Extract div class text and sub tables in rvest

I am trying to recreate a table from this website under "Battle Pass Rewards." The final result is a data.frame with each of the following areas as different co

Scroll inside div doesn't wrk with Puppeteer

I try to scroll area inside div using Puppeteer. I tried to follow these answers: https://stackoverflow.com/a/67490337 and https://stackoverflow.com/a/52031392

I've been trying to scrape profile pictures in Instagram using this code but i keep getting TypeError: 'NoneType' object is not subscriptable

import requests from bs4 import BeautifulSoup as bs User = input("input the username of the user "); url = 'https://instagram.com/' + User +'/' r = requests.get