Category "web-scraping"

Scraping newspaper article titles with google news

Below is my code for scraping news about domestic violence. This code worked perfectly the first time I used it. But back then I covered only 2-3 months period,

=importhtml Google Sheets table

Unfortunately I don't know how to source this and was wondering if someone could show me how. I am trying to learn hear so the correct answer is great, but the

How hunter.io works (find all email address from domain URL)?

I'm working on a tool to find all emails address from a domain name, as snov.io/hunter.io does. Do you have any idea about how to find all email address from th

Why is web scraping stock prices through beautiful soup returning a different price than the one on the Yahoo Finance page?

I am trying to write a program that will give me the stock price for a few different stocks, but when I run my program, it returns 116.71, while Yahoo Finance h

python selenium scraping a betting site

I recently started coding a program with python selenium. The goal of the project is to calculate arbitrage possibilitys between two sport bookies (but that doe

scraping with rvest with different number of rows in r

https://www.transfermarkt.de/alexander-bade/profil/spieler/31 Is it possible to scrape the whole table in one scrape?

=importxml, Website to Google Sheets - getting #N/A every time

Website Link https://redacted xml options I have tried so far <span aria-labelledby="amount">722</span> //*[@id="amount"]/h3/span[2] /html/body/di

node js puppeteer How do I scrape values ​from duplicate selector?

I want to scrape the value, but its selector is duplicate, I don't know how to solve. It will always scrape the value of the one above. here my code: const pupp

puppeteer bypass cloudflare by enable cookies and Javascript

(In nodeJs -> server side only). I'm doing some webscraping and some pages are protected by the cloudflare anti-ddos page. I'm trying to bypasse this page. B

How to publish a .ics file into my iPhone Calendar?

I have an iPhone and I recently discovered about exportable .ics files as a method to add events/calendars. I have searched Internet for a while and I didn't fi

how to use selenium on web apps built with gradio?

I am trying to click a button on selenium on a web app built using selenium. But ending up with error "element not found". I want to click the submit button usi

how do i press enter on a cr-button with selenium python

# import the webdriver from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.common.keys import Keys import time im

Scraping a simple website with colly in golang does not return any data

I'm trying to scrape a simple website that looks like this: <html> <head> </head> <body> <pre> "Name Surname 1 Name Su

AttributeError: 'NoneType' object has no attribute 'strip' - Scrapy doesn't crawl all the elements

My spider doesn't crawl all the elements. As I can see now, one of the errors is an attribute error which I don't know how to fix it. This is a non-English webs

Facebook Group Post Scraping Using Selenium Only Returns One Post

I'm in the process of building a Facebook Group Scraper, I have managed to write the code to log-in + scrape the name of the, but for some reason, my code is on

Scrapy: Can't Crawling App store Reviews Page

Hi guys I'm having some issues to get data from this page from app store: app store reviewshttps://apps.apple.com/us/app/mathy-cool-math-learner-games/id1476596

Scraping the English Vivino.com reviews from the website

I have two questions about web scraping information from Vivino.com: 1.) With the code below I can scrape information and reviews from the Vivino website, howev

How to import data from a HTML table on a website to excel?

I would like to do some statistical analysis with Python on the live casino game called Crazy Time from Evolution Gaming. There is a website that has the data t

How Can I log-in Amazon with Golang Colly

I am trying login to my amazon buyer account for getting tracking info. I made wordpress-woocommerce login and getting infos but I could not for Amazon. package

Scraping two-column PDF

I try to scrape the texts of hundreds of PDFs for a project. The PDFs have title pages, headers, footers and two columns. I tried the packages pdftools and tabu