Category "web-scraping"

how do i press enter on a cr-button with selenium python

# import the webdriver from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.common.keys import Keys import time im

Scraping a simple website with colly in golang does not return any data

I'm trying to scrape a simple website that looks like this: <html> <head> </head> <body> <pre> "Name Surname 1 Name Su

AttributeError: 'NoneType' object has no attribute 'strip' - Scrapy doesn't crawl all the elements

My spider doesn't crawl all the elements. As I can see now, one of the errors is an attribute error which I don't know how to fix it. This is a non-English webs

Facebook Group Post Scraping Using Selenium Only Returns One Post

I'm in the process of building a Facebook Group Scraper, I have managed to write the code to log-in + scrape the name of the, but for some reason, my code is on

Scrapy: Can't Crawling App store Reviews Page

Hi guys I'm having some issues to get data from this page from app store: app store reviewshttps://apps.apple.com/us/app/mathy-cool-math-learner-games/id1476596

Scraping the English Vivino.com reviews from the website

I have two questions about web scraping information from Vivino.com: 1.) With the code below I can scrape information and reviews from the Vivino website, howev

How to import data from a HTML table on a website to excel?

I would like to do some statistical analysis with Python on the live casino game called Crazy Time from Evolution Gaming. There is a website that has the data t

How Can I log-in Amazon with Golang Colly

I am trying login to my amazon buyer account for getting tracking info. I made wordpress-woocommerce login and getting infos but I could not for Amazon. package

Scraping two-column PDF

I try to scrape the texts of hundreds of PDFs for a project. The PDFs have title pages, headers, footers and two columns. I tried the packages pdftools and tabu

Scraping video url from multiple pages of a website

I am doing scraping using selenium. i want to scrape all the video urls that are present in the 626 products on the 25 pages. but while extracting the url it gi

WebScraping using Python & Azure : How to run an Azure Function to store pandas dataframe into a CosmoDB?

My porpouse is to automate the aquisition of data from a website. I have my python scripts to scrape tables and i can store it as a dataframe or in a csv,xlsx f

Why I'm getting "UnicodeEncodeError: 'charmap' codec can't encode character '\u25b2' in position 84811: character maps to <undefined>" error?

I'm getting UnicodeEncodeError: 'charmap' codec can't encode character '\u200b' in position 756: character maps to error while running this code:: from bs4 imp

How to use selenium for webscraping google flights?

I'm trying to pull the airline names and prices of a specific flight. I'm having trouble with the x.path and/or using the right html tags because when I run the

Exception has occured: WebDriverException // Session deleted because of page crash

while I was able to get help on another issue with a python-based web scraper, another issue comes up when I run the code. Page crash Can someone tell me why it

google_play_scraper cannot crawl all reviews

AS I followed codes of reviews_all from https://github.com/JoMingyu/google-play-scraper I sitll cannot get all reviews, just only a few and not even sorted by d

How to ignore infobox when scraping title from Wikipedia anchor text?

I am trying to scrape the first 20 links on a Wikipedia page but I want to ignore the infobox on the right side. It has a 'table' tag. Here is what I have so fa

Webscraping Google Search Results Using Google API - Returns same result over and over again

My problem Hi everyone I am attempting to develop my very first web scraper using the Google API and Beautiful Soup in Python. The aim is for the scraper to

I disabled loading images in chrome while using webdriver with selenium now cant enable it

I disabled loading images in chrome while using webdriver with selenium now cant enable it. I was using python to webscrape on instagram so thought it would be

Is Scrapy Asychronous by Default?

I recently ran a spider in my project but I feel like scrapy it is waiting until one page is finished to move on the other one. if I am correct in scrapy's natu

How to open a new tab using Python Playwright by feeding it a list of URLs?

According to the Playwright documentation, the way to open a new tab in the browser is as shown in the scrap_post_info() function? However, it failed to do so.