'Unable to get correct CSS tag for webscraping in R using SelectorGadget
I am trying to web scrape data in R
from this url but cannot seem to get the correct css
tag. For now I just need help retrieving the professor's name. Any help would be appreciated. I am using the Selector Gadget to get the css
tag.
https://www.ratemyprofessors.com/search/teachers?query=*&sid=1245
library(rvest)
library(tidyverse)
url <- "https://www.ratemyprofessors.com/search/teachers?query=*&sid=1245"
url %>%
read_html() %>%
html_nodes(css=".cJdVEK") %>%
html_text()
Solution 1:[1]
I was able to get the names with the following code :
library(RSelenium)
shell('docker run -d -p 4445:4444 selenium/standalone-firefox')
remDr <- remoteDriver(remoteServerAddr = "localhost", port = 4445L, browserName = "firefox")
remDr$open()
remDr$navigate('https://www.ratemyprofessors.com/search/teachers?query=*&sid=1245')
web_Obj_Popup <- remDr$findElement("xpath", "/html/body/div[5]/div/div/button")
web_Obj_Popup$clickElement()
for(i in 1 : 100)
{
print(i)
remDr$executeScript("window.scrollBy(0, 25)")
}
web_Obj_Teacher_Name <- remDr$findElements("class name", "cJdVEK")
professor_Names <- lapply(X = web_Obj_Teacher_Name, FUN = function(x) x$getElementText()[[1]])
professor_Names
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
Solution | Source |
---|---|
Solution 1 | Emmanuel Hamel |