Access Popup Source code - Python requests

Question

I'm trying to scrape some data from a website.

Adress: https://park4night.com/carte_lieux?lat=41.25807499996962&lng=9.426118999656891&zoom=38

I would like to get the link in the popup appearing when clicking on the pin-point. But I'm having trouble getting the source code to scrape. Using request the source code of the website does not include the one appearing when clicking the marker.

Pin:

Popup link is the redirecting link by clicking "Voir le lieu":

Xpath link I would like to access: /html/body/div[5]/main/div[2]/div[1]/div[6]/div/div[1]/div/div/div/div[2]/div[1]/a

By running $x('/html/body/div[5]/main/div[2]/div[1]/div[6]/div/div[1]/div/div/div/div[2]/div[1]/a/@href')[0] in the javascript console I'm able to get the desired link. I don't know how to perform the same action in python.

How can I access that information? Do I have to use selenium? (since I don't need to perform more actions besides of the ones already possible with requests and lxml.html)

Premise, I'm pretty new to this kind of stuff. Getting handy on handling requests, xPath, ...

Thank you!

Answer 1

When you click on P then click on Voir le lieu then it generates the desired data along with a new url which is not dynamic. So you can follow the next example.

import requests
from bs4 import BeautifulSoup

url='https://park4night.com/fr/lieu/161323/parking-camping-car-jour-et-nuit/porto-massimo-strada-punta-lunga/italy/provincia-di-olbia-tempio#.YnROyNpBzIV'
req=requests.get(url)
#print(req)

soup= BeautifulSoup(req.content,'lxml')

for box in soup.select('.box_template'):
    address = box.select_one('[style="float: left;width: 85%;"] > div ').get_text(strip=True)
    print(address)

Output:

(07024) La Maddalena , Unnamed Road
(07024) , Unnamed Road
(07024) Case Dell'abbatoggia, 2 Via Abbatoggia
(07024) La Maddalena,4 Bassa Trinita
(07024) , 4-2 Bassa Trinita
(07024) La Maddalena,
(07024) La Maddalena,Località Giardinelli     
(07024) La Maddalena,Località Giardinelli
(07024) La Maddalena,
(07024) , Località Giardinelli

step-1 The original url drag and click on p

step-2

After click on P

step-3

After click on location(picture From here you will see the url that is the data container url