'BeautifulSoup: How to find all href links in a div with a class?
On disboard.org/ I am trying to collect all href's within a div with a class of 'server-name'.
Source-Code:
def scrape():
url = 'https://disboard.org/search?keyword=hacking'
response = requests.get(url).content
soup = BeautifulSoup(response, 'html.parser')
areas = soup.find_all('div', class_='server-name')
for area in areas:
print(area.get('href'))
The error message given when this function is called is 'none' instead of the links. Example:
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
Solution 1:[1]
Replace by:
area.find('a').attrs['href']
Full code
import requests
from bs4 import BeautifulSoup
def scrape():
url = 'https://disboard.org/search?keyword=hacking'
response = requests.get(url).content
soup = BeautifulSoup(response, 'html.parser')
areas = soup.find_all('div', class_='server-name')
for area in areas:
print(area.find('a').attrs['href'])
if __name__ == '__main__':
scrape()
Output
/server/484696439063314482
/server/560847285874065408
/server/715563459739385886
/server/720783958966796309
/server/471545766134153237
/server/733350720690061383
/server/653642434948890626
/server/589905664277610521
/server/729633522565775381
/server/734257173890334832
/server/637702746954530865
/server/326839256758616068
/server/495986950478757891
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
Solution | Source |
---|---|
Solution 1 | Philippe Remy |