'Scrapping Reddit post with PRAW gives limited amount of posts
I want to scrape Reddit comments using the PRAW package. My problem is that I cannot scrape the maximum number of posts (which is 1000, I believe), but that my search only gives me 249 results. I need a code that allows me to increase the number of scrapped posts.
Specifically, I want to search for Reddit posts that contain search terms (e.g., "dog"; see code below). I set the limit of the search to 1000. However, I always get about 245 results (even though I know that there should be more than 1000 posts).
import praw
reddit = praw.Reddit(
user_agent="X",
client_id="Y",
client_secret="Z",)
all = reddit.subreddit("all")
e = -1
for i in all.search(query = "dog", sort = "relevance", time_filter = "year", limit=1000):
print ('https://www.reddit.com' + i.permalink)
e = e+1
x = 'https://www.reddit.com' + i.permalink
print(e)
249
Can anyone of you help with this?
Any help would be really great because this problem really hunts me for a while now!
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
Solution | Source |
---|