How to bypass Cloudflare with Python on GET requests?

Question

I want to bypass Cloudflare on a GET request I have tried using Cloudscraper which worked for me in the past but now seems decreped.

I tried:

import cloudscraper

ses = requests.Session()
ses.headers = {
    'referer': 'https://magiceden.io/',
    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.84 Safari/537.36',
    'accept': 'application/json'
}
scraper = cloudscraper.create_scraper(sess=ses)
hookLink = f"https://magiceden.io/launchpad/planetarians"
meG = scraper.get("https://api-mainnet.magiceden.io/launchpads/planetarians")
print(meG.status_code)
print(meG.text)

The issue seems to be that Im getting a captcha on the request

Answer 1

The python library works well (I never knew about it), the issue is your user agent. Cloudflare uses some sort of extra checks to be able to determine you're faking it.

For me, any of the following works:

ses.headers = {
    'referer': 'https://magiceden.io/',
    'accept': 'application/json'
}

ses.headers = {
    'accept': 'application/json'
}

And also just:

scraper = cloudscraper.create_scraper()
meG = scraper.get("https://api-mainnet.magiceden.io/launchpads/planetarians")

EDIT:

You can use this dict syntax instead to fake the user agent (as per the manual)

scraper = cloudscraper.create_scraper(
    browser={
        'browser': 'chrome',
        'platform': 'windows',
        'desktop': True
    }
)