'How to bypass Cloudflare bot protection in selenium

I need to grab some information from a site just for education purpose, however i cannot send requests because of the protection. I get The typical Checking-your-browser page shows up first and then i'm being redirected repeatedly. how i can bypass this protection in python selenium?

Solution 1:^[1]

I had this problem a long time ago and I was able to solve it. Use the code below and enjoy :)

options = webdriver.ChromeOptions()
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
options.add_argument("--disable-blink-features=AutomationControlled")
driver = webdriver.Chrome(options=options, executable_path=r"webdriver\chromedriver.exe")

///////// EDIT //////////////// this way now is not working !

Solution 2:^[2]

As March 2022 :

Hi, I had the same problem when using headless Selenium on a Docker Linux image.

I solved it by creating a virtualdisplay right before calling the webdriver:

from pyvirtualdisplay import Display
display = Display(visible=0, size=(800, 800))  
display.start()

Don't forget to install both pyvirtualdisplay and xvfb: pip install pyvirtualdisplay and sudo apt-get install xvfb

And you must remove the "headless" option in ChromeDriver, here is the complete code I use :

    #Display in order to avoid CloudFare bot detection
    display = Display(visible=0, size=(800, 800))  
    display.start()
  
    options = webdriver.ChromeOptions()
    options.add_argument('--no-sandbox')
    options.add_argument('start-maximized')
    options.add_argument('enable-automation')
    options.add_argument('--disable-infobars')
    options.add_argument('--disable-dev-shm-usage')
    options.add_argument('--disable-browser-side-navigation')
    options.add_argument("--remote-debugging-port=9222")
    # options.add_argument("--headless")
    options.add_argument('--disable-gpu')
    options.add_argument("--log-level=3")
    driver = webdriver.Chrome(ChromeDriverManager().install(), chrome_options=options)

Since it was working nicely without headless on my local computer, I figured emulate a real display might do the work aswell. I do not really understand why, but from what I've understood, CloudFare tries to execute javascript code in order to confirm you're not a bot. Having a emulated webpage display helps to do so.

Solution 3:^[3]

A bit of a late response and no wonder why developers still face this issue repetitively. I am using Java v17 and Gradle v7.4.2. My solution is related to the one explained above but the code is in Java.

@Before
public void setup() {
    WebDriverManager.chromedriver().setup();
    ChromeOptions options = new ChromeOptions();
    // Bypass Cloudflare checks
    options.setExperimentalOption("useAutomationExtension", false);
    options.addArguments("--disable-blink-features=AutomationControlled");
    driver = new ChromeDriver(options);
    driver.manage().window().maximize();
}

Please refer to the Selenium ChromeOptions documentation for further details.

Happy coding.

Solution 4:^[4]

SOLUTION JULY 2021

just add user agent argument in chrome options and set user agent to any value

ops = Options() ua='cat' ops.add_argument('--user-agent=%s' % ua) driver=uc.Chrome(executable_path=r"C:\chromedriver.exe",chrome_options=ops)

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution	Source
Solution 1
Solution 2	Jules Civel
Solution 3	Spicy strike
Solution 4	Haider Asad

'How to bypass Cloudflare bot protection in selenium

Solution 1:[1]

Solution 2:[2]

Solution 3:[3]

Solution 4:[4]

Sources

Related Questions

Solution 1:^[1]

Solution 2:^[2]

Solution 3:^[3]

Solution 4:^[4]