Is it possible to speed up move_to_element() in Selenium or what are other alternatives?

Question

What is the fastest way to trigger an onmouseover event when scraping a webpage?

So I want to move the mouse over a div element, which is then calling a javascript function to update another div (displaying a tooltip, which I want to scrape). When moving the mouse off the first div, the tooltip disappears, but it can still be scraped, because only the display style is set to none. I would like to make this procedure as quick as possible.

Currently I use Selenium and move_to_element which takes roughly 0.55s. This is quite long, as I have to repeat this procedure eventually >60 times.

start_time = time.time()
action = ActionChains(driver)
action.move_to_element(box)
action.perform()
print("Time: ", time.time() - start_time()) # ~0.5s

When I use this in the scrapy shell, I see the following message displayed for this selenium request.

[selenium.webdriver.remote.remote_connection] DEBUG: POST http://127.0.0.1:57309/session/49aea696ad5c6a23dd1a292adacf1d92/actions {"actions": [{"parameters": {"pointerType": "mouse"}, "type": "pointer", "id": "mouse", "actions": [{"duration": 250, "x": 0, "type": "pointerMove", "y": 0, "origin": {"element-6066-11e4-a52e-4f735466cecf": "63f7afad-d9d6-49c2-989a-75c6d083c055"}}]}, {"type": "key", "id": "key", "actions": [{"duration": 0, "type": "pause"}]}]}

At "actions" it says "duration: 250", I suspect this is the duration time of the move_to_element action. I wasn't able to find any way to manually reduce this duration, is this possible? I also heard that scrapy-splash is way quicker than selenium, but haven't found alot on how to use it. Can it also emulate hover events, or call the javascript function directly with the correct arguments?

Answer 1

Although this is old. If it helps anyone... at least in Python selenium module you can set the duration of the mouse movement by setting the following property on the action chain:

driver = webdriver.Firefox()
act = ActionChains(driver)
act.w3c_actions.pointer_action._duration = 0

The default value appears to be set to 250.

Answer 2

There are a couple of things:

• onmouseover: onmouseover event executes a JavaScript when moving the mouse pointer onto an image. An example:

  <img onmouseover="bigImg(this)" src="smiley.gif" alt="Smiley">
  

• move_to_element(to_element): move_to_element() moves the mouse to the middle of an element.

• The time which you have measured i.e. ~0.55s involves a couple of other activities along with move_to_element(to_element) such as:

  • action = ActionChains(driver)
  • action.perform()
  • print() etc.

Now it is worth to mention that, methods of action_chains by default ensures that that the desired element is is_displayed() and is_enabled() within the DOM Tree.

---

Conclusion

Considering the above mentioned facts, ~500-550 ms of timespan looks pretty optimized.