'How to run a Azure DataBricks Notebook and get it's result via Rest API
I have a use case where I need to run a set of notebooks developed in Azure Databricks (that performs several queries and calculations), but the end user (non-technical) needs only the final result, displayed in a web page.
I searched the Azure Databricks APIs but didn't find anything about programmatically running (and getting the results) of notebooks.
Did someone already performed some task like that?
Is there a REST API that allows developers to execute Azure Databricks Notebooks (or some kind of Library for Node.js)?
Solution 1:[1]
Yes, it's covered by the Jobs REST API:
You can execute notebook:
- either by creating a new job (you need
notebook_task) and then triggering the new job run - or creating a single run (also called RunSubmit) - also
notebook_task
In either case, you will get a run ID, and then you need to wait until job is finished (checking the state via get run API until it's get life_cycle_state equal to TERMINATED), and then retrieve the output of the run
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | Alex Ott |