Category "azure-databricks"

How can I access python variable in Spark SQL?

I have python variable created under %python in my jupyter notebook file in Azure Databricks. How can I access the same variable to make comparisons under %sql.

how to pass parameter to python script from a pipeline [closed]

I am building an Azure Data Factory pipeline and I would like to know how to get this parameter into the python script. The python script is l

Databricks Cluster terminated. Reason: Cloud Provider Launch Failure

I'm using Azure Databricks with a custom configuration that uses vnet injection and I am unable to start a cluster in my workspace. The error message being give

define environment variable in databricks init script

I want to define an environment variable in Databricks init script and then read it in Pyspark notebook. I wrote this: dbutils.fs.put("/databricks/scripts/i

Azure Databricks workspace using terraform

Trying to create Databricks workspace using terraform but unsupported arguments: resource "azurerm_databricks_workspace" "workspace" { name = "

Azure DevOps CD Pipeline to Deploy Library to Databricks DBFS 403 Forbidden Error

I'm following the tutorial Continuous integration and delivery on Azure Databricks using Azure DevOps to automate the process to deploy and install library on a

Update using JOIN or CTE in Databricks

I am trying to update a delta table in Databricks using the Databricks documentation here as an example. This document talks only about updating a literal value

How to run a Azure DataBricks Notebook and get it's result via Rest API

I have a use case where I need to run a set of notebooks developed in Azure Databricks (that performs several queries and calculations), but the end user (non-t