Azure Databricks - Generate SQL Select Statement with Columns

Question

I have tables in Azure Databricks that I am using SQL to interact with via a notebook. I need to select all columns from a table with 200 columns, I need to select all of them but I need to modify some for a select insert (To modify specific columns for a PK). Therefore I can not use a select *. (There are multiple scenarios this is just my current objective)

How can I generate a select statement on a table with all the column names in a sql statement. This would be equivalent of a 'Select top N' in SSMS where it generates a select for the table I can than edit.

I have seen functions like describe and show but they can't build a select statement. I am new to Databricks. Any help is appreciated.

Answer 1

I have the same problem. It is really tough to make and modify SELECT statement for this kind of tables. I have tried many ways and found using the 3rd party software to connect to the table on Azure Databricks worked fine.

Here is what I do:

1. Download the 3rd party software such as DBeaver
2. Download Databricks JDBC driver form this page.
3. Configure Databricks driver. Luckily there is an official doc for DBeaver.
4. Connect to the Databricks and find the table to generate SELECT statement.
5. Use DBeaver built-in function to generate it. See the screenshot below.

That's it!

I found this setup took just 10-15 minutes to complete saving much time.