Databricks is a unified cloud-based analytics platform that streamlines data engineering,
data science, and machine learning on large datasets at scale.You can use Refuel to read and write data back into Databricks.
For Refuel to read and write data back into Databricks, we need these pieces of information:
1
SQL Warehouse Details
Open your databricks workspace console
Navigate to your SQL Warehouse
Open the connection details tab, get the Server hostname and HTTP path
Get the Unity Catalog name. This is often the same as the workspace name
2
Authentication
Open your databricks workspace console
Click on your user profile icon in the top-right corner, and head to settings, then developer, then manage under access tokens
Enter a token description and the token validity (leave blank for a permanent token)
3
Setup Integration in Refuel
In order to setup the databricks integration, navigate to Integrations within Refuel Cloud Settings, select Databricks, and fill in the following fields:
Server Hostname
Http Path
Access Token (Generated in Step 2)
Unity Catalog Name
Output Schema (Where Refuel will create output tables)