Databricks

I am trying to figure out how to work with Microsoft’s Azure cloud platform and all their services. These are notes to self for how to do things and how to get set-up on OSX. Microsoft has the documentation, this is just faster and less verbose in case I need to duplicate or help a colleague get setup.

Things to figure out

Databricks command line tool

Install the thing.

pip3 install databricks-cli

Authentication: set up a token. To get the token go to profile -> user settings, generate token there.

databricks configure --token

List the workspaces.

databricks workspace list

List subfolders in a workspace.

databricks workspace list /Users

Copy the content of whatever is online to local machine. Use export_dir. First I thought of “import”, but turned out I have to think of the view point of databricks.

databricks workspace export_dir /Users/xxx@xxx.xxxxxx.com path-to-local-folder

Push a file from local machine to databricks.

databricks workspace import testsync.py  /Users/xx@xx.xxxx.com/testsync.py --language PYTHON