Today, I met someone at Databricks and faced some challenges between them and the customer.
The challenges
- The customer wants to connect from Databricks directly into DB (skipping S3) to get the previewed data (in sorted order) for their Data Analyst to view and understand the data.
=> The SA guy at Databricks said: it can not be done, due to some limitations.
- Make the POC of Databricks dashboard with AWS infrastructure.
=> The free account of Databricks does not have the AWS Private Link feauture. It only available on the Enterprise edition.
I just found some thing
The document in the link below has information about the ports used when configuring AWS network objects.
https://docs.databricks.com/en/security/network/classic/privatelink.html#:~:text=0.0%2F0%20for%20these%20ports,6666%3A%20for%20PrivateLink
Keywords:
Databricks, AWS infrastructure, challenge, customer, connect, database, sorted data, limitation, proof of concept, dashboard, free account, Enterprise edition, document, ports, network objects.