databricks

The only info I found on our site is:

https://venturebeat.com/2019/07/10/kyndi-raises-20-million-for-explainable-ai-that-derives-insights-from-documents/

Not very helpful.

On their website (https://docs.databricks.com/user-guide/clusters/mlruntime.html) they say:

Warning - Databricks Runtime ML includes high-performance distributed machine learning packages that use MPI (Message Passing Interface) and other low-level communication protocols. Because these protocols do not natively support encryption over the wire, these ML packages can potentially send unencrypted sensitive data across the network.

Thought that was kinf of funny...

Databricks Runtime 5.4 ML (June 2019)

Databricks Runtime 5.4 ML provides a ready-to-go environment for machine learning and data science based on Databricks Runtime 5.4. Databricks Runtime for ML contains many popular machine learning libraries, including TensorFlow, PyTorch, Keras, and XGBoost. It also supports distributed deep learning training using Horovod.

See:
https://docs.databricks.com/release-notes/runtime/5.4ml.html

Pretty sure we could implement the listed Python and R Libraries in Vertica as UDxs.

These feature will help us compete with Databricks:

Import Spark ML models into Vertica, enable efficient scoring
http://confluence.verticacorp.com:8080/display/DEV/Import+Spark+ML+models+into+Vertica,+enable+efficient+scoring

Spark is one of the most popular tools for distributed machine learning with an increasing number of data scientists using it. We have recently received multiple requests from customers to integrate with other tools with some customers/prospects asking specifically for Spark. Having the capability to import models from Spark is expected to greatly increase Vertica's adoption for machine learning uses in big data. We may plan to use Databrick DBML-local library to implement model export and import.

Import Spark ML models into Vertica, enable efficient scoring
http://jira.verticacorp.com:8080/jira/browse/VER-61405

Integrate TensorFlow with Vertica
http://jira.verticacorp.com:8080/jira/browse/VER-58499

IMO, Databricks is to Spark what Cloudera/Hortonworks is to Hadoop.

Basically, they offer professional support & services for Spark. They also develop bug fixes/feature requests from their customers and help maintain the open source Spark code.

They also do the Spark Summit which we've a booth for almost every year.

Ok, that rings a bell. I think they've been at our local big data conference before. So, databricks=Spark. Got it.