Data Connections to Kusto in Azure

Data movement from one source to another source is a common use case in general we usually see due to different reasons.

Example: let’s say that we want to do some analytics/aggregations on the data where in Kusto is one of the best technologies in Azure which helps in performing wide range of analytics. However your source data is either streaming data or may be stored in some other sources like CosmosDB/Storage Account blob. We can think of using the Azure Data Factory (ADF) for moving the data but however it comes with additional implementation of pipelines , maintenance and monitoring.

Question : So how do we move the data from sources (Cosmos DB/Storage Account Blob/Event Hubs/ IOT Hubs) to Kusto without having to do any implementation/maintenance. ?

Answer : Data Connections in Azure Data Explorer (Kusto) provides the capability to perform the movement with almost zero or minimal efforts involved in setting up data connection.

Data Connections setup in Azure

Data Connection can be setup either from the Azure Portal or by deploying to Azure via ARM templates. Setting up from Azure Portal is quite straight forward. Go to Azure data explorer resource -> Databases -> Click on one of the data bases already created -> left side there is section called Data Connections.

There are four different types of Data Connection that are currently available in the Azure Data Explorer for moving the data. So we talk about two to three data connections that are widely used in general.

Cosmos DB Data Connection

Data Connection from Cosmos to Kusto is built on the concept of Change Feed capability offered by the Cosmos DB. Once the data connection is setup the data is moved seamlessly from Cosmos to Kusto without having to do further maintenance.

Event Hub Data Connection

Data Connection from Event Hub to Kusto is one of the other use cases which can be used to analyse the streaming data coming from event hubs.

Event Grid Data Connection

Data Connection from Event Grid to Kusto is useful when we want to analyse the data stored in the blob.