Real-Time Analytics is a data analytics SaaS experience in the Microsoft Fabric offering Eventstream for capturing, transforming, and routing real-time events, KQL database for data storage and management, and KQL queryset to run queries, view, and customize query results on data.
This blog post will explain how to use Power Automate to send events to Fabric Eventstream by using the Custom App as the Source of the Eventstream.
Video:
Eventstream in Microsoft Fabric:
The event streams feature in Microsoft Fabric gives you a centralized place in the Fabric platform to capture, transform, and route real-time events to various destinations with a no-code experience. It's part of the Real-Time Analytics experience. The eventstream item you create in the portal is an instance of Fabric event streams. When you create an eventstream, you add event data sources, routing destinations, and the event processor when you need the transformation.
Eventstream is part of the Microsoft Fabric Real-Time Analytics, and it gives you a centralized place to capture, transform, and route real-time events to various destinations with a no-code experience.
In this blog the custom app will be a flow in Power Automate to send events to an Eventstream on Microsoft fabric:
Create an Eventstream with Custom App:
Open Microsoft Fabric, click on the icon at the button left and select "Synapse Real-Time Analytics". Click on the Eventstream icon and on the new window, add a name for your eventstream, and click on Create.
Once your eventstream is ready, you can add the source at the left side and destination on the right side to it.
Click on the New source and select Custom App. Add a name for your App, and click on Add button. Once it's ready you can see the details of your App as this image:
On the details section, you can see 3 tabs (highlighted in Red on the above image) for the details of the App in 3 different streaming technologies or data communication models.For each of those 3 item there are 3 sections on the left side (highlighted in Blue on the above image) with Basic, Key, and Sample code details.
I want to use the Event Hub in this blog, so I will use the Event Hub tab.
Create a Power Automate Flow:
Once your Eventstream with Custom App as the source is ready, open the Power Automate and create a flow with any Trigger that suits your project.
I will use the "Instant cloud flow - Triggered manually as needed" to be able to test my flow manually and at the end when the flow is completed, It is possible to change the trigger to any other trigger that I want.
As the first step, I will switch to the old designer that I like it a lot. and then I will add a new step to the flow by selecting the Event Hub - Send Event action. Keep in mind, this action is Premium action, so you need to have the proper Power Automate license to be able to use it
Once you added the Send event action to the flow, you need to set up the connection for that action. Click on the ellipsis or ... at the top right of the action to open the menu and then select add new connection.
Set up the Connection to connect the Send Event Action to the Eventstream Custom App:
Clicking the add new connection button opens a new window with two fields, Authentication Type and Connection String/Namespace Endpoint.
Authentication Type has two options: Access Key and Microsoft Entra ID Integrated. To connect to the Microsoft Fabric Eventstream custom app, you need to select Access Key.
A Connection String is now required, which you can find in the details tab of your Custom App in Microsoft Fabric Eventstream. Click on Event hub, then Keys, and you can see 6 fields as this:
Click on the eye icon for the Connection string-primary key, the key will be revealed and you will be able to copy the entire key by clicking on the copy icon. Copy that key, back to the Power Automate, and paste the key as the connection string for the Event Hubs connection and click on the create button. Now, the connection has been set, and you need to add the Event Hub name for the Send event action.
Click on the Select an item on Event Hub name field, and you will see the dynamic invocation request failed error which is totally fine, because the Event hub name for the Fabric eventstream custom app needs to be set manually:
Click on the Add a custom item, and now you need to copy the Event hub name from the details tab of your Custom App and paste it there:
Open the details tab and copy the Event hub name:
Paste the name to the Add a custom item (Event Hub name), and click n Ok button:
Now the Send event action on your flow is ready to use.
Set up the Content to send data to Eventstream :
To send data to the eventstream using the Send event action on Power Automate, you need to add your data in JSON format in the Content section of the Send event action.
Any standard JSON string will work there, you can create something simple like this:
Add that JSON string to the content section of the action and let's test the flow:
If you created your flow same as what I explained, then click on the test, select Manually and click on Save & Test, and then Run flow. When you see the green Your flow ran successfully notification, back to the Fabric and let's check the result.
In your eventstream page, select the eventstream item between source and destination, and check the Data preview tab:
Click on the Data Insights tab, to see more details about input and output of the eventstream in different periods:
Note: Make sure to send the first data in the right structure (Number of columns and name of them) because you can't use that app for the new structure.
You can use my previous blog for creating Power Automate flow to generate random data and send them to the Fabric eventstream for testing different scenarios.
It is now possible to update your Power Automate flow with any actions and items you want and send the output to the Fabric eventstream. This metohod would be useful for running APIs and send the result to a Fabric eventstream.
In the next blog I will use this method to collect data and save them in KQL database for creating real-time Power BI report.