Azure IoT Hub is a fully managed service that enables reliable and secure bidirectional communications between millions of IoT devices and a solution back end. Azure IoT Hub:
- Provides multiple device-to-cloud and
- cloud-to-device communication options. These options include one-way messaging, file transfer, and request-reply methods.
- Provides built-in declarative message routing to other Azure services.
- Provides a queryable store for device metadata and synchronized state information.
- Enables secure communications and access control using per-device security keys or X.509 certificates.
- Provides extensive monitoring for device connectivity and device identity management events.
- Includes device libraries for the most popular languages and platforms.
- Establish bi-directional communication with billions of IoT devices
- Authenticate per device for security-enhanced IoT solutions
- Register devices at scale with IoT Hub Device Provisioning Service
- Manage your IoT devices at scale with device management
- Extend the power of the cloud to your edge device
-
Learn to Create IoT Hub
-
Learn to use Simulator to connect to IoT Hub and send Data
-
Learn to setup MXChip, connect to IoT Hub and send data
The infrastructure for your application is typically made up of many components – maybe a virtual machine, storage account, and virtual network, or a web app, database, database server, and 3rd party services.
You do not see these components as separate entities, instead you see them as related and interdependent parts of a single entity. You want to deploy, manage, and monitor them as a group. Azure Resource Manager enables you to work with the resources in your solution as a group. You can deploy, update, or delete all the resources for your solution in a single, coordinated operation.
You use a template for deployment and that template can work for different environments such as testing, staging, and production. Resource Manager provides security, auditing, and tagging features to help you manage your resources after deployment.
Create a resource group to collect and manage all your application resources for this lab
Click on + Add button
Enter Resource group name, Select subscription and region
Create an IoT Hub to connect your real device or simulator to this IoTHub and start sending data.
Click on Create a resource and click on Internet of Things
Click on IoTHub
Make sure you select the resource group you created in previous step.
In the Name field, enter a unique name for your IoT hub. The name of your IoT hub must be unique across all IoT hubs.
Click Size and Scale button.
In the Tier filed, select S1 tier.
You can choose from several tiers depending on how many features you want and how many messages you send through your solution per day. The free tier is intended for testing and evaluation. It allows 500 devices to be connected to the IoT hub and up to 8,000 messages per day. Each Azure subscription can create one IoT Hub in the free tier.
The S1 tier allows total of 400,000 messages per unit per day.
For details about the other tier options, see Choosing the right IoT Hub tier.
Click Review and Create button.
Connect a Simulator to your IoT Hub and stream data.
-
Learn to create a device using Azure Portal
-
Connect the simulator to IoT Hub
-
Send telemetry data to Azure
Go To your IoT Hub in the portal and click on IoT Devices
Click on + Add and enter a Device ID and click Save.
Click on the device and copy the primary key connection string.
Click on the link below to go to the PI Simulator
Replace the connection string with the primary key connection string copied in the previous steps
After you copy the connection string should look like below
Click Run and start sending messages. LED will start blinking
Messages will start flowing into IoT Hub
You will Visualize the Data flowing into IoT Hub by connecting to Time Series Insights
Under IoTHub, Messaging click on Endpoints
Click on Events
Create two consumer groups for
- Time Series Insights
- Stream Analytics
This Lab assumes you are using MXChip as the Device
- updating firmware
- connecting to Wifi
- connecting to Azure to select a subscription and IoTHub
- uploading device code
Prepare MXChip to Connect to IoTHub
Once Device Connects to IoTHub, messages flow into IoThub
Azure Time Series Insights is a fully managed analytics, storage, and visualization service for managing IoT-scale time-series data in the cloud. It provides massively scalable time-series data storage and enables you to explore and analyze billions of events streaming in from all over the world in seconds. Use Time Series Insights to store and manage terabytes of time-series data, explore and visualize billions of events simultaneously, conduct root-cause analysis, and to compare multiple sites and assets.
Time Series Insights has four key jobs:
- First, it's fully integrated with cloud gateways like Azure IoT Hub and Azure Event Hubs. It easily connects to these event sources and parses JSON from messages and structures that have data in clean rows and columns. It joins metadata with telemetry and indexes your data in a columnar store.
- Second, Time Series Insights manages the storage of your data. To ensure data is always easily accessible, it stores your data in memory and SSD’s for up to 400 days. You can interactively query billions of events in seconds – on demand.
- Third, Time Series Insights provides out-of-the-box visualization via the TSI explorer.
- Fourth, Time Series Insights provides a query service, both in the TSI explorer and by using APIs that are easy to integrate for embedding your time series data into custom applications.
In this lab you will learn
- how to set up a Time Series Insights environment
- explore
- analyze time series data of your IoT solutions or connected things
Click on Create a Resource and click on Internet of Things
Click on Time Series Insights
Select the resource group you previously created and click Create button
Create Event Source to connect to IoTHub. Please make sure you use a unique Consumer Group. Time Series Insights has a requirement to have its own unique consumer group
Select the appropriate consumer group and click Create button
Go To Time Series Insights, Click on Go To Environment which will take you to Time Series Insights Explorer
If you get Data Access Policy Error execute the following steps
Go To Environment Topology and
Click on Add Button
Select Contributor Role
Select User
Go To Time Series Insights Explorer
Split By ID. You will see data flowing from two devices. MXChip and Pi Simulator.
Select humidity and Split By ID. You will see data flowing from two devices. MXChip and Pi Simulator.
Right Click to Explore events. You can download events in CSV and JSON format by clicking on CSV or JSON buttons
Create a perspective by clicking on the image shown below
Click + to add a new query
Select Temperature and split by Device ID and click on perspective image.
Create a chart by selecting a timeframe with drag feature
Create a Chart by adding a predicate
Perspective with 4 different charts and also changed Title
Click on Heatmap
View data in a table
Azure Data Lake Store is an enterprise-wide hyper-scale repository for big data analytic workloads. Azure Data Lake enables you to capture data of any size, type, and ingestion speed in one single place for operational and exploratory analytics. Data Lake Store can store trillions of files. A single file can be larger than one petabyte in size. This makes Data Lake Store ideal for storing any type of data including massive datasets like high-resolution video, genomic and seismic datasets, medical data, and data from a wide variety of industries.
Create a hyper scale data lake store to store IoT Data. Click on Create a resource
Click on Data + Analytics
Click on Data Lake Store
During creation of data lake you have the choice to encrypt the store
Data Lake Store protects your data assets and extends your on-premises security and governance controls to the cloud.
Your data is
- always encrypted
- while in motion using SSL
- at rest using service or user-managed HSM-backed keys in Azure Key Vault.
Single sign-on (SSO), multi-factor authentication, and seamless management of millions of identities is built-in through Azure Active Directory. Authorize users and groups with fine-grained POSIX-based ACLs for all data in your store and enable role-based access controls. Meet security and regulatory compliance needs by auditing every access or configuration change to the system.
Click on Create button
Create /workshop/streaming folder to store Streaming data coming from your device through IoTHub using Stream Analytics Job
Create /workshop folder
Create /workshop/streaming folder
You should have the folder structure below in place to start streaming data to data lake store
Azure Stream Analytics is a managed event-processing engine set up real-time analytic computations on streaming data. The data can come from devices, sensors, web sites, social media feeds, applications, infrastructure systems, and more
<iframe src="https://channel9.msdn.com/Shows/Internet-of-Things-Show/Stream-Analytics-in-IoT-solutions/player" width="480" height="270" allowFullScreen frameBorder="0"></iframe>Create a hyper scale data lake store to store IoT Data. Click on Create a resource
Click on Data + Analytics
Click on Stream Analytics Job
Stream Analytics job cab be created to run on the cloud as well as on the Edge. You will chose to run this on the cloud
Add Input for Streaming Job
Select IoTHub as Input
Make sure to provide a consumer group. Each consumer group allows up to 5 output sinks/consumers. Make sure you create a new consumer group for every 5 output sinks and you can create up to 32 consumer groups.
Add Data Lake Store as Output for Streaming Job
Select Data Lake Store as output sink
Select the Data Lake Store account you created in previous steps and provide folder structure to stream data to the store
/workshop/streaming/{date}/{time} with Date=YYYY/MM/DD format and Time=HH format will equate to /workshop/streaming/2018/03/30/11 on the store
You will have to Authorize data lake store connection for Stream analytics to have access to be able to write to data lake store
- Multi-factor authentication based on OAuth2.0
- Integration with on-premises AD for federated authentication
- Role-based access control
- Privileged account management
- Application usage monitoring and rich auditing
- Security monitoring and alerting
- Fine-grained ACLs for AD identities
You will see a popup and once the popup closes Authorize button will be greyed out after azuthorization is complete. There are exception cases where popup doesnt appear.In this case try again in incognito mode
Edit Query for Streaming Job, Stream Data from IoTHub to Datalake Store
Query
SELECT
*, System.Timestamp as time
INTO
DatalakeStore
FROM
IotHubSave the query
Accept by pressing yes
Start the stream job which will read data from IoTHub and store data in Data lake Store
You can pick custom time to go back a few hours to pick up data from when your device has started streaming
Wait till job goes into running state, if you see errors could be from your query, make sure syntax is correct
Go to Data Lake store data explorer and drill down to /workshop/streaming folder.You will see folders created with YYYY/MM/DD/HH format.
You will see json files, with one file per hour, explore the data
