04 - Monitor and optimize data storage and data processing

Log Analytics with Azure Synapse Analytics – Bradley Schacht
Log Analytics with Azure Synapse Analytics – Bradley Schacht
While Auditing can be turned on for a database by either enabling it at the workspace or database level, diagnostics must be enabled at the database level. There are diagnostics at the workspace level, but those are different metrics and do not apply to monitoring the dedicated SQL pool.
Auditing can be enabled at the workspace level, which will cover all databases on the workspace automatically, or at the individual database level.
·bradleyschacht.com·
Log Analytics with Azure Synapse Analytics – Bradley Schacht
Azure SQL Auditing for Azure SQL Database and Azure Synapse Analytics - Azure SQL Database
Azure SQL Auditing for Azure SQL Database and Azure Synapse Analytics - Azure SQL Database
Use Azure SQL Database auditing to track database events into an audit log.
Auditing for Azure SQL Database and Azure Synapse Analytics tracks database events and writes them to an audit log in your Azure storage account, Log Analytics workspace, or Event Hubs.
Helps you maintain regulatory compliance, understand database activity, and gain insight into discrepancies and anomalies that could indicate business concerns or suspected security violations.
Enables and facilitates adherence to compliance standards, although it doesn't guarantee compliance. For more information about Azure programs that support standards compliance, see the Azure Trust Center where you can find the most current list of Azure SQL compliance certifications.
·docs.microsoft.com·
Azure SQL Auditing for Azure SQL Database and Azure Synapse Analytics - Azure SQL Database
Monitoring Azure Blob Storage
Monitoring Azure Blob Storage
Learn how to monitor the performance and availability of Azure Blob Storage. Monitor Azure Blob Storage data, learn about configuration, and analyze metric and log data.
·docs.microsoft.com·
Monitoring Azure Blob Storage
Monitor Azure Cosmos DB
Monitor Azure Cosmos DB
Learn how to monitor the performance and availability of Azure Cosmos DB.
·docs.microsoft.com·
Monitor Azure Cosmos DB
Streaming Units in Azure Stream Analytics
Streaming Units in Azure Stream Analytics
This article describes the Streaming Units setting and other factors that impact performance in Azure Stream Analytics.
·docs.microsoft.com·
Streaming Units in Azure Stream Analytics
Use query parallelization and scale in Azure Stream Analytics
Use query parallelization and scale in Azure Stream Analytics
This article describes how to scale Stream Analytics jobs by configuring input partitions, tuning the query definition, and setting job streaming units.
Calculate the max streaming units for a job
·docs.microsoft.com·
Use query parallelization and scale in Azure Stream Analytics
New metric in Azure Stream Analytics tracks latency of your streaming pipeline
New metric in Azure Stream Analytics tracks latency of your streaming pipeline
Azure Stream Analytics is a fully managed service for real-time data processing. Stream Analytics jobs read data from input sources like Azure Event Hubs or IoT Hub. They can perform a variety of tasks from simple ETL and archiving to complex event pattern detection and machine learning scoring.
The watermark represents a specific timestamp in the event time timeline. This timestamp is used as a pointer or indicator of progress in the temporal computations. For example, when Stream Analytics reports a certain watermark value at the output, it guarantees that all events prior to this timestamp were already computed. Watermark can be used as an indicator of liveliness for the data produced by the job. If the delay between the current time and watermark is small, it means the job is keeping up with the incoming data and produces results defined by the query on time.
·azure.microsoft.com·
New metric in Azure Stream Analytics tracks latency of your streaming pipeline
Azure Data Lake Storage query acceleration
Azure Data Lake Storage query acceleration
Query acceleration enables applications and analytics frameworks to dramatically optimize data processing by retrieving only the data that is required for a processing operation.
Query acceleration enables applications and analytics frameworks to dramatically optimize data processing by retrieving only the data that they require to perform a given operation. This reduces the time and processing power that is required to gain critical insights into stored data.
Query acceleration supports CSV and JSON formatted data as input to each request.
·docs.microsoft.com·
Azure Data Lake Storage query acceleration