Monitor data factories using Azure Monitor - Azure Data Factory
Learn how to use Azure Monitor to monitor Azure Data Factory pipelines by enabling diagnostic logs with information from Data Factory.
04 - Monitor and optimize data storage and data processing
Log Analytics with Azure Synapse Analytics – Bradley Schacht
While Auditing can be turned on for a database by either enabling it at the workspace or database level, diagnostics must be enabled at the database level. There are diagnostics at the workspace level, but those are different metrics and do not apply to monitoring the dedicated SQL pool.
Auditing can be enabled at the workspace level, which will cover all databases on the workspace automatically, or at the individual database level.
Azure SQL Auditing for Azure SQL Database and Azure Synapse Analytics - Azure SQL Database
Use Azure SQL Database auditing to track database events into an audit log.
Auditing for Azure SQL Database and Azure Synapse Analytics tracks database events and writes them to an audit log in your Azure storage account, Log Analytics workspace, or Event Hubs.
Helps you maintain regulatory compliance, understand database activity, and gain insight into discrepancies and anomalies that could indicate business concerns or suspected security violations.
Enables and facilitates adherence to compliance standards, although it doesn't guarantee compliance. For more information about Azure programs that support standards compliance, see the Azure Trust Center where you can find the most current list of Azure SQL compliance certifications.
Monitor workload - Azure portal - Azure Synapse Analytics
Monitor Synapse SQL using the Azure portal
sys.dm_pdw_exec_requests (Transact-SQL) - SQL Server
sys.dm_pdw_exec_requests (Transact-SQL)
sys.dm_pdw_waits (Transact-SQL) - SQL Server
sys.dm_pdw_waits (Transact-SQL)
Monitoring Azure Blob Storage
Learn how to monitor the performance and availability of Azure Blob Storage. Monitor Azure Blob Storage data, learn about configuration, and analyze metric and log data.
Monitor Azure Cosmos DB
Learn how to monitor the performance and availability of Azure Cosmos DB.
Monitor and debug with insights in Azure Cosmos DB
Use metrics in Azure Cosmos DB to debug common issues and monitor the database.
Manageability and monitoring - query activity, resource utilization - Azure Synapse Analytics
Learn what capabilities are available to manage and monitor Azure Synapse Analytics. Use the Azure portal and Dynamic Management Views (DMVs) to understand query activity and resource utilization of your data warehouse.
Streaming Units in Azure Stream Analytics
This article describes the Streaming Units setting and other factors that impact performance in Azure Stream Analytics.
Send Databricks app logs to Azure Monitor - Azure Architecture Center
Learn how to send application logs and metrics from Azure Databricks to a Log Analytics workspace using the Azure Databricks Monitoring Library.
Use query parallelization and scale in Azure Stream Analytics
This article describes how to scale Stream Analytics jobs by configuring input partitions, tuning the query definition, and setting job streaming units.
Calculate the max streaming units for a job
New metric in Azure Stream Analytics tracks latency of your streaming pipeline
Azure Stream Analytics is a fully managed service for real-time data processing. Stream Analytics jobs read data from input sources like Azure Event Hubs or IoT Hub. They can perform a variety of tasks from simple ETL and archiving to complex event pattern detection and machine learning scoring.
The watermark represents a specific timestamp in the event time timeline. This timestamp is used as a pointer or indicator of progress in the temporal computations. For example, when Stream Analytics reports a certain watermark value at the output, it guarantees that all events prior to this timestamp were already computed. Watermark can be used as an indicator of liveliness for the data produced by the job. If the delay between the current time and watermark is small, it means the job is keeping up with the incoming data and produces results defined by the query on time.
Log Analytics workspace data export in Azure Monitor (preview) - Azure Monitor
Log Analytics workspace data export in Azure Monitor lets you continuously export data per selected tables in your workspace, to an Azure Storage Account or Azure Event Hubs as it's collected.
Workload management - Azure Synapse Analytics
Guidance for implementing workload management in Azure Synapse Analytics.
Monitor your dedicated SQL pool workload using DMVs - Azure Synapse Analytics
Learn how to monitor your Azure Synapse Analytics dedicated SQL pool workload and query execution using DMVs.
Filter data by using Azure Data Lake Storage query acceleration
Use query acceleration to retrieve a subset of data from your storage account.
Azure Data Lake Storage query acceleration
Query acceleration enables applications and analytics frameworks to dramatically optimize data processing by retrieving only the data that is required for a processing operation.
Query acceleration enables applications and analytics frameworks to dramatically optimize data processing by retrieving only the data that they require to perform a given operation. This reduces the time and processing power that is required to gain critical insights into stored data.
Query acceleration supports CSV and JSON formatted data as input to each request.