DP-203 References

DP-203 References

113 bookmarks
Newest
UNION (Azure Stream Analytics) - Stream Analytics Query
UNION (Azure Stream Analytics) - Stream Analytics Query
Combines the results of two or more queries into a single result set that includes all the rows that belong to all queries in the union.
The following are basic rules for combining the result sets of two queries by using UNION: The number and the order of the columns must be the same in all queries. The data types must be compatible. Streams must have the same partition and partition count
·docs.microsoft.com·
UNION (Azure Stream Analytics) - Stream Analytics Query
Spark Streaming - Different Output modes explained - Spark by {Examples}
Spark Streaming - Different Output modes explained - Spark by {Examples}
This article describes usage and differences between complete, append and update output modes in Apache Spark Streaming. outputMode describes what data is written to a data sink (console, Kafka e.t.c) when there is new data available in streaming input (Kafka, Socket, e.t.c)
Use complete as output mode outputMode("complete") when you want to aggregate the data and output the entire results to sink every time.
It is similar to the complete with one exception; update output mode outputMode("update") just outputs the updated aggregated results every time to data sink when new data arrives.
Use append as output mode outputMode("append") when you want to output only new rows to the output sink.
·sparkbyexamples.com·
Spark Streaming - Different Output modes explained - Spark by {Examples}
Maximize throughput with repartitioning in Azure Stream Analytics |...
Maximize throughput with repartitioning in Azure Stream Analytics |...
Customers love Azure Stream Analytics for its ease of analyzing streams of data in movement, with the ability to set up a running pipeline within five minutes. Optimizing throughput has always been...
When joining two streams of data explicitly repartitioned, these streams must have the same partition key and partition count. The outcome is a stream that has the same partition scheme.
·azure.microsoft.com·
Maximize throughput with repartitioning in Azure Stream Analytics |...
Materialized View pattern - Azure Architecture Center
Materialized View pattern - Azure Architecture Center
Generate prepopulated views over the data in one or more data stores when the data isn't ideally formatted for required query operations.
Creating this materialized view requires complex queries. However, by exposing the query result as a materialized view, users can easily obtain the results and use them directly or incorporate them in another query. The view is likely to be used in a reporting system or dashboard, and can be updated on a scheduled basis such as weekly.
·docs.microsoft.com·
Materialized View pattern - Azure Architecture Center
Log Analytics with Azure Synapse Analytics – Bradley Schacht
Log Analytics with Azure Synapse Analytics – Bradley Schacht
While Auditing can be turned on for a database by either enabling it at the workspace or database level, diagnostics must be enabled at the database level. There are diagnostics at the workspace level, but those are different metrics and do not apply to monitoring the dedicated SQL pool.
Auditing can be enabled at the workspace level, which will cover all databases on the workspace automatically, or at the individual database level.
·bradleyschacht.com·
Log Analytics with Azure Synapse Analytics – Bradley Schacht
Azure SQL Auditing for Azure SQL Database and Azure Synapse Analytics - Azure SQL Database
Azure SQL Auditing for Azure SQL Database and Azure Synapse Analytics - Azure SQL Database
Use Azure SQL Database auditing to track database events into an audit log.
Auditing for Azure SQL Database and Azure Synapse Analytics tracks database events and writes them to an audit log in your Azure storage account, Log Analytics workspace, or Event Hubs.
Helps you maintain regulatory compliance, understand database activity, and gain insight into discrepancies and anomalies that could indicate business concerns or suspected security violations.
Enables and facilitates adherence to compliance standards, although it doesn't guarantee compliance. For more information about Azure programs that support standards compliance, see the Azure Trust Center where you can find the most current list of Azure SQL compliance certifications.
·docs.microsoft.com·
Azure SQL Auditing for Azure SQL Database and Azure Synapse Analytics - Azure SQL Database
New metric in Azure Stream Analytics tracks latency of your streaming pipeline
New metric in Azure Stream Analytics tracks latency of your streaming pipeline
Azure Stream Analytics is a fully managed service for real-time data processing. Stream Analytics jobs read data from input sources like Azure Event Hubs or IoT Hub. They can perform a variety of tasks from simple ETL and archiving to complex event pattern detection and machine learning scoring.
The watermark represents a specific timestamp in the event time timeline. This timestamp is used as a pointer or indicator of progress in the temporal computations. For example, when Stream Analytics reports a certain watermark value at the output, it guarantees that all events prior to this timestamp were already computed. Watermark can be used as an indicator of liveliness for the data produced by the job. If the delay between the current time and watermark is small, it means the job is keeping up with the incoming data and produces results defined by the query on time.
·azure.microsoft.com·
New metric in Azure Stream Analytics tracks latency of your streaming pipeline
Azure Data Lake Storage query acceleration
Azure Data Lake Storage query acceleration
Query acceleration enables applications and analytics frameworks to dramatically optimize data processing by retrieving only the data that is required for a processing operation.
Query acceleration enables applications and analytics frameworks to dramatically optimize data processing by retrieving only the data that they require to perform a given operation. This reduces the time and processing power that is required to gain critical insights into stored data.
Query acceleration supports CSV and JSON formatted data as input to each request.
·docs.microsoft.com·
Azure Data Lake Storage query acceleration
Customer-managed transparent data encryption (TDE) - Azure SQL Database & SQL Managed Instance & Azure Synapse Analytics
Customer-managed transparent data encryption (TDE) - Azure SQL Database & SQL Managed Instance & Azure Synapse Analytics
Bring Your Own Key (BYOK) support for transparent data encryption (TDE) with Azure Key Vault for SQL Database and Azure Synapse Analytics. TDE with BYOK overview, benefits, how it works, considerations, and recommendations.
·docs.microsoft.com·
Customer-managed transparent data encryption (TDE) - Azure SQL Database & SQL Managed Instance & Azure Synapse Analytics