Scenario I have a notebook that processes hot data every 5 minutes. Meanwhile, another pipeline processes historical data, and I want to create a summary table that uses the hot data incrementally …

Guidance to Maximizing Productivity and Efficiency for your Data Engineering Workloads Data engineering teams often grapple with the complexities of planning and configuring compute resources for their data platforms. This is especially true when working with large-scale, complex datasets and demanding downstream SLAs. A one-size-fits-all approach is rarely effective, as different data layers and datasets … <p class="link-more"><a href="https://blog.fabric.microsoft.com/en-us/blog/optimizing-spark-compute-for-medallion-architectures-in-microsoft-fabric/" class="more-link">Continue reading<span class="screen-reader-text"> “Optimizing Spark Compute for Medallion Architectures in Microsoft Fabric”</span></a>

Identity column feature is an excellent example of the "missing puzzle" in Microsoft Fabric. Hence, we still have to rely on some workarounds and "hacks" to get the job done. This article helps you choose the solution that will play to your strengths

introduction. Currently, when proactively

One of the amazing things about Microsoft Fabric is the number of options you have for moving data. For example, you can use Dataflow Gen2, Pipelines, Notebooks, or any combination of the three. However, with all the options available, it can also ma...

As part of Microsoft Fabric, a new storage mode to connect from Power BI to data in OneLake has been introduced. Direct Lake it makes to possible to use your data from OneLake in Power BI without t…

A guest post by Štěpán Rešl describing an approach to automated collection of Usage Metrics across many workspaces, without using the Admin APIs. This is highlighting a free solution that Štěpán has created to automate this workflow.

We are thrilled to announce the use of LCA

In this blog article, we will explore four end-to-end scenarios that are typical paths our customers take to extract value and insights from their data using Microsoft Fabric. In these scenarios, we will show you how Fabric seamlessly addresses common architectural scenarios with a single service that encompasses data ingestion, transformation, storage, and exposure through reports or SQL endpoints.

Microsoft unveiled Microsoft Fabric last week and one of the questions I’m getting is, how does Databricks integrate with Microsoft Fabric…

An alternative way to import SharePoint list data into Power BI.

Learn more about delta lakes and how thwy work from Gerhard.

Discussions with clients regarding the performance of their data models in Power BI have been a frequent occurrence for me recently. The issues often involve failed refreshes, queries that run slow…

Genie Framework improves Spark Pool utilization by executing multiple Synapse notebooks on the same spark pool instance. Uses vaccum and optimize to maintain efficient delta tables.

Using optimize and vaccum to removed unused delta patition files. Helps maintain a minimized delta table size footprint.

A tool for customizing and dynamically generating a tabular model from an existing tabular model. - m-kovalsky/MiniModelBuilder: A tool for customizing and dynamically generating a tabular model fr...

Helpful intro for Windowing Functions