Below is a method to use in ADF’s Mapping Data Flows to reduce the data stream in your data flows to only include distinct rows. This sample is available as a pipeline template here. Choose y…
Azure Data Factory Data Flow: Building Slowly Changing Dimensions
Here is a quick walk-through on how to use Azure Data Factory’s new Data Flow feature (limited preview) to build Slowly Changing Dimension (SCD) ETL patterns. The Data Flow feature in ADF is …
ADF Slowly Changing Dimension Type 2 with Mapping Data Flows (complete)
I have been putting together a series of posts and videos around building SCD Type 1 and Type 2 using Mapping Data Flows with Azure Data Factory. In this latest post, I’m going to walk throug…
Azure Data Factory Activities: 5 Comprehensive Aspects
This blog discusses Azure Data Factory Activities and explains the steps required to work with them. Read along to learn more about Azure Data Pipelines.
What is Azure Data Factory and how can you use it? In this visual guide to ADF, we break down the what, why, and how of using ADF for data integration.
What is Azure Data Factory? A beginner's guide to ADF
Get a beginner-friendly deep dive into what Azure Data Factory (ADF) is and how to use this Microsoft serverless data integration solution. Let's dig in!
Ingest and Explore LEGO Datasets using Pipelines and Serverless SQL in Azure Synapse Analytics | Cathrine Wilhelmsen
In this post, we use Azure Synapse Analytics pipelines to ingest LEGO datasets into our data lake and then explore those datasets using Serverless SQL.
Creating Dynamic LinkedIn Share Links with Power Automate
In this video, Devin wraps up this 3 part Power Automate video series by showing you how to make dynamics Linkedin share links so you can encourage others to celebrate their new skills with others!…
Discover how Databricks SQL is powering a new generation of analytics and data applications, running directly on the data lake, at the world’s leading companies.
I recently worked on a project where we needed to copy some large files from a specified library in SharePoint Online. In that library, there were several layers of folders and many different types…
Creating a Lake Database using the Database Designer – Serverless SQL
Synapse Analytics allows the creation of External Tables and Views within a Serverless SQL Pools database and also a Spark database. These objects can be shared between Spark and Serverless using shared metadata. An example is an External Table created using a Spark Pool that can be queried by using a Serverless SQL Pool. During...
Configure Postman to be easily used with any Azure AD protected API (SharePoint, Graph, custom etc.)
As a developer, how many times have you had a need to test different Microsoft 365 APIs? For me, this is a fairly frequent task. Most often it's SharePoint REST API or MS Graph. Every time you should think about the authentication part because all of... [Click to read more]
Extract Data from PDFs at Scale with Form Recognizer
In this article, Dr. Tom Weinandy demonstrates a set of tools that can extract, compile, and visualize data from large swaths of intractable files - proving that even data on a PDF can have a second life.
data.world's cloud-native data catalog makes it easy for everyone—not just the "data people"—to get clear, accurate, fast answers to any business question.
Enterprise Data Catalog Software is Essential for Data Driven Businesses. Learn How Alation's Data Catalog Capabilities Boost Efficiency & Drive Decision Making.