automation

Dynamic SQL in Databricks and SQL Server

What is dynamic SQL? Dynamic SQL is a programming technique where you write a general purpose query and store it in a string variable, then alter key words in the string at runtime to alter the type of actions it will perform, the data it will return or the objects it will perform these actions …

Dynamic SQL in Databricks and SQL Server Read More »

Dynamic Datasets in Azure Data Factory

In a previous post linked at the bottom, I showed how you can setup global parameters in your Data Factory that is accessible from any pipeline at run time. This post will show you how you can leverage global parameters to minimize the number of datasets you need to create. Specifically, I will show how …

Dynamic Datasets in Azure Data Factory Read More »

Global Parameters 101 in Azure Data Factory

What the heck are they? Global Parameters are fixed values across the entire Data Factory and can be referenced in a pipeline at execution time. They have many applications e.g. when multiple pipelines require identical parameters and values at run time and you don’t want to duplicate them in variables across said pipelines. When you utilize …

Global Parameters 101 in Azure Data Factory Read More »

Dynamic External Tables in Azure Synapse Analytics On-Demand

What is an External Table? This article will focus on the Synapse Analytics implementation of External Tables.However, note that there are other flavours of external tables and they behave slightly differently depending on which product you are using to defined it. SQL Server SQL Database Azure Synapse Analytics Analytics Platform System (PDW) External Tables in …

Dynamic External Tables in Azure Synapse Analytics On-Demand Read More »