In this episode of “How do I create a date table from nothing in my chosen platform”? I am covering Databricks. No lengthy life story of what bubble tea I was drinking when creating this or childhood memories like some food bloggers. The code is linked below, look at the comments in the notebook if …
Date Table Generation Notebook for Databricks Unity Catalog Read More »
Project Management Like A Boss with Notion There are many project management tools out there, but for small companies or start-ups they are all either too expensive or too complicated for your needs, or are just plain bad. Notion can serve as a project management / company management tool for you if you have basic …
In Snowflake, you can access files external to the Snowflake instance using something called a STAGE. A STAGE is a named object that points to a location such as AWS S3, Azure Storage Account or Google Cloud Storage Buckets, and contains the required authentication credentials to access the files in that storage location. To programmatically …
Persist the List of Files in an External Stage in Snowflake Read More »
If you want a project to be successful and deliver what is required on time and within budget, your team has to follow the correct processes using adequate technology in order to track the progress of the project and ensure you have visibility on the status of all tasks and dependencies. There are many tools …
Notion Agile Project Management Kanban Board Template Read More »
There are many situations where you need to identify the iteration of a weekday within a given month. Most commonly though is for the identification of public holidays e.g., the 4th Thursday in November – Thanksgiving in USA. There are no built-in methods to get this information in any SQL-based platform I have used thus …
Get the Iteration of a Weekday in a Month on a Virtual Calendar Read More »
I have been searching for the perfect tool to manage my tasks for my personal life and work-life for years. I tried so many tools – Microsoft ToDo, TickTick, Any.do, Google Tasks etc. I had to move on from all of them eventually because they could never give me what I needed. I have been …
What is dynamic SQL? Dynamic SQL is a programming technique where you write a general purpose query and store it in a string variable, then alter key words in the string at runtime to alter the type of actions it will perform, the data it will return or the objects it will perform these actions …
In any data warehousing or analytics solution, you will eventually need to generate a calendar table in order to perform time intelligence analytics. One key aspect that most businesses will require is to view KPIs by working days only i.e. days when the business was operating normally. Each business will have their own unique rule sets …
What does it mean to mount a storage account to Azure Databricks Databricks has a built in “Databricks File System (DBFS)”. It is a distributed file system mounted onto your Databricks workspace. It is mounted directly to your cluster and is only accessible while the cluster is running. It is an abstraction on object storage. …
Function to mount a storage account container to Azure Databricks Read More »
There are many ways to orchestrate a data flow in the cloud. One such option is to have an independent process pull data from source systems and land the latest batch of data in an Azure Data Lake as a single file. The next layer where you process the data can be handled in many …
Get the latest file from Azure Data Lake in Databricks Read More »
