- Azure Synapse Analytics Custom Rolesby AndyRecently I had a challenge in terms of providing a least permissions possible security model for a Data Engineer persona … Read more
- Future Data Driven 2023 – Fabric SQLby AndySession Abstract It all started with Parallel Data Warehouse, then came Azure SQL Data Warehouse…then Dedicated SQL Pools in Synapse! … Read more
- Azure Synapse Analytics Managed Virtual Networks for the Developerby AndyIt’s recommended in a Synapse environment, especially a production/live environment, to lock a Synapse Analytics workspace down behind a Virtual … Read more
- Synapse Link for Dataverse with Azure Synapse Analytics eBookby AndyIf you’re interested in using Synapse Link for Dataverse with a Dynamics 365 environment to export data for analysis in … Read more
- Transactions and Isolation Levels in Fabric Warehouseby AndyI trust everyone has been having fun with Fabric over the last few weeks, diving into various workload types like … Read more
- Temp Tables in Fabric Lakehouse SQL Endpoint and Warehouseby AndyThis blog has been adapted from an earlier blog about using temp tables in Synapse Serverless SQL Pools. As the … Read more
- Play Around: Deploy Database Projects to Fabric Warehouseby AndyI always like to have a dig around, try a few things and see if something works (most likely unsupported!) … Read more
- Fabric in Productionby AndyThis is a placeholder blog which will be revisted many, many times over the coming months. This is a list … Read more
- Getting Started with Microsoft Fabric: Enabling 60-Day Trial and Analysing Databy AndyThere’s been an exciting new announcement at Microsoft Build in the form of Microsoft Fabric, a unified platform that brings … Read more
- Synapse Link for Dataverse: Delta Lake Export Configurationby AndyWelcome to part 2 in the Synapse Link for Dataverse Delta Lake series. In the recently released functionality in Synapse … Read more
- Synapse Link for Dataverse: Exporting to Delta Lakeby AndyMicrosoft recently released the ability to configure Synapse Link for Dataverse and use Delta Lake as the export format. As … Read more
- Parallel Loading of Tables with Power BI & Azure Synapse Analyticsby AndyIn this blog I’m continuing my series about using Power BI and Azure Synapse Analytics together and the features between … Read more
- What Does A “Conformed Dimension” Actually Do?by AndyEvery so often the “star schema is dead! dimensional modelling is dead!” discussion rolls around again, and we end up … Read more
- Azure Synapse Serverless SQL Pools Cheat Sheetby AndyHere’s an Azure Synapse Serverless SQL Pools Cheat Sheet I created to help anyone reference any objects or syntax they … Read more
- Why Cloud Data Platforms?by AndyAlways ask “why.” I really couldn’t think of a better blog title, I usually do technical stuff and the title … Read more
- Understanding ResultSet Caching in Dedicated SQL Pools with Power BIby AndyWelcome to this new blog series in which I’ll be diving into both Power BI and Azure Synapse Analytics features … Read more
- Mastering DP-500 Exam: Upcoming Changes on February 6thby AndySome of the skills measured for the DP-500 Azure Enterprise Data Analyst Associate certification are changing on February 6th 2023 … Read more
- Partition Pruning with Delta Tables in Azure Synapse Analyticsby AndyA question that is often asked about Azure Synapse Analytics is how it handles Delta tables that are partitioned. Delta … Read more
- Synapse Link for Dataverse: Video Seriesby AndyI’ve been diving into Synapse Link for Dataverse recently for a project and created a series of videos going through … Read more
- Azure Analysis Services Migration to Power BI Premiumby AndyAround November 2022, Microsoft announced a new feature within Power BI which allows you to migrate Tabular models from Azure … Read more
- Azure Synapse Link for Dataverse: Understanding Advanced Configuration Settingsby AndyIn the first part of this blog series we looked at how to configure Synapse Link for Dataverse with Dynamics … Read more
- Azure Synapse Analytics in 2022: Retrospectiveby AndyRetrospective Another year has rolled by (where does the time go…) and another year in the life of Azure Synapse … Read more
- Azure Synapse Analytics at SQLBit 2023by AndyThe General Session agenda for SQLBits 2023 is now available here and covers the agenda on Thursday, Friday, and Saturday. … Read more
- Mastering DP-500 Exam: Configuring Synapse Analytics Source Control with Azure DevOpsby AndyThis blog forms part of the DP-500 blog series on this site here and also from Data Mozart’s blog here. … Read more
- Databricks Data Engineer Associate Certification Overviewby AndyI’ve been looking at taking the Databricks Data Engineer Associate Certification (the Databricks page for the certification is here) as … Read more
- Azure Synapse Link for Dataverse: Getting Started with Dynamics 365 Sales and Synapse Analyticsby AndyPart of the Azure Synapse Analytics suite of services is Synapse Link. This enables synchronization of data from sources including … Read more
- Connecting Power BI to Azure SQL Database using Private Endpointsby AndyThis was originally posted on my Datahai company blog in November 2021. This blog has now been updated as of … Read more
- Mastering DP-500 Exam: Querying JSON Data in Serverless SQL Poolsby AndyThis blog forms part of the DP-500 blog series on this site here and also from Data Mozart’s blog here. … Read more
- Dedicated SQL Pools DMVs: Show Name of Synapse Studio Script Being Executedby AndyI’m currently in the process of building a Query and Workload Analyser Power BI report which queries several DMVs (Dynamic … Read more
- User Permissions in Serverless SQL Pools: External Tables vs Viewsby AndyIn a recent Synapse Analytics Serverless SQL Pools QnA session, the Serverless SQL Pools team stated that using External Tables … Read more
- Up and Running: How to make your Azure Synapse Analytics Proof of Concept a Success (Data Relay Session)by AndyFirstly, I’d like to say a huge thank you to the Data Relay team, the helpers, the sponsors, the attendees, … Read more
- Elastic Queries with Azure SQL Database and Synapse Analytics Serverless SQL Poolsby AndyRecently I have been exploring an option to use an Azure SQL Database for a small footprint Data Warehouse and … Read more
- Use Azure Log Analytics to analyse Azure Data Lake Gen2 activity from Serverless SQL Poolsby AndyThis blog is marked as intermediate knowledge as an understanding of Serverless SQL Pools is preferable, but not essential. Refer to Getting Started … Read more
- Use Azure Storage Logs to analyse Synapse Analytics Serverless SQL Pools Activityby AndyThis blog is marked as intermediate knowledge as an understanding of Serverless SQL Pools is preferable, but not essential. Refer … Read more
- PASS Summit Recording Available: Creating a Logical Data Warehouse using Azure Synapse Analytics Serverless SQL Poolsby AndyMy PASS Data Community Summit 2021 recording is now available, please click here for the recording. I’ve re-watched the session … Read more
- How does Serverless SQL Pools deal with different file schemas? Part 3 – Deltaby AndyWelcome to the 3rd and final part in this blog series in which we’re looking at how Azure Synapse Analytics … Read more
- How does Serverless SQL Pools deal with different file schemas? Part 2 – Parquetby AndyWelcome to part 2 in this blog series in which we’re looking at what happens when schema changes are made … Read more
- Lake Databases And The Importance of Data Typesby AndyI encountered an issue with Azure Synapse Analytics Lake Databases and I’d like to share the problem and the solution. … Read more
- Mastering DP-500 Exam: Using the Data Map in Microsoft Purviewby AndyPart of the upcoming Azure Enterprise Data Analyst Associate exam, the DP-500, is to understand how to identify data sources … Read more
- Mastering DP-500 Exam: Integrating Microsoft Purview with Power BIby AndyPart of the upcoming Azure Enterprise Data Analyst Associate exam, the DP-500, is to understand how Microsoft Purview integrates with … Read more
- Mastering DP-500 Exam: Querying Partitioned Sources in Azure Storageby AndyThis blog forms part of the Mastering DP-500 exam series. A list of available blogs is here and is in … Read more
- Mastering DP-500 Exam: Using Synapse Analytics Serverless SQL Pools to query Azure Storageby AndyThis blog forms part of the Mastering DP-500 exam series. A list of available blogs is here and is in … Read more
- Mastering DP-500 Exam: Blog Resourcesby AndyI am collaborating with Data Mozart on blogging around the upcoming Microsoft Certified: Azure Enterprise Data Analyst Associate certification. Please … Read more
- New Certification: Microsoft Certified Azure Enterprise Data Analyst Associateby AndyMicrosoft has created a new certification, the Microsoft Certified Azure Enterprise Data Analyst Associate (link here). There is a single … Read more
- How does Serverless SQL Pools deal with different file schemas? Part 1 – CSVby AndyServerless SQL Pools offers a straight-forward method of querying data including CSV, JSON, and Parquet format stored in Azure Storage. … Read more
- Integrating Power BI & Azure Synapse Analytics – Power BI UK User Groupby AndyHere is the link to the slides for the Power BI UK User Group session on Thursday 17th March, 2022 … Read more
- SQLBits 2022 – The Dream Team: Synapse Analytics Serverless SQL Pools and Pipelinesby AndyI presented a 20 minute session at SQLBits 2022 in which I talked about how useful Synapse Pipelines can be … Read more
- Trigger The Process – Using CETAS in Serverless SQL Pools to Generate Pre-Calculated Datasets – Part 2 of 2by AndyIn part one we looked at a process in which a pre-aggregated dataset can be created by Azure Synapse Analytics … Read more
- Using CETAS in Serverless SQL Pools to Generate Pre-Calculated Datasets – Part 1 of 2by AndyDedicated SQL Pools have a feature called Materialized Views which can generate a pre-calculated/processed dataset ready for consumption and is … Read more
- DataMinutes #2 Session Companion for Turbocharging Power BI using Synapse Analyticsby AndyThe slides for the DataMinutes #2 session Turbocharging Power BI using Synapse Analytics is available here to download. GitHub Code … Read more
- SQLBits Session – The Dream Team: Synapse Analytics Serverless SQL Pools and Pipelinesby AndySQLBits is back this year and is running from Tuesday 8th to Sunday 12th March at the ExCel in London. … Read more
- DataMinutes #2 Conference – Azure Synapse Analytics Sessionsby AndyThe next DataMinutes conference (#2) is taking place on Friday 21st January. What’s different about DataMinutes is that it is … Read more
- Power BI Hybrid Tables with Synapse Analytics Serverless SQL Poolsby AndyIn this video we’re talking a look at the new Power BI Hybrid Tables feature which combines Import, Incremental Refresh, … Read more
- View Generation in Serverless SQL Poolsby AndyOne of the objects that can be created within a Serverless SQL Pools database is a View. A View, along … Read more
- Testing Error Handling in Serverless SQL Poolsby AndyA recent feature to be released for Serverless SQL Pools is error handling when processing external data (please see the … Read more
- Data Toboggan Azure Synapse Analytics Event – Call For Speakersby AndyData Toboggan is a community-driven conference focused on Azure Synapse Analytics, Microsoft’s flagship data warehousing, big data, and analytics service. … Read more
- PASS Data Community Summit Companion Resourcesby AndyPASS Data Community Summit 2021 Slides Please click here for a PDF copy of the slides. Polaris (Serverless SQL Pools … Read more
- Creating a Lake Database using the Database Designerby AndySynapse Analytics allows the creation of External Tables and Views within a Serverless SQL Pools database and also a Spark … Read more
- Azure Synapse Analytics in the Azure Architecture Centreby AndyMicrosoft provide an Azure Architecture Centre which contains resources to understand which Azure services are appropriate in certain scenarios and … Read more
- Using Access Control Lists To Manage Fine-Grain Data Lake Permissionsby AndyWhen data is stored in Azure Data Lake Gen2, we can use Azure Active Directory and Access Control Lists (ACLs) … Read more
- Integrating Azure Purview with Synapse Analytics Serverless SQL Poolsby AndyAzure Purview is a Microsoft Data Governance service which enables an organisation to map their data infrastructure including data lineage … Read more
- Using Temporary Tables within Serverless SQL Poolsby AndyTemporary (temp) tables have been a feature of Microsoft SQL Server (and other database systems) for many years. Temp tables … Read more
- External Tables vs Views…Which to Use and Why?by AndyServerless SQL Pools do not contain data (unlike Dedicated SQL Pools) but rather allow you to cast structure over data … Read more
- Azure Storage Folder Filtering using the Filepath Functionby AndyServerless SQL Pools includes 2 SQL functions, filepath and filename, that can be used to return the folder path/name and … Read more
- 10 Billion Rows: Parquet File Size and Distribution When using CETASby AndyWhen using Serverless SQL Pools to write data to Azure Storage/Data Lake Gen2 using the CREATE EXTERNAL TABLE AS SELECT … Read more
- Creating a Logical Data Warehouse with Synapse Serverless SQL: Part 4 of 4 – Connecting Power BI to the Dimensional Modelby AndyWelcome to Part 4 in the Creating a Logical Data Warehouse with Synapse Serverless SQL Pools series. We are now … Read more
- Creating a Logical Data Warehouse with Synapse Serverless SQL: Part 3 of 4 – Incremental Fact Loading and Slowly Changing Dimensionsby AndyWelcome to part 3 of the Logical Data Warehouse with Serverless SQL Pools series. In this part we’ll be covering … Read more
- Creating a Logical Data Warehouse with Synapse Serverless SQL: Part 2 of 4 – Creating a Dimensional Modelby AndyWelcome to part 2 of this 4 part blog series on creating a Logical Data Warehouse with Azure Synapse Analytics … Read more
- Creating a Logical Data Warehouse with Synapse Serverless SQL: Part 1 of 4 – Setting Up and Querying Source Databy AndyWelcome to part one of a four part blog series where we’ll be looking at one of the main scenarios … Read more
- Azure Synapse Analytics Serverless SQL Pools Learning Resourcesby AndyWelcome to this collection of links to resources focussed on Azure Synapse Analytics Serverless SQL Pools. It includes official Microsoft … Read more
- Azure Synapse Analytics Serverless SQL Best Practicesby AndyMicrosoft have a best practices document available here which highlights certain actions that can be taken to optimise use of … Read more
- Getting Started with Azure Synapse Analytics SQL Serverlessby AndyThe new Azure Synapse Analytics SQL Serverless service enables querying file-based data stored in an Azure Data Lake Gen2 storage … Read more
- Harnessing Azure Synapse Analytics SQL Serverless in Power BI Dataflowsby AndyPower BI Dataflows are a Power BI Service component that allow the creation of centralised, cleansed, transformed and certified tabular … Read more
- Serverless SQL Pools Managed Private Endpointby AndySteps Created new Synapse Workspace and selected Managed VNet option Opened Synapse Studio and clicked Manage > Manage Private Endpoints … Read more
- Why the Star Schema is better than One Big Tableby AndyIntroduction In the ever-evolving landscape of data warehousing and business intelligence, data modeling approaches are central to the success of … Read more
- Why One Big Table is better than the Star Schemaby AndyIntroduction In the realm of data warehousing and business intelligence, architects and data professionals are often faced with the crucial … Read more