Document & Diagram Downloads:
Checklist for Finalizing a Data Model in Power BI Desktop (V2 as of 12/27/2017)
Data Lake Zones (V2 as of 4/8/2018)
Virtualized Queries in Azure (V1 as of 4/8/2018)
Power BI End-to-End Features <--A new version is currently in progress-available soon
Presentation Info & Slides:
Designing Modern Data and Analytics Solutions in Azure (Full-Day Session)
Co-presented with Meagan Longoria
This full-day session will focus on principles and practices for architecting modern analytics/BI/DW systems in Azure. We will discuss Azure fundamentals, implementation strategies, key decision points, and lessons learned from customer projects. Cloud design patterns will be explored, including cloud-specific concerns and considerations, as well as key differences from traditional on-premises deployments. Reference architectures will be presented which address scenarios such as real-time data ingestion vs. batch loads, data virtualization vs. data integration, schema on read vs. schema on write, SQL on Hadoop, high data volumes, varying file formats, enabling data science, and facilitating self-service BI. After considering various reference architectures, we will proceed with building an end-to-end solution for one reference architecture based on requirements presented to the audience.
Approximately 30% of the day will be hands-on labs, 50% presentation, and 20% open discussion and questions. Specific technologies discussed will include: Azure SQL Database, Azure SQL Data Warehouse, Azure Data Lake Store, Azure Data Lake Analytics, U-SQL, Azure Storage, Azure Data Factory, Azure Databricks, Spark, Hive, HDInsight, Azure Analysis Services, PolyBase, Elastic Queries, Azure Event Hub, Azure Stream Analytics, Azure Data Catalog, Machine Learning Services, Azure Machine Learning, Azure Cognitive Services, Power BI, BIML, ARM templates, PowerShell, and Azure Virtual Machines. Attendees of this session will gain a broad understanding of the fundamentals for designing data solutions in Azure, techniques for navigating the wide variety of platform choices in Azure, and suggestions for developing sound architectural systems.
- Explore reference architectures and cloud design patterns for building analytics/BI/DW systems.
- Share lessons learned, customer project stories, and implementation strategies.
- Provide scripts and instruction to gain hands-on experience setting up a cloud analytics solution in Azure.
Level: 200 (Intermediate)
Target Audience: The ideal audience member has some experience as a data engineer, BI professional, or database developer, and is in the early stages of migrating or building solutions in Azure.
Prerequisites: Familiarity with developing BI/analytics systems will be very helpful, but is not required. No Azure experience is required.
- PASS Summit 2018, Seattle, WA - November 5, 2018
Designing Azure Data and Analytics Solutions (Full-Day Session)
Co-presented with Meagan Longoria
This full-day session will walk through building modern BI/analytics solutions in Azure. We’ll discuss Azure fundamentals and then dive in to implementation strategies, architecture and development decisions, and lessons learned from previous projects. With so many different services available in Azure, it can be overwhelming to decide which ones to use. We’ll explain reference architectures and design patterns for scenarios such as real-time data ingestion vs. batch loads, data virtualization vs. data integration, schema on read vs. schema on write, SQL on Hadoop, high data volumes, varying file formats, enabling data science, and facilitating self-service BI. We’ll also discuss hybrid environments where some components are on-premises while others are in Azure. Once we are familiar with the reference architectures, we’ll will get hands on and build an end-to-end solution for one scenario. Attendees of this session will gain a broad understanding of the fundamentals for designing data solutions in Azure, techniques for navigating the wide variety of platform choices in Azure, and suggestions for developing sound architectural systems.
- SQL Trail, Richmond, VA - October 12, 2018
Azure Data Lake: What, Why, and How
We will explore the capabilities of Azure Data Lake (ADL), its use cases, as well as when to implement ADL as part of your data architecture. Options for integration of the data lake with SQL Server, Azure SQL DW, Azure SQL DB, and Azure Blob Storage will be discussed, as well as the roles of U-SQL and PolyBase in a multi-platform system. You will leave this session with an understanding of the benefits, challenges, and suggestions for getting started with Azure Data Lake technologies.
- 24 Hours of PASS Summit Preview 2018 - June 12, 2018
- Triad SQL BI User Group, Greensboro, NC - June 26, 2018
- Charlotte BI Group, Charlotte, NC - August 7, 2018
- PASS Summit, Seattle, WA - November 6-9, 2018
- SQL Saturday, Washington DC - December 9, 2017
Tips for Getting Started with the Azure Data Platform
This session is packed with practical tips and lessons learned about using Azure as a database platform. You will learn the fundamentals about how Azure is structured to help you make architectural decisions. Ideas will be shared for planning resource groups, naming conventions, and the separation of Dev, Test, and Prod. We will discuss database platform options, data storage options, and why PowerShell and ARM are so important to deployment scenarios. Cost-saving techniques and cloud efficiencies will be discussed as well.
Recording: Tips for Getting Started with Azure from PASS Summit 2017 (1hr 15 min) - recorded Oct 2017
Target Audience: Database developers and DBAs who are looking for a primer on the Azure platform
Selecting a Data Warehousing Technology in Azure
There are numerous choices in the Azure platform to implement a data warehouse for supporting analytical, big data, and business intelligence workloads. In this session we will talk through reference architectures for common scenarios, beginning with relational choices for traditional data warehousing, progressing to non-relational and composite architectures to support modern data warehousing and analytical environments. We will bring clarity to when Azure SQL Data Warehouse really is the best choice, versus when another Azure service may be a more suitable solution. Practical suggestions to inform your decision-making process will be shared throughout the session.
Level: Intermediate (some exposure to Azure concepts is beneficial for attendees, but not required)
Recording: Selecting a Data Warehousing Technology in Azure (56 minutes) - recorded Jan 2018
Target Audience: Technologists who are looking to understand data platform choices in Azure for DW workloads.
- BlueGranite webinar, January 31, 2018
Architecting a Data Lake (full day workshop)
This full-day session will focus on principles for designing and implementing a data lake. There will be a mix of concepts, lessons learned, and technical implementation details. This session is approximately 70% demonstrations: we will create a data lake, populate it, organize it, query it, and integrate it with a relational database via logical constructs. You will leave this session with an understanding of the benefits and challenges of a multi-platform analytics/DW/BI environment, as well as recommendations for how to get started.
Target audience: Technologists who are considering or beginning a data lake implementation. No data lake experience is required. Familiarity with a relational database such as SQL Server is helpful, as some of the scenarios discussed will focus on integrating a data lake with a relational data warehouse.
You will learn in this session:
Scenarios and use cases for expanding an analytics/DW/BI environment into a multi-platform environment which includes a data lake
Methods for planning & organizing a data lake which focuses on optimal data retrieval and data security
Determining when to use Azure Data Lake Analytics (U-SQL) vs. HDInsight vs. Azure Databricks vs. relational functionality for data processing
Deciding between Azure Blob Storage vs. Azure Data Lake Store vs. a relational platform for data storage
Use cases and syntax basics for U-SQL, PolyBase, and elastic queries
Benefits and challenges of schema-on-read vs. schema-on-write approaches for data integration and on-demand querying needs
Specific technologies discussed and/or demonstrated in this session include:
Azure Data Lake Store | Azure Data Lake Analytics | HDInsight | Azure Databricks | U-SQL |
Azure SQL Data Warehouse | PolyBase | Azure SQL Database | Elastic Queries | Azure Storage
If you have an Azure account and your own laptop, you will be able to follow along during the demonstrations if you'd like. Demo scripts will be provided with the workshop materials.
Designing a Modern Data Warehouse + Data Lake
Join us for a discussion of strategies and architecture options for implementing a modern data warehousing environment. We will explore advantages of augmenting an existing data warehouse investment with a data lake, and ideas for organizing the data lake for optimal data retrieval. We will also look at situations when federated queries are appropriate for employing data virtualization, and how federated queries work with SQL Server, Azure SQL DB, Azure SQL DW, Azure Data Lake, and/or Azure Blob Storage.
Level: This is an intermediate session suitable for attendees who are familiar with data warehousing fundamentals.
Slides: Designing a Modern DW + Data Lake <--Slides last updated March 2017
Fundamentals of Designing a Data Warehouse
In this session we will review sensible techniques for developing a data warehousing environment which is relevant, agile, and extensible. We will cover practical dimensional modeling fundamentals and design patterns, along with when to use techniques such as partitioning or clustered columnstore indexes in SQL Server. We'll also review tips for using a database project in SQL Server Data Tools (SSDT) effectively. The session will conclude with tips for planning the future growth of your data warehouse.
Level: This is an introductory session best suited to attendees who are new to data warehousing concepts.
Slides: Fundamentals of Designing a DW <--Slides last updated February 2017
Note that older archives with outdated information and/or older technologies have been removed from this archive.
The What, Why, and How of Collecting Telemetry Data
To better understand SentryOne usage patterns and deliver maximum value to our customers, release 11.2 now anonymously collects telemetry data on an opt-in basis. In this session, we will show actual examples of telemetry data collected, as well as an overview of the technical implementation to send, ingest, store, and analyze this data. We will also share key observations so far from the data.
Target Audience: SentryOne customers
- PASS Summit, Seattle, WA - October 31, 2017
Building Blocks of Cortana Intelligence Suite in Azure
Join us for a practical look at the components of Cortana Intelligence Suite for information management, data storage, analytics, and visualization. Purpose, capabilities, and use cases for each component of the suite will be discussed. If you are a technology professional who is involved with delivering business intelligence, analytics, data warehousing, or big data utilizing Azure services, this technical overview will help you gain familiarity with the components of Cortana Intelligence Suite and its potential for delivering value.
Level: A fast-moving introductory session
Target Audience: Technology professionals seeking to gain a high level understanding of the capabilities of the Cortana Intelligence Suite
Slides: Building Blocks of Cortana Intelligence Suite <--Slides last updated April 2017
- Azure Bootcamp, Charlotte, NC - April 22, 2017
- SQLBits 16, Telford, England - April 8, 2017
- PASS Cloud Virtual Chapter - Sept 28, 2016
- SQL Saturday, Charlotte, NC - Sept 17, 2016
- SQL Saturday, Spartanburg, SC - Aug 20, 2016
- Hampton Roads SQL Server User Group, Virginia Beach, VA - July 20, 2016
- Charlotte Microsoft Cloud Meetup Group, Charlotte, NC - July 14, 2016
- Carolina IT Professionals Group (CITPG), Charlotte, NC - June 20, 2016
- Charlotte BI Group (CBIG), Charlotte, NC - June 7, 2016
- SQL Saturday, Atlanta, GA - May 21, 2016
Tales from Building a SQL Server Data Warehouse in Azure
In this session, we share our experiences and lessons learned from a recent migration to Azure for a SQL Server data warehousing environment. We begin with sharing our reasoning for IaaS vs. PaaS, our carefully-selected naming conventions, and how we structured development, test, and production within subscriptions and resource groups. We cover the what, why, and how for decisions around storage, encryption, and backups. Finally, the session wraps up with a brief discussion of the use of Azure Resource Manager (ARM) templates and PowerShell, as well as techniques for monitoring the environment in Azure.
Level: A fast-moving introductory session
Slides: Tales from Building a SQL Server DW in Azure <--Slides last updated August 2017
Target Audience: Technology professionals responsible for creating and managing resources in Azure