Governance & Administration - Tenant Settings: Visual Cues

With all the Fabric announcements in the last months, some of the Admin announcements might have slipped through. As you might know, the Admin part of Fabric is dear to my heart. I've posted about it earlier here, here, here, and here, to name a few 😀.

So in the next weeks I am going to highlight a few things with short, informative posts.

You can find other posts in the series here:

• Fabric (Tenant Settings) API's
• Tenant Settings: Searching
  

Governance & Administration - Fabric (Tenant Settings) API's

With all the Fabric announcements in the last months, some of the Admin announcements might have slipped through. As you might know, the Admin part of Fabric is dear to my heart. I've posted about it earlier here, here, here, and here, to name a few 😀.

So in the next weeks I am going to highlight a few things with short, informative posts.

This first post in the series is focused on the Tenant settings API and other Fabric API's available at the time of writing.

You can find other posts in the series here:

• Tenant Settings: Visual Cues
• Tenant Settings: Searching

Get Tenant Settings API

Obviously, to use the Get Tenant Settings API you need to have at least Tenant.Read.All permissions, or have the Fabric Administrator role (or higher) in Azure.

There are a few use cases I see for getting these settings exported with this API:

• Documentation purposes when you have multiple Fabric Administrators
• Distributing tenant settings to users, and explaining why we (as a team of Administrators/within the Center of Excellence) made certain choices
• Get notified of the changes in the tenant settings, without having to use Microsoft Defender or M365 Security & Compliance center like mentioned here

Get Capacities Tenant Settings Overrides

With the introduction of Fabric, there has also been the ability to have capacity admins override certain tenant settings. 

Example of a tenant setting that can be delegated

Some settings in the Admin portal are automatically delegated to capacity admins, like shown above. 

Other Fabric API's

Next to the API's on the tenant settings, there are already a lot more API's in the Admin section for you to check out, for example to handle workspaces, users, domains, and items as a Fabric Admin.

Next to that, so not as a Fabric Admin, you can also interact with Lakehouse tables, and there's a bunch of Core API's, for example to use Git or OneLake Shortcuts.

Preview note

Please note that at the time of writing this post, most of the API's are still in preview and subject to change.

Except for the Get Tenant Settings API, all the API's are currently in preview:

The Conference Season Has Begun!

The conference season is kicking off again! I have quite some events and sessions already lined up for the next months, so I though I'll update you here.

First up is Data Community Day Austria, in Vienna. I'm actually at Schiphol Airport now, waiting to board my plane 😀 I'm presenting a new session on Direct Lake in Microsoft Fabric.

In March, 7-9, the Dutch Power BI Gebruikersdagen are in Utrecht and I'm presenting a session on Saturday: How to use Power BI External Tools.

After that is SQLBits, which I'm skipping this year. MVP Summit is unfortunately also online for me this year, but that has a very good reason...

Because, in the last week of March the Microsoft Fabric Community Conference is happening in Las Vegas, and I'm delighted and excited to say I am presenting 2 sessions there!
The first session is with Ulrich Christ, Program Manager Azure Data Factory at Microsoft, and we are going to talk about Connecting to SAP Data with Data Factory in Microsoft Fabric.
Because the second session is still in the works I can't share any details yet, so you'll just have to wait a bit longer for that.

After that, there's also DataGrillen in Lingen, Germany on May 16 & 17. I'm presenting a session dear to my heart: Fabric Admin in a Day. Do check the amazing 2-day schedule on their site. It's also happening during the Kievelinge Fest, which happens the weekend right after DataGrillen, how convenient... 😁

I'm very excited for the coming months and hope to see many of you there, meeting up with old friends and making new ones! 💛

Ignite News: Microsoft Fabric is Now GA (and more)!

Over the (roughly) last 6 months we all had the ability to play with Microsoft Fabric, when it went Public Preview at Microsoft Build.

Now that the keynote of Microsoft Ignite by CEO Satya Nadella has been delivered, and also the amazing in-depth Fabric session by Amir Netz, Arun Alagaratchagan: Make your data AI ready with Microsoft Fabric and Azure Databricks. Back then at Build, Satya called it:

"...the biggest launch of a data product from Microsoft since the launch of SQL Server!"

Generally available

But now, Satya went one step further and announced GENERAL AVAILABITIY of Fabric!

Also, Copilot in Fabric is now in public preview.

More updates

There's a ton of updates inside Fabric on existing features on the latest Fabric blog here.

And there are also some exciting new features:

• Seamlessly connect your data sources to Fabric
  The ability to create shortcuts was already there, where you virtualize data in OneLake without having to move or duplicate that data.. You can create shortcuts to another Lakehouse and Warehouse, but also to files on ADLS or even Amazon S3 or Google storage.
  The newest feature just announced is called Mirroring, where you can add and manage existing cloud data warehouses (and databases) in Fabric's Data Warehouse experience. The way this works, is like replication in SQL Server. Fabric replicates a snapshot of that database to OneLake in Delta Parquet files and keeps that in sync in near real time, which relies on the Change Data Capture feature of the underlying source. Initially it's supported for Azure Cosmos DB, Azure SQL DB and Snowflake, more sources will follow next year.
• Copilot in Power BI (public preview)
  Just be aware that Copilot will be rolling out in stages, 
• Smart Narrative is an existing visual in Power BI Desktop, now rebranded to Narrative with Copilot
• The November Desktop update let's you generate synonyms for your fields, measures and tables using Copilot.
• In the future, there's also going to be:
• a report creation experience
• a DAX writing experience
• Direct Lake support on Data Warehouse
  There's also an update on the size limits of your Fabric capacity and when it will fallback to Direct Query.
• Stored credentials for Direct Lake semantic model
  You can now specify a fixed identity (like a service principal) for a Direct Lake mode semantic model
• Pricing on Fabric is updated!
  Reserved pricing is now available, with a rough discount of around 41% off of Pay-As-You-Go pricing.
  
• OneLake integration for Import-mode semantic models is coming!
  This allows for a seamless (at least that's what Microsoft claims 😄) integration for your import Power BI Desktop models into OneLake. I wonder if this also implies that you can convert your import report to a Direct Lake mode model afterwards. It's not totally clear to me at this point.

[update on November 16th]

• Power BI Desktop November is also out!
  Power BI November 2023 Feature Summary
• DAX query view
• Stored credentials for Direct Lake
• Explore

Conclusion

Will I now go all-in on Fabric? "It depends", but probably not 😀

I think it still depends on a case per case basis. Is it a newish customer with not too many investments in other data platform services like Synapse/Data Factory. Then it makes sense to evaluate the requirements and see if it's worthwhile to start with Fabric, considering a lot of features are still in preview, and others are not there yet.

On the other hand, if it's an existing customer with real estate in Synapse and databricks for example, where they implemented a medaillon structure and have infrastructure running there, I'd seriously reconsider if it's worth moving to Fabric. I'm leaning towards a no for now.

Keep a look out on the official Fabric and Power BI blogs and Microsoft Learn for all new content:

• https://powerbi.microsoft.com/en-au/blog/
• https://blog.fabric.microsoft.com/en-US/blog
• https://learn.microsoft.com/en-us/fabric/get-started/whats-new

I'm updating this post live while the updates are rolling out, so come back later for more updates!😀

Data, Insights, and Community: My Reflections on Data Saturday Holland and dataMinds Connect

The Lamot conference center alongside the river Dijle in Mechelen, Belgium

The world of data and analytics is constantly evolving, with new tools, technologies, and best practices emerging almost daily. As a data enthusiast and professional, I'm always on the lookout for opportunities to expand my knowledge and stay up to date with the latest trends in the field. Recently, I had the pleasure of attending two fantastic events: Data Saturday Holland and dataMinds Connect, and you may already guess—it was amazing!

Data Saturday Holland - Where Passion Meets Expertise

Data Saturday Holland, formerly known as SQL Saturday Holland, is a renowned event that brings together data professionals, enthusiasts, and experts to share their knowledge and experiences.

One of the things that struck me the most at Data Saturday Holland was the passion of the speakers and participants. From Power BI to data engineering, there was a wide range of sessions to choose from. I attended sessions covering various topics like Direct Lake in Power BI and improving your Power BI report. These sessions provided me with valuable insights and practical tips that I could immediately apply to my work. It was enlightening to learn from experts who shared their real-world experiences and demonstrated the tools and techniques they use on a daily basis.

A highlight of the Saturday was being able to attend my favorite podcast, Knee-Deep in Tech, live in a movie theater!

Knee-Deep in Tech by Heini Ilmarinen, Alexander Arvidsson and Simon Binder

Another highlight of the event were the networking opportunities. I had the chance to meet old friends and new people, exchange ideas, and even discuss potential collaborations. It's incredible how the event fostered a sense of community and encouraged knowledge sharing. I left Data Saturday Holland inspired and motivated!

DataMinds Connect - A Deep Dive into Data and AI

Just when I thought my week couldn't get any better, I had the privilege of attending dataMinds Connect. This event is an annual conference organized by the dataMinds community, focusing on Microsoft Data Platform technologies. The event spans (for now 😏) two days and covers a wide range of topics related to data and AI.

One of the standout features of dataMinds Connect was the depth and breadth of the sessions. Experts from various domains within data and AI shared their knowledge. From advanced SQL Server features to leading AI applications, there was something for everyone. I especially enjoyed the in-depth sessions that allowed me to explore complex topics in detail, like Mathias Thierbach's Power BI Source Control precon.

I also volunteered during the two days this year, assisting speakers and visitors in making the most of their own experience. During Mathias' workshop, I helped answer questions and on the second day, I addressed general visitor questions and took care of the speakers, so they didn't have to worry about the technical aspects, drinks, or other logistics.

The community at dataMinds Connect was incredibly welcoming, and I had the opportunity to engage in conversations with participants and speakers. This sense of community and feeling of being welcome was a common theme throughout the event, and it's something that truly sets this conference apart. A beer and some chocolate certainly works wonders too! :-)

The Value of Conferences

Attending Data Saturday Holland and dataMinds Connect provided me with a comprehensive overview of the Power BI and Fabric landscape and the latest developments in this field. I left with new skills and insights and a better understanding of the latest trends and technologies. Moreover, the connections I made during these events were invaluable. Networking with professionals who share my passion and interests can lead to future collaborations, career opportunities, or simply the joy of being part of a vibrant and supportive community.

In summary, my week at Data Saturday Holland and dataMinds Connect was an incredible experience! These events not only expanded my knowledge but also allowed me to connect (and continue to connect) with fellow data enthusiasts and experts!

I have a few more things coming up in the following monhts:

• New Stars of Data with my mentee Ioannis Philippides
• Data Left Unattended
• Data Saturday Vienna
• ... A few more things are in the works, but nothing definitive yet...

Do I see you there? 😀

Pausing a Fabric Capacity - What Does It Actually Mean?

After an initial question by my friend and fellow MVP Koen Verbeeck, myself and a bunch of people started answering, amongst others was Mohammad Ali, Group Program Manager for Power BI.

If a #MicrosoftFabric capacity is not running, you cannot use the #PowerBI reports? cc @BenniDeJagere @cwebb_bi

— Koen Verbeeck 🇺🇦 🌈 (@Ko_Ver) September 27, 2023

After a while it got me thinking:

• What does it actually mean when I pause a Fabric capacity?
• What will stop working?
• What can I still do and won't stop working?

Important considerations

Microsoft Fabric is a prerelease online service that is currently in public preview and may be substantially modified before it's released. Preview online service products and features aren't complete but are made available on a preview basis so that customers can get early access and provide feedback.

A note before you start and might be aware of, Microsoft Fabric is still in preview, so be aware of the available functionality, availability and supportability, which is described in detail here.

TL;DR

After playing around and testing various scenario's, I was quite surprised on a few answers I got, so keep reading if you want to find out!

In case you are not interested in the setup, you can also skip right to my tests or the conclusions.

Start setup

The steps I took to start exploring the capacity capabilities are the following:

• I created a Fabric capacity in the Azure Portal for my tenant. You can even start an Azure (30 day) free trial and use that to create a Fabric capacity. Erwin did a great job explaing how to create a Fabric capacity, so I won't go into details here.
• Then I set up a basic Lakehouse from the Lakehouse tutorial on Microsoft Learn. I followed the tutorial up untill step 3 (Build a lakehouse), where I end up with a dataflow Gen2, a lakehouse and a Power BI (Direct Lake) report on the default dataset.
• I also created 2 workspaces:
  
  

• Test Fabric Capacity holds all my Fabric artifacts items and has the Fabric capacity (nickyscapacity, see below) assigned. This is the workspace I used for my tutorial. Let's call this the Fabric workspace.
• Test Fabric semantic model has no capacity assigned, so it's a regular (pro) workspace. Let's call this workspace.
• After that, I've created a few datasets/reports (or semantic models if you will 😀) (with Direct Lake, DirectQuery and Import) on top of the SQL Endpoint of my lakehouse.

The basic report I created, it's not really important how it looks for now:

My tests

The first thing is of course pausing my capacity, which is an easy push of a button in the Azure portal.

Next I want to see what happens when I access certain items or take certain actions in my workspaces.

Here's a list of things I tried to do:

• Access dataflow Gen2
• Access the Lakehouse
• Access SQL Endpoint (of the Lakehouse)
• Access Direct Lake model from Fabric workspace
• Access Direct Lake model from workspace
• Access DQ model from Fabric workspace
• Access DQ dataset from workspace
• Access Import dataset from Fabric workspace
• Access Import dataset from workspace
• Download import dataset and re-publish to workspace
• Refresh Import dataset from workspace
• Move the Fabric workspace to Pro
• Access the Fabric Capacity Metrics app
• Edit capacity settings

Access my dataflow Gen2

Not a very helpful error message 😀

Access the Lakehouse

This is very helpful, it actually mentions my capacity (ID) is not running.

Access SQL Endpoint (of the Lakehouse)

Not very helpful, it doesn't say anything about my capacity.

Access Direct Lake model from Fabric workspace

This one IS helpful, it actually mentions my capacity is not active, although it doesn't give the ID like with accessing the lakehouse.

Access Direct Lake model from workspace

I created a copy of the report into the regular workspace and opened the report.

Strange enough, I suspect because of some caching that was already done before pausing the capacity, some parts of the report still work. I assume that cache is then copied over (with the report) to the regular workspace.

Some interactions worked, but when I clicked a filter without any cache, I got the same error message as below with the DQ model.

Access DQ model from Fabric workspace

The visual itself gives me the above error, see the detailed error message below: not very helpful.

Access DQ dataset from workspace

This one is a bit inconsistent because I got different errors on this action.

I'm accessing the DQ report in the other, regular workspace. I'm getting a slightly different error in the visual, but the detailed error message is totally different then from the Fabric workspace. It's more a SQL server error message.

It at least tells me there's something wrong with the SQL endpoint.

But when I tried this same action later, I got the following error, which is very much helpful because it mentions the CapacityNotActive.

Access Import dataset from Fabric workspace

The error itself is helpful, because it mentions the capacity.

However, this one got me a bit surprised, because I'm accessing an imported model, so the data is no longer in OneLake. But as we'll see a bit further, nothing from a Fabric workspace can be accessed anymore when the capacity is paused.

Access Import dataset from workspace ✅

The difference with the action above is that this is the regular workspace. This one succeeds, because the data is in the imported model in the regular workspace, which is active and running. It has nothing to do with the Fabric capacity.

Download import dataset and re-publish to workspace ✅

Surprisingly (to me), I can still download the dataset from the Fabric workspace. So it seems the dataset itself is not stored in OneLake, since that is paused. Still a bit strange why then the import model doesn't start from this Fabric workspace.

Republishing to the (regular) workspace succeeds and gives me the report below:

Refresh Import dataset from workspace

This action pertains to refreshing the Import dataset from the regular workspace, the dataset which I could open. However, the refresh action itself fails, because it needs the lakehouse data to refresh, which is not available.

Move the Fabric workspace to Pro

When trying to move the Fabric workspace to a regular (Pro) workspace, you might be thrown off by this message in the workspace Premium settings in the bottom:

However, moving a Fabric workspace to Pro is only possible when there are no Fabric items inside:

This is also mentioned as one of the current restrictions.

I would urge you to carefully read those restrictions, the known issue(s) and final way of working when the known issue is resolved, especially if you plan to move items between regions after a workspace has been created.

Access the Fabric Capacity Metrics App ✅

The Fabric Capacity Metrics app just keeps functioning. It doesn’t need the capacity itself to operate on, it uses the analytics/telemetry from the capacity that is logged and reports on that.

Editing capacity settings

The capacity settings in the Fabric Admin portal are italic and cannot be edited, unless you resume the capacity.

Conclusion

So to conclude: all items in a Fabric workspace become unavailable (for interactive opening) when a capacity is paused. Also Power BI-only items.

You can still download an import dataset from the workspace. You can also export the .json file of a dataflow (gen 1 and gen 2). But that's about it you can do on a workspace with a paused Fabric capacity.

1: Depending on the way you created the report, it might still have some cache, so it might work partially

Thanks to Štěpán Rešl for pointing out the Usage Metrics report.

So for now, it's best to separate the two, Fabric and non-Fabric items, in separate workspaces so you can always access the Power BI only items when the capacity is paused. In case you are not running your own capacity, but a free trial capacity, you don't have to take this into account for now.

I hope this overview was usefull to you, I can at least use it as a reference and lookup post :-)

I am sure a couple of things will change in the near future, as Fabric updates keep coming out regularly.

If you are missing something from this overview let me know in the comments and I can see if I can add it here.

SQLBits 2024 Has Landed!

I've written about SQLBits before here and here, but in case you still don't know what SQLBits is, it's the greatest Data conference in Europe, spanning a whopping 5 days, including the free community event on Saturday.

From the SQLBits website:

SQLBits is the industry’s leading Data Platform conference with over 300 sessions across 5 days covering data technologies including SQL Server, Azure, Big Data, Power BI, Machine Learning, and more.

And if you need a reason (for your boss) to attend SQLBits: there are 10 reasons listed on their website!

Next to all the great content, there's also lot's of extra's happening at SQLBits, like a board game night, a pub quiz, a SQLBits run and last but not least the famous Friday night party!

Previous Editions

I've also had the pleasure of presenting at 2 SQLBits events. In 2022 it was virtual, but last year I was excited to be attending and speaking at my first in-person SQLBits! I delivered 2 sessions, one (20-minute) lightning talk and a general session on Write-back with Power Apps in Power BI (the recording can be found here).

2024

Now on to the 2024 event, last week there was a live stream with Marco Russo, Alberto Ferrari (both from SQLBI) and Simon Sabin (founder of SQLBits) announcing the dates and location for this years event. This year SQLBits will be from March 19th - 23rd, in Farnborough.Hampshire, UK.

The theme for this year will be Aviation, because Farnborough is the birthplace of aviation and the home of pioneering spirit.

The pricing and Call for Speakers for the event will be open very soon, so be sure to keep an eye on their website!

See you there? 😀