Team Quorum at Pass Data Community Summit - Sharing Our Collective Experience

PASS Community Summit 2023 Article Website Page Banner

James Frost, Tom Sykes and Abhi Jayanty attended this year’s Pass Data Community Summit Conference 2023. The event provided a unique opportunity to connect with some of the world’s leading data practitioners as well as senior members of the Microsoft data practice.

Quorum were not just there as attendees, but also delivered several talks during the conference, here are some of the more exciting announcements!

We were especially excited that Quorum Data Engineer, Abhi Jayanty, was invited to speak as part of the “New Stars of Data” programme, organised by Ben Weissman who also helps mentor the up-and-coming speakers. Abhi delivered an excellent, and very well received, talk titled “Moneyball – how data and AI has revolutionised analytics in sport”.

In addition, Abhi was asked to present as part of a panel of speakers at the “PowerHour” session, which is one of the premier sessions at the conference. The whole purpose of this session is to come up with fun and interesting uses for data which are not necessarily useful in any real world context. Abhi gave a really fun session on the new Scottish version of PowerBI, named PowerB-Aye ™ which was also a very received segment of the session.

Microsoft Fabric : General Availability and enhanced capabilities

During the conference we attended the official launch of Microsoft Fabric, which is now generally available. This represents a huge leap forward in not just the Microsoft Data offering, but we believe also represents a huge leap for the wider data industry.

In a nutshell, Fabric blends data lakes, data warehouses, data engineering, data science and BI into a single, unified, Software as a Service product that provides unprecedented levels of integration between the various components in a data solution.

Previously you would have had to use multiple platforms such as Synapse Analytics, Azure Datalake Services, Azure Data Factory and PowerBI and a lot of effort integrating the various components into a single solution – now you can create a datalake with just a few clicks. (See our introduction to Fabric article here)

As well as the official launch, new areas of functionality were announced and demonstrated, including:

OneSecurity – providing row level security at the datalake level, so you can restrict what an individual or group can see at the data level (e.g. to view only their own department information), and this flows to all downstream products such as PowerBI. Previously you would have to define this kind of security in every part of the solution, which resulted in duplication of effort and risks of mistakes being made. OneSecurity isn’t yet available, but is going to be a key part of the Microsoft Fabric architecture in the (hopefully) near future.

Additional shortcuts – Shortcuts allow you to link to both data sources within Fabric and external sources such as Azure Datalake services and AWS blob storage. Recently they added Dataverse (and hence Dynamics365) as a source, and new sources coming will include GCP storage.

Mirroring – as well as shortcuts, a new facility is now in private preview which allows you to synchronise both traditional RDBMS databases as well as NoSQL DBs. Currently Azure Cosmos DB, Azure SQL DB and Snowflake are supported, with a number of other database platforms being targeted next year. This basically gives you a close to real time copy of your database system which you can use for data science or reporting, and also allows you to join information between completely different platforms.

Data CoPilots and GPT integration

It’s been the year of CoPilots, and the data landscape is no different. Announced CoPilots (now in public preview) include:

Microsoft Fabric

Copilot to help you write Python code in notebooks based on the data you have available in OneLake.

PowerBI

Copilot can generate comprehensive dashboards from a dataset, producing functional and well-designed results.

PowerBI

GPT will soon automatically generate synonyms for data items, simplifying the process of starting with the natural Q&A feature.

SQL Server

Use Copilot, which understands the data schema, to write SQL queries, such as “show me the top 10 products by sales.”

Improvements to Azure SQL Server Managed Instances

Whilst not as obviously exciting as some of the above announcements, PassSummit announcements included some big changes to Azure SQL Managed Instance, including:

• The ability to easily failover from and back to an on-premise SQL instance.

• Pause instances to save money (e.g. pause outside working hours, or shut down a dev instance until it is needed).

• Bring your own license, again saving costs against SQL MI where you already have SQL licensing in your estate.

• Use reserved instances to further reduce spend.

These changes make it much easier to transition to Managed Instances, and combined can deliver huge reductions in spend, making SQL MI a much more attractive option in future.

Tomato Spaghetti

Finally, and most importantly, Pass was all about the community and meeting interesting new people in the data industry. Our very own SQL Tomato, Tom Sykes, managed to meetup with Spaghetti DBA, Gianluca Sartori, making it the first meetup of Tomato Spaghetti!