Insight announcements How Data Platform as a Service Is the Fabric of More and More Gen AI Solutions
By Dan Kronstal / 10 Sep 2024 / Topics: Microsoft Fabric
Image created by DALL-E 3
As our clients explore the benefits of generative AI, the foundation for successful adoption is clear: a data strategy that sets up productive implementation using corporate knowledge bases. On top of tremendous investment in AI services and technologies, Microsoft has introduced Fabric as a unified data service and a platform. Fabric notably streamlines the ingestion, processing, management, governance, and presentation of corporate data. Insight Canada has subsequently been working with clients to implement Fabric as the key set piece in their data estate. Our most recent project is a great example.
The generative AI angle
The client in this case operates in the manufacturing sector. They needed a new data platform to improve performance and scalability. Insight designed and built it based on Fabric, transitioning them from an on-premises data warehouse to a cloud-based platform. We also implemented new Extract, Transform, Load (ETL) processes to improve data quality, accessibility, and reporting. By using gen AI through Microsoft Copilot (and Fabric), they’re taking steps to future-proof themselves, making innovating down the line easier.
Fabric is a Software-as-a-Service (SaaS) data platform environment — effectively Data Platform as a service. It gives you the opportunity to do all your data ingestion, storage, transformation, publication, governance, lineage tracking, and then consumption through the Power BI front-end all in one place.
Now, that doesn’t sound like it’s inherently related to gen AI. However, think of something like Retrieval-Augmented Generation (RAG). It refers to when you grab data from an internal source to augment the knowledge base upon which a large language model is trained. In Fabric, you have all that information ready to access. You can use Azure OpenAI Service, which also integrates with Fabric, to extract that data for RAG. It then provides the best possible responses to end users after they ask their questions.
Introducing Fabric
Insight has Fabric-specific offerings like a pilot and a readiness assessment to determine how Fabric best fits an organization. Larger-scale offerings are available for companies that want to invest in Fabric outside the context of gen AI. Copilot or gen AI can be additional value-adds layered atop that deployment. It all depends on what clients need.
Like the client I mentioned, many others are already exploring how these offers can support their adoption journey for Fabric. We’re getting to the point where they’re gaining awareness of it as a tool and solution. And we’re also introducing it to clients as a potential solution in specific instances where it fits. Good fits include clients using Microsoft Dynamics 365. That’s because the dataverse and the entities that are exposed through Dynamics 365 are like first-party data entities in the Fabric ecosystem.
Fabric gives easy access to your Enterprise Resource Planning (ERP) and Customer Relationship Management (CRM) data. Before, you needed to build a lot of manual integration. Also, anyone on Snowflake, which offers more traditional cloud-based data platform capabilities, would be interested as well. Now with the new integrations Microsoft has made available to Snowflake, you can have a really mature level of capability that Fabric affords through notebook development, balanced on the more traditional sort of skill sets of your Snowflake and traditional Transact SQL (T-SQL) developers.
Power BI Premium clients or Microsoft 365 E5 license-level clients would also be interested, because Microsoft is retiring Power BI Premium per capacity SKUs. So, those organizations will logically want to move to Fabric at some point anyway. Just to reiterate, it’s not an “if.” It’s a “when.”
Not everyone needs Fabric, though. Some clients don’t have Power BI Premium. Some companies are committed to Databricks, which is a viable alternative. For anyone using stored procedures or more traditional types of data processing as part of their ETL processes, Fabric represents a pretty big skill gap. Transitioning to Fabric requires a lot of upscaling to benefit from the SaaS’ capabilities.
“Lakehouse” or “Data Platform” as a Service
Fundamentally, Fabric provides “Lakehouse-as-a-Service” capabilities, combining a data warehouse and lake. Bringing the storage and the compute together in one service enables end users to just turn on a data platform with a very robust portfolio of features to meet data management needs.
“Data Platform as a Service” is more accurate, though. On top of the lakehouse capabilities, Fabric offers:
- Real-time analytics
- Data ingestion and transformation pipelines via Azure Data Factory
- Notifications and telemetry through Data Activator, a no-code data-monitoring feature in Fabric
Before, if you wanted all these capabilities, you’d need to deploy an Azure landing zone. Into that landing zone, you’d have deployed a Data Factory service, a storage account to operate as your data lake, Microsoft Synapse or Databricks or Snowflake to act as that data warehousing compute engine, and Power BI licensing to enable your end users to access the data and do their own self-serve reporting. Fabric simplifies a lot of the infrastructure work when setting up a similar platform.
Easier to deploy and run
If you want something that’s really custom, you can have it — with all the same tools you know and love, like Data Factory and Synapse, etc. You can still spin up an environment manually. When you do that, though, you will need to manage, protect, and harden that environment yourself.
Fabric has taken away all that administrative and management activity. As a result, you can focus on operating the data platform. You don’t need to manually integrate and create, say, private endpoint peering between all the different services that are part of your data platform. All those things are protected. They also speak to each other natively through the integration in Fabric.
Fabric is easy to deploy and run. That’s the simple version. Especially compared to data platforms that manually integrate multiple services together. As gen AI becomes more entrenched in the day-to-day of organizations boosting efficiency, Fabric becomes a logical eventual investment. Based on the projected uptick in demand, “eventual” is coming sooner rather than later.
Dan Kronstal
Principal Architect, Solutions, Insight Canada
Dan leads the technical delivery and go-to-market engagement for our Modern Apps, Data & AI, and Intelligent Edge practices for Insight Canada. These solution areas cover a diversity of capabilities from advanced network design to hybrid data center and cloud services to application modernization and immersive experience. Dan and his teams are passionate enthusiasts of emerging technology and creative problem solving, and thrive on achieving technology-driven business value for clients spanning size and industry spectrums.
