5 min read
We are excited to announce our first phase of support for Microsoft Fabric, which focuses on integrating OneLake as a storage option within our Data Ingestion Layer (ODX). This integration not only accommodates the widely-used Delta Lake standard but also offers a streamlined method for data ingestion into Fabric.
With this integration, we are also setting the stage for subsequent phases of our support for Microsoft Fabric, such as incorporating the Fabric Data Warehouse for robust analytical capabilities and leveraging PowerBI DirectLake endpoints for enhanced business intelligence insights.
For more information on Microsoft Fabric, check out our Ultimate Guide.
Our Journey to Microsoft Fabric Support
In our progressive journey towards integrating with Microsoft Fabric, we are ensuring our holistic data integration solution aligns with the principles of the medallion architecture.
According to Microsoft:
"The medallion architecture describes a series of data layers that denote the quality of data stored in the lakehouse... The terms bronze (raw), silver (validated), and gold (enriched) describe the quality of the data in each of these layers."
Our support for Microsoft Fabric is a progressive, three-phase journey towards comprehensive integration in alignment with this architecture:
Phase One: Establishing a Foundation with OneLake (Complete)
At the outset, the first phase introduces support for OneLake (Delta Parquet) as the chosen storage mechanism within TimeXtender's Data Ingestion Layer (ODX). This foundational step provides support for the widely adopted Delta Lake standard, capitalizing on its robustness and scalability.
Our Data Ingestion (ODX) server utilizes REST API interfaces, supporting a seamless and flexible interaction between various data sources and the ODX, paving the way for an agile and responsive data environment By adopting the Delta format within our Ingestion Layer’s Bronze/Silver Lakehouse, TimeXtender aligns with the medallion architecture, leveraging Spark's analytics capabilities to enhance data processing within these initial layers.
We are thrilled to announce that Phase One of our support for Microsoft Fabric, featuring the integration of OneLake as a storage option within our Data Ingestion Layer (ODX), is live now.
Phase Two: Advancing to Fabric Data Warehouse (Coming Soon)
Transitioning to the next phase, TimeXtender sets its sights on the Data Preparation Layer (MDW), where the Silver/Gold tier of the data's lifecycle is optimized for analytics and business intelligence in a data warehouse.
By selecting Microsoft Fabric for data warehouse storage and maintaining data in the Delta format, TimeXtender ensures that its users can capitalize on the data warehouse's advanced query capabilities, including the efficiency of T-SQL for writing operations when necessary.
This phase underscores TimeXtender's commitment to offering a solution where the data warehouse's enhanced capabilities are fully harnessed, ensuring data integrity and readiness for complex analytics use cases.
Phase Three: Enabling PowerBI Integration with DirectLake (Coming Soon)
The journey culminates with the final phase, where the focus is on maximizing the analytics and reporting functions through PowerBI's DirectLake Endpoint for Fabric.
This integration enables users to draw directly from the wealth of data within OneLake. The strategic use of a Semantic/Delivery Layer (SSL) and the formulation of Gold-tier data products through DAX further refines the approach, allowing for more intuitive data interactions and accessible analytics.
This phase embodies TimeXtender's vision of an integrated, user-friendly environment where business intelligence tools like PowerBI can swiftly interact with your holistic data infrastructure to provide actionable insights.
New Support for Delta Parquet Tables as a Data Source
As part of Phase One of our support for Microsoft Fabric, we are also introducing a new data source connector, specifically designed for the seamless and efficient ingestion of Delta Parquet tables. This enhancement facilitates the direct transfer of data from Microsoft Fabric's OneLake into your organization's preferred storage solution through TimeXtender’s Data Ingestion Layer (ODX).
A Dedicated Feature for Enhanced Performance
It is important to note that while this feature is tailored for extracting data from OneLake, it is distinct from utilizing Fabric OneLake as your Ingestion Layer (ODX) storage option. This dedicated feature is intended to maximize performance and efficiency for the specific task of ingesting Delta Parquet data into TimeXtender’s Ingestion Layer (ODX,) highlighting our commitment to providing targeted solutions for complex data integration needs.
Data Source Connectivity Improvements
In a determined effort to enhance our users' experience and provide more efficient data management solutions, TimeXtender is also proud to announce significant improvements in data source connectivity. By building upon the robust foundation of the new TimeXtender REST Provider, we are introducing source-specific adapters that are meticulously crafted to meet the unique requirements of different data sources. This initiative begins with a keen focus on popular services: Hubspot and Exact Online, followed by Salesforce, Trello, and more in the near future.
New Data Source Connectors: Hubspot and Exact Online
We've supercharged our Hubspot and Exact Online data source connectors, expanding on the capabilities of the previous CData options. TimeXtender users can now benefit from the following enhancements:
-
Advanced Table Flattening: Dealing with complex data structures can be a daunting task. Our connectors simplify this challenge with robust table flattening capabilities, making it easier to transform nested data into a user-friendly format that’s ready for analysis.
-
Custom Endpoint Support: This feature allows TimeXtender users to precisely define and connect to custom API endpoints within these platforms, enabling a more targeted and efficient data ingestion strategy. Whether you're looking to pull in specialized datasets or leverage unique functionalities, our connectors empower you with the flexibility to access the data you need, exactly how you need it.
-
Seamless Transition: The direct integration of these connectors into TimeXtender's ecosystem means that transitioning from CData is a smooth process, free from the hassle of extensive setup or configuration.
With these improved connectors, TimeXtender users will experience a more streamlined data integration process. The connectors have been optimized for speed, enabling faster data ingestion which, in turn, accelerates the entire analytics workflow. The result is a smoother path from raw data to actionable insights.
Advanced Data Sharing Through Secure, Configurable REST API Endpoints
Get ready to elevate your data sharing capabilities with TimeXtender's new feature that enables users to expose datasets as secure, configurable REST API endpoints. This innovative semantic model endpoint is designed to empower users to publish their curated datasets as REST APIs with ease and precision.
Previously, TimeXtender users encountered challenges when attempting to share their TimeXtender data with external parties or internal applications. The intricacies of manual API development, coupled with a shortage of necessary expertise among Data Warehouse Developers and Data Engineers, made data sharing impractical and cumbersome.
With the latest update, we address these issues head-on, ensuring that:
-
APIs Are Automatically Created: TimeXtender now has the built-in capability to automatically generate REST APIs from within the TimeXtender environment, providing a dedicated URL endpoint for each table.
-
Secure Authentication Protocols: We've incorporated standard authentication protocols, including Bearer token and OAuth 2.0, to ensure secure data access. This level of security means that data sharing is not only straightforward but also meets the high standards required for data protection and compliance.
-
On-Premises Deployment: In line with our commitment to flexibility and control, the API engine can be deployed as a web service on the customer's premises. This is compatible with IIS and cloud-based Virtual Machines, aligning with the deployment model of the TimeXtender Ingestion Layer (ODX) service to provide the peace of mind that comes from maintaining data within your own infrastructure.
Unlock the full potential of your data with TimeXtender's latest features and transform the way you share information across your business ecosystem.
Read the Full Release Notes
For a detailed breakdown of all the updates and enhancements included in this release, please refer to the complete release notes available on our website: https://support.timextender.com/product-updates/timextender-6590-1-1832
Start Building a Robust Data Foundation 10x Faster with TimeXtender
Click here to get started with a FREE trial to unify your data stack, automate data integration processes, and build a robust foundation for analytics and AI 10x faster.