The SunnyData Blog

Explore insights and practical tips on mastering Databricks Data Intelligence Platform and the full spectrum of today's modern data ecosystem.

AWS

ETL

EDW

Integrations, Migrations, Databricks, ETL Nicanor Medina Integrations, Migrations, Databricks, ETL Nicanor Medina

Seamless Data Integration: SAP to Databricks

Learn how to integrate SAP data into Databricks with this comprehensive blog. Discover the essential components of the SAP ecosystem, including SAP HANA, S/4HANA, and BTP, and explore proven integration methods using SparkJDBC and Azure Data Factory. Perfect for data engineers and architects looking to combine SAP's enterprise management capabilities with Databricks' advanced analytics.

Read More
Databricks, Cost, ETL, Benchmark Josue Bogran Databricks, Cost, ETL, Benchmark Josue Bogran

Performance, Benchmarks, and Optimization Tips for Databricks Users

Josue Bogran interviews Jeremy Lewallen from Databricks’ Performance Team, exploring benchmarks, storage cost optimization, rightsizing SQL Serverless Compute, and common compute mistakes. Discover why Databricks continually enhances performance, tips for using the latest DBRs, and how their innovations provide a fast, efficient, and developer-friendly data platform.

Read More
Migrations, Databricks, ETL, EDW, AWS Nicanor Medina Migrations, Databricks, ETL, EDW, AWS Nicanor Medina

Redshift to Databricks - Part 2: Technical Implementation Guide

This guide dives into the technical steps required to migrate from Amazon Redshift to Databricks. Covering everything from discovery and data evaluation to security protocols and cost estimation, it offers detailed, practical strategies for managing dependencies, optimizing queries, and planning for future scalability within Databricks’ robust ecosystem.

Read More
Migrations, Databricks, ETL, EDW, AWS Nicanor Medina Migrations, Databricks, ETL, EDW, AWS Nicanor Medina

Redshift to Databricks - Part 1: Why and How to Start Your Migration

This blog introduces the strategic benefits and challenges of migrating from Amazon Redshift to Databricks. It covers Redshift’s legacy limitations, Databricks' advantages, and critical migration factors. The article provides an overview of key planning steps, including architecture considerations and phased migration strategies, setting the stage for technical execution in the upcoming part two.

Read More
Migrations, Databricks, ETL, EDW, AWS, AI, Fabric, Hadoop, Snowflake Nicanor Medina Migrations, Databricks, ETL, EDW, AWS, AI, Fabric, Hadoop, Snowflake Nicanor Medina

How to migrate your ETL workloads and EDW from Snowflake to Databricks

In this blog, we outline the essential steps for migrating ETL workloads and EDW from Snowflake to Databricks. From data migration to report modernization, we break down five key phases for a seamless and efficient transition to Databricks.

Read More
Databricks, Integrations, ETL, Benchmark, AI, Fabric Jeronimo Acosta Databricks, Integrations, ETL, Benchmark, AI, Fabric Jeronimo Acosta

Fabric Meets Databricks: A Preliminary Review for Data Practitioners

This article provides an initial analysis of Microsoft Fabric, comparing it with Databricks. It discusses why Fabric may not meet enterprise needs and explores potential integrations between Fabric and Databricks, highlighting how they can complement each other in data projects.

Read More
Databricks, ETL, AI, Hadoop, Snowflake Jeronimo Acosta Databricks, ETL, AI, Hadoop, Snowflake Jeronimo Acosta

As the snow melts, it's time to build a data skyscraper with Databricks

Funny or not, building a secured, governed and scalable data platform that supports multiple types of use cases along with the data management processes and practices is very similar to building a skyscraper - the higher the building grows and supports more units and people, the complexity increases.

This guide will help you understand the complexities of Databricks, ensuring your data skyscraper stands tall and proud.

Read More
Migrations, Databricks, ETL, AWS, Hadoop Kailash Thapa Migrations, Databricks, ETL, AWS, Hadoop Kailash Thapa

Hadoop to Databricks: A Guide to Data Processing, Governance and Applications

In the intricate landscape of migration planning, it is imperative to map processes and prioritize them according to their criticality. This implies a strategic process to determine the sequence in which processes should be migrated according to business.

In addition, organizations will have to define whether to follow a "lift and shift" approach or a "refactor" approach. The good news is that here we will help you choose which option is best for the scenario.

Read More
Databricks, Migrations, ETL, EDW, AWS, Hadoop Kailash Thapa Databricks, Migrations, ETL, EDW, AWS, Hadoop Kailash Thapa

Migrating Hadoop to Databricks - a deeper dive

Migrating from a large Hadoop environment to Databricks is a complex and large project. In this blog we will dive into different areas of the migration process and the challenges that the customer should plan in these areas: Administration, Data Migration, Data Processing, Security and Governance, and Data Consumption (tools and processes)

Read More