Snowflake is undoubtedly one of the leading data warehouse technologies and provides multiple benefits over its traditional counterparts. I already covered the benefits in one of my articles.
In this article I am going to capture the other side of it — Limitations, challenges and shortcomings of Snowflake.
Snowflake is available as a SAAS (Software as a Service) offering, which means we cannot host it on premise. This leads to relatively less control over the implementation.
Though snowflake provides multiple cloud options, the count is just three — AWS, Azure and Google Cloud. If you are on another cloud platform…
Before we jump into the world of Microservices, let's spend some time understanding the fundamentals. As it involves, a multitude of technologies, it's easy to lose track. If you think, you are already aware of the fundamentals, associated patterns, and frameworks, you can skip it and move to the series directly.
The new architecture pattern is been adopted by almost all sizes of organizations, be it small, medium, or large. Organizations have started realizing the value of it. In spite of such widespread adoption of this pattern, it's unfortunate, there is no consistent definition of Microservices.
Spring Cloud Config is one of the Spring Cloud Projects. This is directly mapped to one of the important patterns of Microservice Architecture — Centralized Configuration Service. This provides the means to centralize and manage the externalized configurations across different applications/services in a distributed system.
We will walk through this topic in the following steps
Configurations increase the flexibility of our applications(services). We can deploy the same application in multiple environments or we can change the providers (like a database) with minimal or no code change.
Spring Cloud Gateway provides a library to build an API Gateway. This is the preferred gateway implementation provided by Spring Cloud. It's built with Spring 5, Spring Boot 2, and Project Reactor.
To understand the offerings of Spring Cloud Gateway we must understand the API Gateway pattern in detail. Let's assume, we are implementing the microservices architecture for our e-commerce system. One of the microservices in the system is Product Catalog Service, which is responsible to manage product lifecycle through — create, update, delete, and get operations. Let’s go through some common scenarios, we might come across —
As the name suggests, the pattern derives its inspiration from the electrical switches, which are designed to protect an electrical circuit from damage, caused by excess current from an overload.
When a particular microservice or resource is not responding, this pattern helps in registering the fault, switching off the communication, and restoring it back when the service is ready to serve the requests. This helps the microservice ecosystem in multiple ways —
AWS Redshift is one of the leading cloud based data warehouse system. It promises to deliver 3X (times) higher performance than any of its peers i.e any other cloud based data warehouses. So whats the secret recipe?
In this article, I am going to highlight and discuss the key strategies and components which help in achieving the high performance for Redshift.
Redshift stores the data in columnar format which provides the option to optimize the performance from multiple aspects.
Columnar format suggests to keep the column specific data together, rather than the row entries. Because the same type of data…
Snowflake is the leading data warehouse technology. But that’s not all, it provides support for many other data oriented implementations including data lake, advanced analytics, data engineering, etc.
On top of it, it provides many additional features such as data security, time travel, disaster recovery, etc.
The platform is offered as software as a service, which means we do not need to worry about the hardware or software needs of such implementations. Internally it utilizes the compute and storage options available over the cloud. This also makes it a highly scalable and available system.
In subsequent sections, we are going…
Snowflake started as the cloud based data warehouse in 2014 and has evolved into an end to end data solution provider, since then. It can be defined as —
A fully managed, software as a service for data warehousing, data lakes, data engineering, data science, data application development, and for securely sharing and consuming shared data.
Because of its promising set of data solutions, it is listed as one of the leading technologies in the technology radar by thoughtworks. In this article, I will take a deeper dive into the benefits it brings on the table.
One of the USP(unique…
Delta lake is one of the key products offered by Databricks, which is the “Data + AI” company. The company was founded in 2013 by the original creators of Apache Spark™.
When it comes to data analytics, the current ecosystem is divided in two segments — one which is based on traditional data-warehouse systems (like Teradata) and the other which is based on datalakes using technologies like Amazon S3, GCS, etc. Both these models have their pros and cons and typically co-exist in an organization.
Lakehouse architecture combines the benefits of both these worlds — data lakes and data warehouses.
As per the paper presented in CIDR 2021 (Conference on Innovative Data Systems Research), Lakehouse Architecture can be defined as a data management system based on low-cost and directly-accessible storage that…