Hybrid NoSQL-SQL Database Architectures for Real-Time Big Data Analytics and Scalable Cloud Applications
Year: 2024
Type: article
Abstract: The increasing data volume, velocity and variety has been a serious challenge to the conventional data management systems. To overcome scalability, heterogeneity and low-latency processing issues, this paper discusses a hybrid SQL-NoSQL architecture, which combines in-memory databases (MMDB), relational databases (RDB) and distributed file systems (DFS) to manage spatio-temporal big data efficiently. The framework that is proposed allows real-time ingestion of data, on-the-fly processing of analytics, and scalable long-term data storage using a single integrated hybrid framework. The paper also analyzes data interoperability, ACID- BASIS consistency trade-offs, streaming systems/frameworks, e.g., Spark, Flink and Storm and event-driven high-velocity analytics architectures. Also, the use of AI/ML methods in predictive scaling and performance optimization of cloud-based systems is pointed out. Literature review is conducted to establish major gaps in the areas of schema transformation, distributed query optimization, and hybrid consistency management. The results reveal that hybrid SQL-NoSQL systems provide a robust platform of real-time analysis of large amounts of data and scalability cloud applications as well as modern decision-making systems. On the overall, the study presents a thorough architectural framework to the creation of scalable, real-time, big data analytics systems.
Institution: Microsoft (United States)
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W7142014055