IntelStack: An Open-Source, Sovereign and Resilient Data-Science Platform for Resource-Constrained Research Environments - Preprint
Year: 2026
Type: preprint
Abstract: Small and medium-sized organisations that need advanced data-science capability rarely have the budget, staff or legal capacity to adopt large commercial cloud platforms in a sovereign way. At the same time, single-purpose open-source tools deployed in isolation do not by themselves constitute a usable, governed data-science environment. This paper presents IntelStack, a sovereign and resilient platform that integrates multiple concurrently running applications on a single virtual machine with internal integration and unified governance: fourteen mature, carefully selected open-source projects (Apache Airflow, LinkedIn DataHub, DHIS2, Project Jupyter's JupyterHub, Keycloak, MinIO, NGINX, ODK Central, Ollama, Open WebUI, pgAdmin4, PostgreSQL, SNOMED International's Snowstorm, and Apache Superset) and five custom-built components (GlobalSecurity, GlobalStatus, IntelWeb, RWKV and SnowstormExtensionsAndAI) that supply identity-aware entry-point security, unified observability, Open-Source Intelligence (OSINT) capability, AI-assisted information analysis, public-facing dissemination, sovereign large-language-model usage and AI-assisted terminology extensions. The platform was developed and first operationally tested in the context of the research project "Interoperability and utilisation of epidemiological surveillance data with artificial intelligence", which required the rapid development of a low-cost, standalone/sovereign and resilient data-science environment for public-health data experimentation. The architecture, however, is domain-agnostic and is reported here as a general-purpose sovereign data-science platform rather than as a public-health-specific tool. We describe the platform's layered architecture (acquisition, orchestration, storage, cataloguing, semantic enrichment, prediction/model execution, analytics, dissemination, identity, entry-point security and observability), map each of the thirteen stages of a generic data-science lifecycle to the component or components that implement it (eight with open-source-only coverage, three with mixed open-source/custom coverage and two with custom-only coverage), document the criteria (licence/sovereignty, fit-for-purpose, technology maturity) used to select each open-source component, and report operational telemetry captured from the live deployment, including per-container memory footprint, disk space, CPU usage, initial storage provisioning and the managed reverse-proxy and certificate-binding mechanism that keeps the independently built applications under one coherent identity and routing layer. The evidence shows that a lifecycle that would conventionally require several disjoint commercial services can be covered end-to-end on commodity single-host hardware (15 GiB RAM, 4 vCPU, 200 GB disk), at a measured aggregate memory utilisation of approximately 90% with zero configured swap, 16% CPU utilisation and 90 GB of storage in use, and that the custom components close specific gaps that the open-source components do not address on their own. We claim that the research gap this platform covers is not a missing set of tools but a missing reference architecture and evaluation for a small-footprint, interoperable, sovereignty-preserving, resilient data-science platform assembled from heterogeneous open-source and custom parts, and we propose a measurement matrix - covering completeness, interoperability, resource efficiency, security-control enforcement and reproducibility of deployment - for comparative evaluation across different installations. This is a single-instance, descriptive case report rather than a controlled experiment. The contribution is an architecture, an empirical operating profile and a replication mechanism, not a claim of superiority over any specific alternative.
Authors: Ktenas, Spyros, Chrysoula, Samara
Institutions: Ionian University, University of Thessaly
Language: English
Subfield: Information Systems and Management
Field: Decision Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7225403622