Automating Data Imports in a DSpace-CRIS’s Institutional Repository
Year: 2025
Type: conference-paper
Abstract: The migration of Infoscience, EPFL’s institutional repository, to DSpace-CRIS required a custom Python-based pipeline to automate the ingestion of research outputs and datasets. Limitations in default DSpace-CRIS import tools, such as insufficient query controls, incomplete metadata mappings, and a lack of deduplication mechanisms, necessitated a tailored approach. The pipeline leverages the DSpace REST API to enable precise queries, metadata reconciliation, and robust deduplication. It incorporates fallback mechanisms, such as publisher-specific APIs, for full-text retrieval when standard tools like Unpaywall and CrossRef prove insufficient. Key challenges included reconciling authorship with EPFL directories, aligning metadata across diverse collections, and maintaining data consistency during imports. The developer track presentation will provide a visual breakdown of the pipeline’s architecture, highlight key challenges, and illustrate the solutions implemented. The presentation will complement this by delving deeper into the technical details and lessons learned. Both formats will offer practical insights for repository managers and developers seeking to automate data imports and optimize workflows in institutional repositories.
Authors: Rodrigues de Matos, Jorge, Sicot, Julien
Institution: École Polytechnique Fédérale de Lausanne
Language: English
Topic: Optics and Image Analysis
Subfield: Management Information Systems
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W6948438114
FWCI:
Cites:
Cited by:
Related to: