Login

CVMFS and Zenodo Jupyter extension weekly

→ Europe/Zurich
31/S-023 (CERN)

31/S-023

CERN

22
Show room on map
CVMFS and Zenodo Jupyter extension weekly
Zoom Meeting ID
69795520826
Host
Giovanni Guerrieri
Useful links
Join via phone
Zoom URL
    • 11:30 → 12:00
      Roundtable 30m
      Speakers: Giovanni Guerrieri (CERN), Oliver Geneser, Oshika Jain, Elisabeth Hannah Juttner

      Zenodo Extension

      Implemented changes

      • Migration to the new Zenodo/InvenioRDM API
        • differs in response format for record details, information lives in different places
        • Some routes still use old format but mostly complete
      • New features
        • "Create new version" for published user records
        • Dropdown for switching between record versions
        • Request the user's permissions for the record ("edit", "view"...) to only display the available UI actions
          • Not fully working because Zenodo API does not expose the necessary information
          • Alex is planning to fix this
        • Bugfixes, structural changes (SSE for live updates in UI, unified progress bars) for smoother workflow
      • Rate Limiting Issue: Extension has many features now (switching versions, dealing with different acccess permissions...) that need many API calls. Getting rate limited quite quickly
        • Examined and documented how extension currently uses Zenodo API
        • Simplified and removed unnecessary API calls
        • Caching User ID in extension backend
        • => Planned how to further save some API calls by restructuring/ simplifying workflow in extension

      Next To Do's

      - Complete migration to inveniordm record format
      - Continue working on fix rate limiting
      - Finally start with integration tests (and add more unit tests)

      Lightning Talk

      Rough first draft of presentation:

      • Project Goal
        • Log into Zenodo
        • Search Zenodo records and communities
        • Download files into jupyter server
        • Upload files from jupyter server
        • State shortly what Zenodo and JupyterLab is
        • Goal: Develop a JupyterLab extension that allows you to…
      • Main challenge: Not implementing features, but designing an architecture that securely handles long-lived Zenodo access tokens across different Jupyter deployments, while staying maintainable
      • Solution:
        • OAuth Login for secure token exchange and usable authentication experience
        • Optional isolated Zenodo API proxy to store access tokens
          • Access tokens never reach the browser or Jupyter server
        • Backend-first architecture
      • Results:
        • Show screenshot(s) of the extension
        • Architecture diagram

       

      CVMFS Extension

      Features added:

      • generated lua modules files for LCG_106, LCG_107, LCG_108, LCG_109, LCG_110 - x86_64-el9-gcc13-opt and x86_64-el9-gcc14-opt
      • creating a browsable catalog creating a browsable catalog for lua module files to be displayed there which will look like this

      To Dos:

      • platform detection with archspec
      • sync kernel and terminal to the same python version
      • spawn a new session in mid without losing work
      • create tests for these

       

      Lightning Talk:

      • Project goal:
        • generate modules files for LCG stack
        • make a browsable catalog
        • platform detection with archspec
        • sync kernel and terminal to the same python version
        • spawn a new session in mid without losing work
      • Challenges until now:
        • Creating a correct LUA modulefile by testing several paths
        • sync kernel and terminal to the same python version

       

      Homework

      Documentation: operator (lazydocs, programmatically done?)

      Document bugs etc of upstream services.

      Docs website: https://github.com/vre-hub/vre-hub.github.io
      Open PR there :D