CephFS is awesome, but without a VPN it's not safe to be routed over the internet, and it lacks client support on many platforms. The Linux Kernel Library allows for the linux kernel to be built as a cross-platform user-space library and integrated into a regular Android, Windows, macOS, etc. application. This project combines CephFS, LKL and wireguard, and sprinkles some io-uring on top.

My objectives are:

  • rebase LKL atop v5.6-rc1 ✅
  • implement basic CephFS mount support within LKL tools ✅
  • fix mount-time stall due to lack of entropy within LKL ✅
  • get the CephFS kernel client building against the Android NDK via the LKL ✅
  • learn about Wireguard usage and internals ✅
    • modify wireguard-tools so that wg can call into the LKL for VPN deployment ✅
  • implement a UDP packet forwarder based on io-uring
    • the forwarder will route wireguard-encrypted traffic from the LKL network to the Android host network and vice-versa
    • io-uring can only be used on the LKL side. Android doesn't provide it on the host side
    • WIP: I got part way through this task (just epoll, no io-uring yet), but ran out of time
  • stretch goal: create a Android GUI to browse a mounted CephFS filesystem
  • stretch goal: help out with the LKL mainlining effort
    • reported a couple of minor bugs, discussed Android non-root networking with Hajime

Source: https://gitlab.com/ceph-walkley/walkley


  • bmwiedemann
    9 months ago by bmwiedemann | Reply

    Wouldnt librados or the ceph FUSE layer be an easier route to get this to work in userland? I think there are also already userspace wireguard implementations that could be coupled to that.

    • dmdiss
      9 months ago by dmdiss | Reply

      It probably would be, but it wouldn't be anywhere near as fun for me :smiley:. Other reasons for going down the kernel client + LKL route for me are:

      • less bloat
        • Ceph user-space client libraries are very large, and have a huge amount of dependencies
        • with LKL I have zero dependencies, clean C, and a binary that should be much smaller (even with the rest of the Linux kernel)
      • other goodies
        • it should also be possible to add support for mapping+mounting RBD images using the other XFS/Btrfs/ext filesystem drivers within LKL

    • dmdiss
      9 months ago by dmdiss | Reply

      Also regarding the user-space wireguard implementation for android. IIUC it uses a tun interface on the host for routing all traffic through the VPN. This requires extra Android app permissions, and is unnecessary for my current use case where I only want to encrypt Ceph traffic

Similar Projects

This project is one of its kind!