Simplify the codebase by using a more modern toolkit to accelerate maintenance and future development.

Enjoy Hakkaweek!

Problem

Uyuni's codebase has a 10+ year history, with contributions from dozens of individuals at varying points in their technical experience, understanding of the project, goals and timelines.

Inevitably, solutions to some of the common problems have grown organically, with different people solving slightly different instances of the same problem in different and possibly incomplete ways. This makes maintenance and future evolution of Uyuni more difficult, as developers and designers need to keep specifics of all such solutions into consideration.

One such area is the handling of asynchronous tasks within Java code, where currently we have at least 4 different implementations, none of which is really complete.

Spirit of this HackWeek effort

Write a proof-of-concept to replace as many existing, incomplete implementations as possible using one piece of existing, proven open source technology which already fulfills all known requirements (akka.io).

Should such a proof-of-concept succeed, integrating it into the project would:

  • Simplify the code base, and possibly user experience, given there would be 1 solution instead of 4
  • Modernize our project, by using a new toolkit instead of several older, homegrown solutions
  • Accelerate new development of Uyuni, by leveraging existing know-how of Akka, and not requiring developers to know all current solutions

Failure... is definitely an option!

Learning why this can't work will be as valuable as having it working!

Not learning and not having fun... are NOT options!

That's HackWeek folks!

Hacking details

For asynchronous task handling we currently have:

  • Taskomatic's basic facilities, including one-shot tasks
  • Some specific task facilities (eg. ErrataCacheDriver, SSHPushDriver)
  • Tomcat's MessageQueue facility
  • Tomcat's Salt event handler (PGEventListener)

Open problems that in various degrees affect all of the above are:

  1. enforcing dependencies between asynchronous tasks
  2. enforcing dependencies between asynchronous tasks and other parts of the codebase
  3. establishing priority of tasks
  4. tuning, performance
  5. observability

akka.io seems to have all needed mechanisms:

  1. dependencies can be expressed by actors firing messages to other actors when done
  2. any other part of the code base can ask an actor to do something by sending it a message
  3. Akka has a concept of priority messages to actors
  4. Akka seems to be regarded as high performance, has lots of tuning knobs
  5. Akka has a lot of utilities for monitoring

Moreover Akka can run actors on different processes or even hosts transparently to the code. We could exploit this to improve the Tomcat/Taskomatic interface or even offload some tasks to separate machines.

Battle plan

  • spend one day researching Akka and understanding how it works
  • attack the simplest subsystem first, try to have it working somehow with Akka
  • attack as many other subsystems as possible
  • last half day: prepare demo for Monday : : :

Looking for hackers with the skills:

java scala akka uyuni susemanager fun functionalprogramming

This project is part of:

Hack Week 19


Comments

  • moio
    8 months ago by moio | Reply

    Diary - day 1: learning

    We learned many concepts:

    • some immediately relevant for this HackWeek: Actor, ActorSystem, ActorContext, typed Actor, Actor hierarchy, Behavior, Mailbox, Dispatcher, Routing, at-most-once delivery
    • others not immediately relevant, still interesting: Supervision, Artery, Akka Clustering, Akka Streams, backpressure, the Phi Accrual Failure Detector, Akka Persistence/Event Sourcing, consistent hashing
    • on top of that: some new related tech bits: vavr.io, aeron.io, logback, Lightbend Telemetry, HOCON

    We also played with some "hello world" examples.

  • moio
    8 months ago by moio | Reply

    Diary - day 2: MessageQueue classes

    • "hello world" examples moved inside the Uyuni code base and ran in Uyuni
    • replaced "hello world" examples with code to run three MessageQueue tasks, which work after the port to akka!
    • added reflection-based glue code to automatically register Actors, minimizing boilerplate for next iterations
    • in the process, developed several refactoring patches to simplify our codebase that can be accepted even outside of the scope of this effort (and a bugfix and a contribution to an unrelated PR)
    • learnt about the Reflections library

    Always-up-to-date-Uyuni PR with latest results available

    To try:

    • deploy an Uyuni Server (Master branch)
    • use manager-build.xml to resolve Ivy dependencies and deploy

  • moio
    8 months ago by moio | Reply

    Diary - day 3: traditional MessageQueue is obsoleted

    • created a better mechanism to start asynchronous tasks after a Transaction has ended. Current code requires polling every 10ms!
    • more use cases of MessageQueue were moved to Akka. By end of day only the ones for Salt management remain
    • configuration mechanism added to decide concurrency limits (per message type!)
    • some dead code removed

  • moio
    8 months ago by moio | Reply

    Diary - day 4: MessageQueue and Salt event thread pool are removed

  • moio
    7 months ago by moio | Reply

    Diary - day 5: introduce Akka Clustering

    • cleaned up code and revision history until this point
    • started adding Akka Clustering to the codebase. Objective: move jobs ported to Akka to different nodes!
    • Hello World of Akka Clustering works in Uyuni

  • moio
    7 months ago by moio | Reply

    Diary - HackWeekEnd: (a very first) distributed Uyuni works!

    • we feel like we just can't leave this code alone
    • created a worker node alongside the Uyuni Server
    • flagged the Minion Start Event worker (Akka-ified on day 4) to run in the worker
    • fixed bugs and added hacks until registration works, moving load from the Server to the worker!
    • adding more workers distributes load round-robin!

    Screenshot-from-2020-02-15-22-24-43.png

    Soooo...

    [EVERYTHING IS AWESOME

Similar Projects

Provisioning Prometheus exporters with Uyuni revisited by j_renner

There is a number of annoyances and pending imp...


Alexa on Linux - voice commands for SUSE products by calmeidadeoliveira

What is it about?

Learn about AVS (Alexa Vo...


SUSE Manager: Windows client support by pagarcia

Let's see how much, if any, of the steps descri...


Investigate options to introduce Plugins to SUSE Manager by cbosdonnat

For years we have been discussing the idea to m...


Testing GNU/Linux distributions on Uyuni by juliogonzalezgil

Join the rocket chat channel! [https://chat.su...


Investigate options to introduce Plugins to SUSE Manager by cbosdonnat

For years we have been discussing the idea to m...


Provisioning Prometheus exporters with Uyuni revisited by j_renner

There is a number of annoyances and pending imp...


SUSE Manager: Better feedback for scheduled actions by fkobzik

Motivation

Running async actions in SUSE ...


Investigate options to introduce Plugins to SUSE Manager by cbosdonnat

For years we have been discussing the idea to m...


Testing GNU/Linux distributions on Uyuni by juliogonzalezgil

Join the rocket chat channel! [https://chat.su...