There are customer use cases where sharing information via internet or uploading data somewhere is not acceptable for security reasons: this avoid the usage of some tool like the most famous Google Analytics, and prevent developers from understanding how the web application is used by the customers. I don't want to reinvent the wheel and re-implement a copy of Google Analytics, but getting inspired from it, the goal is to reuse information that we already have to extrapolate an analysis of the WebApp customer usage.

I started this project with the aim of learning a programming language where I am not so comfortable yet (python). The purpose of this Hack Week project is to bring this basic tool at a minimal stable and usable state with the purpose of analyze the usage of a WebApp in scenarios where the WebApp is used in an internal network only (offline, disconnected from the internet).

Starting from the current status of the tool at this commit, I'd like to improve it more:

  • fix the patterns finder [DONE]
    • data in UI are badly presented and grouped - [DONE]
    • the algorithm generates a pair of from-to URLs pattern ignoring they comes from a different ip/user, and the data results reflects a non-real pattern actually. This needs to be fixed. [DONE]
  • add filters for the patterns section [DONE]
  • let the table columns to be ordered
  • go through the python backend algorithms and improve [DONE]
  • provide a feature to compare and diff from a given list of URLs (a struts-config.xml for instance) which are the most used and which are never hit [DONE]

Long run roadmap:

  • let the engine keep the history of what has already been read and what not (by date and time? by log file?)
  • provide a simple optional javascript to send an AJAX request with some information (user, date and time, URL, etc) to a configured endpoint
    • this could replace the logic of reading and parsing tomcat logs
    • store this information in the database
    • run the python code against the database instead of tomcat log files

Looking for hackers with the skills:

log python tomcat analyzer analysis statistics web yarn reactjs

This project is part of:

Hack Week 17

Activity

  • almost 2 years ago: dleidi started logorator: an offline internal analytics tool
  • almost 2 years ago: LuNeves liked logorator: an offline internal analytics tool
  • about 2 years ago: dmaiocchi liked logorator: an offline internal analytics tool
  • about 2 years ago: dleidi added keyword "log" to logorator: an offline internal analytics tool
  • about 2 years ago: dleidi added keyword "python" to logorator: an offline internal analytics tool
  • about 2 years ago: dleidi added keyword "tomcat" to logorator: an offline internal analytics tool
  • about 2 years ago: dleidi added keyword "analyzer" to logorator: an offline internal analytics tool
  • about 2 years ago: dleidi added keyword "analysis" to logorator: an offline internal analytics tool
  • about 2 years ago: dleidi added keyword "statistics" to logorator: an offline internal analytics tool
  • about 2 years ago: dleidi added keyword "web" to logorator: an offline internal analytics tool
  • All Activity

    Comments

    Be the first to comment!

    Similar Projects

    Write a commandline client for the geekos by dheidler

    There used to be a tool called tel that would...


    Try to write simple rope-base Python language-server for LSP protocol by mcepl

    Future of tools supporting editors in dealing w...


    openSUSE Leap release process improvements by lkocman

    Goal:

    I'd like to have the release proce...


    Home assistant that doesn't spy on you - developer's edition by DKarakasilis

    There are various home assistant solutions out ...