CYBERSECURITY / IN-PROGRESS

Smart SIEM — Correlation Engine & UEBA

Turn scattered security events into actionable signals.

STACK
PythonFastAPIRedis StreamsElasticsearchPostgreSQLReactViteDocker ComposeNginx
STATUS

in progress

Smart SIEM event ingestion, correlation and UEBA architecture
01 / CONTEXT

Context

Every information system is constantly exposed to threats: intrusion attempts, account abuse, malware, abnormal insider behaviour. To keep track of their activity, servers, workstations, network devices and applications continuously generate event logs. Taken individually, these events look harmless; taken together and placed in time, they reveal scenarios typical of an attack or unusual behaviour.

However, the volume of these logs and their spread across many sources make manual analysis ineffective. A SOC analyst can neither quickly connect signals from different devices nor single-handedly rebuild the timeline of an incident. A system is therefore needed to centralise this data, reconstruct the sequence of events and identify both known attacks and departures from normal behaviour.

IMG-SIEM-01 · CONTEXT

Smart SIEM — context

Step 01 / 06 · Paused

Servers, hosts and network continuously generate logs.

01 · SITUATION02 · IMPLICATIONS03 · FRICTION04 · NEEDInformation systemSERVERS · HOSTS · NETWORKThreatsINTRUSION · ABUSE · MALWAREContinuous logsSYSLOG · AUTH · FLOWSIsolated signalsHARMLESS ALONEVolume and spread⚠ MULTI-SOURCESManual analysis⚠ LOST TIMELINECentralise and connectDETECT · REBUILD
Paused7 nodes · 7 links · Hover a node
  • Actor / entity
  • Component / data
  • Friction point
  • Need
Hover or select a component to see its role and connections.
From the field to the need: the situation behind the project.
02 / PROBLEM

Problem

How can heterogeneous logs be used to detect scenarios that only emerge across several events, and to spot abnormal behaviour that no static rule describes?

03 / SOLUTION

Solution

Design and development of a smart SIEM platform to turn scattered events into contextualised alerts. The platform collects logs from every source, converts them into a common format and analyses them through two complementary approaches: rule-based correlation, which detects known scenarios, and behavioural analytics (UEBA), which detects departures from a user's or machine's usual activity.

IMG-SIEM-02 · SOLUTION

Smart SIEM — solution

Step 01 / 06 · Paused

Starting point: scattered logs, hidden deviations.

01 · PROBLEM02 · BUILD03 · HOW IT WORKS04 · OUTCOMEScattered logs⚠ HETEROGENEOUSHidden deviations⚠ NO RULESIEM pipelineCOLLECT · NORMALISECorrelation engineTHRESHOLDS · SEQUENCESUEBA moduleBASELINES · DEVIATIONSContextual alerts✓ PRIORITISEDTimeline view✓ INVESTIGATION
Paused7 nodes · 7 links · Hover a node
  • Team module
  • My contribution
  • Friction point
  • Outcome
Hover or select a component to see its role and connections.
Elements in blue are my personal contribution; the rest was built by the project team.
04 / CONTRIBUTION

Contribution

Team project. On the back-end, I designed and developed:

  • the correlation engine, which evaluates rules against normalised events (thresholds, repetitions, time windows, sequences) to link signals that mean little on their own;
  • the UEBA module, which builds each user's or machine's usual profile and flags deviations, such as a login at an unusual hour or an atypical activity volume.
05 / FEATURES

Features

  • Logs from multiple sources are collected and mapped to a common structure.
  • Events from different sources are linked by thresholds, repetitions or sequences.
  • Each user or entity has a baseline profile against which its activity is compared.
  • Detected signals are assessed and prioritised by risk level.
  • Analysts review alerts with their context to carry out investigations.

To verify: actual status of SOAR, ticketing, reporting and some frontend views.

06 / ARCHITECTURE

Architecture

An event emitted by a server or network device is first captured by the collectors, then normalised and enriched. Redis Streams distributes it to the processing stages, while Elasticsearch indexes it for search and history. The correlation engine matches it against rules alongside neighbouring events; the UEBA module compares it with the entity's usual profile. Both results feed the risk assessment, which produces the alert shown to the analyst.

IMG-SIEM-03 · ARCHITECTURE

Smart SIEM — overall architecture

Step 01 / 07 · Paused

Sources emit their logs.

01 · SOURCES02 · INGESTION03 · STORAGE04 · ANALYSIS05 · RESPONSELinux / WindowsAGENTSNetworkFIREWALL · FLOWSApplicationsAPI · AUTHCollectorsSYSLOG · APINormalisationENRICHMENTRedis StreamsBUFFERElasticsearchINDEXCorrelationRULES ENGINEUEBABEHAVIOR SCORINGRisk assessmentRANKINGAlerts / SOC UIFASTAPI · REACT
Paused11 nodes · 11 links · Hover a node
  • Actor / entity
  • Team module
  • My contribution
Hover or select a component to see its role and connections.
Elements in blue are my personal contribution; the rest was built by the project team.
07 / STACK

Stack

Python, FastAPI, Redis Streams, Elasticsearch, PostgreSQL, React, Vite, Docker Compose, Nginx.

08 / BENEFITS

Benefits

  • Scattered events are brought into a coherent, chronological view.
  • Attacks that only show across several events become detectable.
  • Behavioural analysis covers what explicit rules cannot anticipate.
  • Analysts prioritise investigations using the context provided.
EDITORIAL NOTE

This case study distinguishes the overall project scope from my personal contribution. No undocumented result or metric is presented.