Upload files to "/"
commit
0fbee0dd7e
|
|
@ -0,0 +1,72 @@
|
||||||
|
# ThreatHunter
|
||||||
|
|
||||||
|
ThreatHunter is a **SIEM‑lite security visibility tool** built for small and mid‑sized businesses.
|
||||||
|
|
||||||
|
It focuses on one simple question:
|
||||||
|
|
||||||
|
> *“What suspicious outbound connections are my computers making?”*
|
||||||
|
|
||||||
|
Without:
|
||||||
|
- Heavy agents
|
||||||
|
- Constant data streaming
|
||||||
|
- Enterprise SIEM complexity
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## What ThreatHunter Does
|
||||||
|
|
||||||
|
- Collects Windows Firewall / WFP connection events
|
||||||
|
- Detects connections to known‑bad IPs using reputation data
|
||||||
|
- Processes data **locally on the endpoint**
|
||||||
|
- Uploads only confirmed security hits
|
||||||
|
- Automatically creates **Security alerts in Syncro**
|
||||||
|
- Provides a clean web UI for technicians and customers
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Why It Exists
|
||||||
|
|
||||||
|
Most SMBs:
|
||||||
|
- Have no visibility into outbound traffic
|
||||||
|
- Can’t justify a full SIEM or EDR
|
||||||
|
- Still want to *see the crazy stuff* touching their machines
|
||||||
|
|
||||||
|
ThreatHunter is designed to be:
|
||||||
|
- Free or low‑cost
|
||||||
|
- Easy to deploy via Syncro
|
||||||
|
- Useful as both a security tool and an educational aid
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Architecture Overview
|
||||||
|
|
||||||
|
**Endpoint**
|
||||||
|
- PowerShell script
|
||||||
|
- Runs on a schedule via Syncro
|
||||||
|
- Buffers and correlates locally
|
||||||
|
- Hashes executables only on suspicious hits
|
||||||
|
|
||||||
|
**Server**
|
||||||
|
- VPS‑hosted API + database
|
||||||
|
- Receives hit data
|
||||||
|
- Creates Syncro Security tickets
|
||||||
|
- Hosts ThreatHunter web portal
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
🚧 Early development / v1 build phase
|
||||||
|
|
||||||
|
See `project.md` for full technical and architectural details.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Disclaimer
|
||||||
|
|
||||||
|
ThreatHunter is **not** a replacement for:
|
||||||
|
- EDR
|
||||||
|
- Full SIEM
|
||||||
|
- IDS/IPS
|
||||||
|
|
||||||
|
It is a **visibility and awareness tool** designed to surface suspicious behavior early and guide next steps.
|
||||||
|
|
@ -0,0 +1,176 @@
|
||||||
|
# ThreatHunter — SIEM‑Lite (project.md)
|
||||||
|
|
||||||
|
## Overarching Goals (Read This First)
|
||||||
|
|
||||||
|
ThreatHunter is a **SIEM‑lite visibility and reputation‑correlation system** designed to:
|
||||||
|
1. Run with **minimal endpoint footprint** (CPU, disk, memory).
|
||||||
|
2. Avoid constant streaming or always‑on services on endpoints.
|
||||||
|
3. **Process data locally** and upload only meaningful security signals (“hits”).
|
||||||
|
4. Integrate directly into **Syncro** as a first‑class **Security** alert/ticket.
|
||||||
|
5. Provide clear value to SMB customers while remaining technician‑grade internally.
|
||||||
|
|
||||||
|
Non‑Goals (v1):
|
||||||
|
- Full packet capture or IDS/IPS
|
||||||
|
- Replacing EDR or enterprise SIEMs
|
||||||
|
- Continuous real‑time streaming agents
|
||||||
|
- Deep behavioral analytics
|
||||||
|
|
||||||
|
Positioning:
|
||||||
|
> “See the suspicious outbound traffic your computers are making — without installing heavy security software.”
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## High‑Level Architecture
|
||||||
|
|
||||||
|
**Endpoint (PowerShell, scheduled via Syncro)**
|
||||||
|
- Collects Windows Firewall / WFP events (short‑lived connection friendly)
|
||||||
|
- Buffers events locally (30–60 minutes)
|
||||||
|
- Performs local correlation against a reputation hotlist
|
||||||
|
- Hashes executables **only on confirmed hits**
|
||||||
|
- Uploads only hit payloads to VPS
|
||||||
|
|
||||||
|
**VPS (ThreatHunter Server)**
|
||||||
|
- Receives hit payloads
|
||||||
|
- Stores and correlates hits per tenant/device
|
||||||
|
- Creates **Syncro Security tickets/alerts**
|
||||||
|
- Hosts ThreatHunter Web UI (tech + customer)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Endpoint Design
|
||||||
|
|
||||||
|
### Execution Model
|
||||||
|
- PowerShell script
|
||||||
|
- Runs every 5 minutes via Syncro
|
||||||
|
- Exits after each run (no persistent service)
|
||||||
|
|
||||||
|
### Data Source
|
||||||
|
- Windows Filtering Platform / Firewall events from Security Event Log
|
||||||
|
- Captures allowed/blocked connection events including short‑lived connections
|
||||||
|
|
||||||
|
### Local Storage Layout
|
||||||
|
```
|
||||||
|
C:\ProgramData\ThreatHunter\
|
||||||
|
├── config.json
|
||||||
|
├── state.json
|
||||||
|
├── buffer\
|
||||||
|
│ └── events_YYYYMMDD_HH.jsonl
|
||||||
|
├── cache\
|
||||||
|
│ └── hashcache.json
|
||||||
|
└── logs\
|
||||||
|
└── threathunter.log
|
||||||
|
```
|
||||||
|
|
||||||
|
### Buffering Rules
|
||||||
|
- Append‑only JSONL files
|
||||||
|
- Retain 30–60 minutes of data
|
||||||
|
- Prune by time, not size
|
||||||
|
|
||||||
|
### Processing Window
|
||||||
|
- Correlate when:
|
||||||
|
- Oldest buffered event ≥ 30 minutes
|
||||||
|
- OR buffer size threshold reached
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Local Correlation Logic
|
||||||
|
|
||||||
|
1. Read buffered events
|
||||||
|
2. Extract **unique remote IPs**
|
||||||
|
3. Load local hotlist into memory (once per run)
|
||||||
|
4. Match remote IPs against hotlist
|
||||||
|
5. For matches:
|
||||||
|
- Group by `(remote_ip, process_name)`
|
||||||
|
- Calculate first/last seen and count
|
||||||
|
6. Enrich:
|
||||||
|
- Resolve executable path (best‑effort)
|
||||||
|
- Compute SHA256 hash **only on hit**
|
||||||
|
- Use local hash cache to avoid re‑hashing
|
||||||
|
7. Upload hit payload(s) to VPS
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Hotlist Strategy
|
||||||
|
|
||||||
|
- Approx. 80,000 IPv4 addresses
|
||||||
|
- Stored locally as `hotlist.json.gz`
|
||||||
|
- Loaded into memory once per processing run
|
||||||
|
- Updated periodically from VPS
|
||||||
|
- Integrity verified via hash/signature
|
||||||
|
- TTL enforced (fallback to server confirmation if stale)
|
||||||
|
|
||||||
|
Future upgrade:
|
||||||
|
- Bloom filter + server confirmation
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Hit Payload Schema (Endpoint → VPS)
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"tenant_id": "...",
|
||||||
|
"device_id": "...",
|
||||||
|
"hostname": "...",
|
||||||
|
"agent_version": "1.0",
|
||||||
|
"hotlist_version": "2025‑01‑01",
|
||||||
|
"hits": [
|
||||||
|
{
|
||||||
|
"remote_ip": "1.2.3.4",
|
||||||
|
"remote_port": 443,
|
||||||
|
"protocol": "tcp",
|
||||||
|
"process_name": "bad.exe",
|
||||||
|
"pid": 1234,
|
||||||
|
"exe_path": "C:\\Path\\bad.exe",
|
||||||
|
"sha256": "...",
|
||||||
|
"first_seen_utc": "...",
|
||||||
|
"last_seen_utc": "...",
|
||||||
|
"hit_count": 42,
|
||||||
|
"sample_events": []
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## VPS Responsibilities
|
||||||
|
|
||||||
|
### API
|
||||||
|
- `POST /api/v1/hits`
|
||||||
|
- Authenticate tenant/device
|
||||||
|
- Store hits in database
|
||||||
|
|
||||||
|
### Syncro Integration
|
||||||
|
- On new hit:
|
||||||
|
- Create or update **Syncro ticket**
|
||||||
|
- Category: **Security**
|
||||||
|
- Tags: `Security`, `ThreatHunter`
|
||||||
|
- Deduplicate tickets using:
|
||||||
|
`(tenant_id, device_id, remote_ip, process_name, sha256)` within time window
|
||||||
|
|
||||||
|
### Web UI
|
||||||
|
- Technician view: all tenants, full detail
|
||||||
|
- Customer view: tenant‑only, simplified fields
|
||||||
|
- Key pages:
|
||||||
|
- Dashboard
|
||||||
|
- Hit list
|
||||||
|
- Hit detail
|
||||||
|
- Device detail
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Milestones
|
||||||
|
|
||||||
|
**M1** — Endpoint pipeline works end‑to‑end
|
||||||
|
**M2** — VPS API + DB + basic UI
|
||||||
|
**M3** — Syncro Security ticket creation
|
||||||
|
**M4** — Tuning, allowlists, suppression, polish
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Definition of Done (v1)
|
||||||
|
|
||||||
|
- Endpoint impact is negligible
|
||||||
|
- Only hit data is uploaded
|
||||||
|
- Syncro Security tickets created reliably
|
||||||
|
- Techs can triage from Syncro and drill into ThreatHunter UI
|
||||||
Loading…
Reference in New Issue