🛡️ Release v1.0.2-rc1: Detection-First AI Scoring & Database Hardening

Release Candidate for the upcoming stable v1.0.2. The architecture and APIs are now frozen — pending real-world validation, this RC will be promoted to the final release. Please test on staging before deploying to production.

This release is the largest functional overhaul since the project began. It refines how Cyber Sentinel reasons about threats, hardens the data layer for long-term operation, and modernizes operator-facing components. The changes touch the AI agent, the MySQL schema, the Vault provisioning workflow, and the alerting pipeline.

🎯 Highlights

• New 1–5 threat scale — replaces the previous 1–10 scale. Score levels are no longer hardcoded in the AI prompt; they are loaded dynamically from the database at every invocation, paving the way for a future self-healing AI workflow.
• URLhaus reweighted as a supporting source — primary scoring now relies on VirusTotal and ThreatFox. URLhaus may add at most +1 to the score, and only when a primary source has already flagged the indicator. This eliminates false positives on legitimate platforms (GitHub, Bitbucket, Pastebin, etc.).
• Partitioned core tables — dns_queries, network_events, and threat_indicators now use monthly RANGE partitioning with automated 6-month retention.
• Color-graded alert emails — score 1–2 renders as green INFO, score 3 as amber REVIEW, score 4–5 as red ALERT. No more red exclamation marks for clean traffic.
• Unified Vault lifecycle playbook — initialization, unsealing, and provisioning are now handled by a single idempotent playbook with pre-flight validation.

🧠 AI Agent — Detection-First Scoring (v3.0)

• Threat scale reduced from 1–10 to 1–5 for clearer operator action mapping:
  • 1 → Allow · 2 → Monitor · 3 → Review · 4 → Block · 5 → Block + Alert
• Scale is now loaded from dic_threat_levels at runtime via the new v_threat_scale_for_agent view. Future workflows can update the scale without touching the prompt.
• Source weighting:
  • Primary: VirusTotal, ThreatFox
  • Supporting: URLhaus (max +1 modifier, never a sole driver)
• Big Player guard hardened: trusted infrastructure (AWS, Cloudflare, Google, Microsoft) is capped at score 2 unless ThreatFox confirms a specific malware family.
• New scoring_rationale field in agent output — explains why the score was assigned, intended as audit input for the future self-healing meta-agent.

🗄️ Database — Schema v3.0

• Threat scale migration: dic_threat_levels rewritten with a 1–5 scale and is_malicious_flag driving downstream logic. Historical scores in ai_analysis_results are preserved via deterministic remapping.
• Composite primary keys on partitioned tables (dns_queries, network_events, threat_indicators) — required by MySQL when partitioning by a non-PK column.
• threat_indicators UNIQUE KEY extended to include last_scan, allowing the same (dns_query, analysis_result) pair to be scanned multiple times over its lifetime.
• Foreign keys removed from partitioned tables (MySQL constraint). Relational integrity is now enforced at the n8n workflow layer.
• Grafana views updated — v_grafana_malicious_stats, v_grafana_daily_trends, v_grafana_threat_explorer, and v_grafana_threat_alerts now use is_malicious_flag instead of a hardcoded score > 5 threshold.

Partitioning & Retention

• Monthly partitions for dns_queries, network_events, threat_indicators.
• Automated maintenance via MySQL Event Scheduler:
  • evt_drop_old_partitions — runs monthly at 02:00, drops partitions older than 6 months.
  • evt_add_future_partitions — runs monthly at 03:00, ensures 3 months of forward partitions exist.
• All maintenance actions logged to partition_maintenance_log (success and failure paths).
• Stored procedures: sp_drop_old_partitions iterates INFORMATION_SCHEMA.PARTITIONS to drop every monthly partition past the 6-month cutoff in a single run; sp_add_future_partitions reorganizes p_future to materialize the next 3 monthly partitions, skipping months that already exist.

📧 Alert Email — Severity-Aware Rendering

• Email styling now adapts to the severity score:
  • Score 1–2 → Green accent, ✅ INFO header, Clean / Monitor badge
  • Score 3 → Amber accent, ⚠️ REVIEW header, Suspicious badge
  • Score 4–5 → Red accent, 🚨 ALERT header, Malicious / Critical badge
• Severity label is now displayed beneath the score for instant context.
• Accent color is consistently applied across the top border, header background, score number, analysis side bar, and action button — no more red ALARM banners for clean traffic.

🔐 Vault — Unified Lifecycle Playbook

• The previously separate 06_1_initialize_vault.yml and 06_2_provision_vault.yml have been merged into a single 06_initialize_provision_vault.yml.
• Pre-flight validation — playbook fails fast if any required variable (API keys, DB passwords, certs) is missing, before any secret is written.
• Idempotent dual-mode operation:
  • First run → initializes Vault, captures fresh credentials, auto-unseals, provisions all secrets.
  • Re-run → detects existing Vault, unseals from group_vars if sealed, updates secrets in place.
• Secure key handling — Unseal Keys are never written to Vault itself (chicken-and-egg problem). They are displayed exactly once at first init and must be saved by the operator to an external secure location.
• Final-message logic: first init shows root token + unseal keys with a save-now warning; re-runs only confirm success without exposing any sensitive material.

⚠️ Migration Notes (1.0.1 → 1.0.2-rc1)

This release contains breaking changes to the database schema. Read this section before upgrading an existing deployment.

Threat Scale: 1–10 → 1–5

If your cyber_intelligence database already contains historical analysis data, the score migration is required. Score remapping:

• Use migration_threat_scale_v3_fixed.sql for in-place migration of existing data.
• For fresh deployments, use db_deployment_v3.sql directly — it includes the new scale.
• Backup your database before running either script.

Schema Changes Requiring Rebuild

The following changes cannot be applied with a simple ALTER on a populated table:

• Composite primary keys on dns_queries, network_events, threat_indicators
• Removal of foreign keys from partitioned tables
• Extension of threat_indicators UNIQUE KEY with last_scan

For environments with existing data, the recommended path is: dump → drop database → recreate with db_deployment_v3.sql → reload data. For fresh deployments, no action needed.

Deployment Order

# 1. Database
mysql -u root -p < db_deployment_v3.sql
mysql -u root -p < db_partitioning_retention_v3_2.sql
2. Vault (idempotent — safe to re-run)
ansible-playbook -i hosts.ini ansible/06_initialize_provision_vault.yml
--vault-password-file ansible/.vault_pass
3. Master playbook (full stack)
ansible-playbook -i hosts.ini ansible/00_main.yml
--vault-password-file ansible/.vault_pass

n8n Workflow Update

The AI agent prompt and email template must be updated in n8n to match the new schema. This is delivered separately — workflow JSON updates will follow shortly after this RC ships.

✅ What's Frozen for Stable v1.0.2

The following are now considered stable and will not change between this RC and the final release (unless a critical bug is found):

• Database schema (tables, views, partitioning strategy)
• AI agent prompt structure and output JSON schema
• Email template variable contracts
• Vault playbook variable names and secret paths

If you find an issue during RC testing, please open a GitHub issue tagged rc-feedback so it can be addressed before promotion to stable.

🐛 Known Issues / Pending

• n8n workflow JSON has not yet been republished — coming in a follow-up commit.
• Self-healing AI meta-agent (auto-tuning of ...

🚀 Release 1.0.1: Streamlined Orchestration & Enhanced Visualization

This release introduces significant improvements to the deployment architecture and the analytical layer of the Cyber AI Sentinel project. The focus was on consolidating Ansible processes and enhancing data presentation within Grafana.

🛠️ Key Features in 1.0.1

• Unified Vault Management: Merged multiple HashiCorp Vault playbooks into a single, optimized .yml file. Secret configuration and access management are now handled through a streamlined workflow.
• Grafana Dashboard v2: Improved visualizations for security metrics. Dashboards are now more intuitive and better integrated with the automated threat reasoning data from the n8n AI pipelines.
• Refined Ansible Logic: Simplified role structure, reducing the deployment time for the core stack (Nginx, Pi-hole, Vault, MySQL, MongoDB).
• Enhanced Secret Handling: Optimized integration between Ansible Vault and HashiCorp Vault for seamless credential provisioning.

⚠️ Critical Deployment Requirements

To successfully deploy this version, ensure the following conditions are met:

1. Unified Playbook Structure: Ensure you are using the new directory structure following the Vault playbook merge.
2. Secrets Provisioning: The ansible/group_vars/all/vault.yml file must be populated with your specific API keys and database passwords.
3. Ansible Vault: A valid .vault_pass file is required to decrypt bootstrap credentials during execution.

📂 Deployment Command

Run the master playbook to deploy the entire stack, including the newly unified Vault module:

# Run the unified master playbook to deploy the entire stack
ansible-playbook -i hosts.ini ansible/00_main.yml --vault-password-file ansible/.vault_pass