TRACE Intelligence, Benchmark & Audit Metrics Report¶
This document records the official benchmark figures, neural intelligence performance, and security audit statistics generated by TRACE (Threat Reconnaissance & Attack-path Correlation Engine) across both the enterprise multi-language benchmark suite (test-repository) and the full-stack production target (TDC-Main).
1. Executive Summary¶
| Metric Dimension | Enterprise Multi-Language Suite (test-repository) |
Full-Stack Production Target (TDC-Main) |
|---|---|---|
| Technology Stacks | Dart Shelf + Kotlin Spring Boot | TypeScript, JavaScript, HTML, CSS, SQL |
| Files Ingested & Analyzed | 15 files | 229 files |
| Languages Detected | 2 (Dart, Kotlin) | 6 (TS, JS, HTML, CSS, SQL, JSON) |
| Attack Surface (Endpoints) | 7 endpoints | 83 endpoints |
| Attack-Path Model (APM) | 28 nodes, 21 edges | 1,202 nodes, 1,283 edges |
| Attack Hypotheses Evaluated | 11 hypotheses | 78 hypotheses |
| Correlated Findings | 11 findings | 77 findings |
| Runtime Exploitation Rate | 11 / 11 (100% CONFIRMED) | Static boundary correlation (offline probe) |
| Vulnerability Severity | Critical: 2 | High: 7 | Med: 2 | Low: 0 | Critical: 13 | High: 64 | Med: 0 | Low: 0 |
| Security Posture Score | 0 / 100 [Grade: F - CRITICAL EXPOSURE] | 0 / 100 [Grade: F - CRITICAL EXPOSURE] |
2. Neural Model Performance & Intelligence Benchmarks¶
TRACE integrates SecureBERT 2.0 (ehsanaghaei/SecureBERT) and a dedicated Laya System 1 Dual-Head Non-Autoregressive Classifier with ONNX Runtime & PyTorch CUDA dual-accelerator support.
2.1 Empirical Head-to-Head Comparison (35 Diverse Enterprise Endpoints)¶
Evaluated across 35 multi-language endpoints spanning Python (FastAPI/Django), Go (Gin), Java (Spring Boot), TypeScript/JavaScript (Express), and Ruby (Rails) across 7 CWE vulnerability families:
| Engine / Model | Architecture | Inference Latency | Identification Accuracy | Primary Superpower / Role |
|---|---|---|---|---|
| Laya System 1 | Dual-Head Encoder (DistilBERT + Linear Heads) |
8.39 ms (GPU) / 20.18 ms (ONNX CPU) | 88.6% Top-1 (97.1% Compound Coverage) | Ultra-fast triage, compound testpack dispatch, APM graph policy decisions |
| SecureBERT 2.0 | Bidirectional AST Encoder (12-layer, 768-dim) | 20.32 ms (GPU) | 22.9% Top-1 | Deep semantic AST code slice inspection, CWE lexical mapping |
| Dual-Engine Ensemble | Cascade + Bayesian Evidence Fusion | 15.23 ms (Ensemble) | 74.3% Balanced | Peak verified accuracy: Laya filters non-critical routes; BERT audits AST sinks |
2.2 Laya System 1 Fine-Tuning Scorecard (Strict Disjoint Enterprise Holdout - Leak-Free)¶
Trained on 471 enterprise APM topologies with strict domain splitting (Validation domains: admin_tenants, fintech, healthcare, iot, webhooks — Zero Data Leakage). All keyword giveaways and target risk tags (SSRF, BFLA, AUTH, INJECTION) were completely purged, and 25% realistic uninstrumented sinks were injected to force authentic semantic learning:
| Epoch | Train Loss | Val Loss (Disjoint) | Priority Accuracy | Testpack Accuracy | Macro Precision | Macro Recall | Macro F1 | Status |
|---|---|---|---|---|---|---|---|---|
| 1 | 2.8169 | 2.6797 | 54.4% | 49.2% | 7.0% | 14.3% | 0.094 | Initializing representation space |
| 2 | 2.2448 | 1.9577 | 57.4% | 66.2% | 31.5% | 38.1% | 0.321 | Learning parameter & route syntax |
| 3 | 1.4929 | 1.4993 | 57.9% | 78.5% | 47.5% | 55.4% | 0.510 | Separating data access from control |
| 4 | 0.8309 | 1.5464 | 56.4% | 79.0% | 45.7% | 56.0% | 0.501 | Boundary stabilization |
| 5 | 0.4219 | 1.3627 | 57.9% | 87.2% | 81.6% | 72.4% | 0.739 | Converged genuine model |
2.3 Hardware Accelerator Benchmarking & Compound Dispatch¶
- NVIDIA RTX 4050 Laptop GPU (CUDA 12.4): Forward-pass latency ~4.44–8.39 ms / endpoint.
- CPU-Only Host (ONNX Runtime ORT_ENABLE_ALL): Forward-pass latency 20.18 ms / endpoint (2.5x faster than CPU PyTorch).
- Multi-Label Compound Vulnerability Output: Returns calibrated softmax probabilities across all 7 families with thresholding ($P_i \ge 0.15$) and APM graph topological invariant detection (e.g. unauthenticated state changes dispatch both
authenticationandbola/injection), achieving 97.1% compound coverage on the enterprise benchmark.
3. Multi-Language Enterprise Audit (test-repository)¶
An enterprise-grade Clean Architecture testbed was audited at runtime via live target http://127.0.0.1:18082.
Ingestion & Graph Topology¶
- Files Analyzed: 15 files across
lib/(Dart) andsrc/main/kotlin/(Kotlin) - Endpoints Extracted: 7 endpoints (5 Dart Shelf, 2 Spring Boot Kotlin)
- APM Graph: 28 nodes, 21 directed edges
- Hypotheses Formulated: 11 security hypotheses
Runtime Exploitation Results (100% Confirmation)¶
Every derived hypothesis was proven exploitable against the running target using automated test packs:
| Rank | Priority | Finding ID | Category | Endpoint | Location | Exploit Latency | Confirmed Attack Proof |
|---|---|---|---|---|---|---|---|
| 1 | CRITICAL | TR-AUTH-008 |
AUTH | GET /api/v2/records/{recordId} |
RecordController.kt:15 |
31.23 ms | Unauthenticated GET returned HTTP 200 with sensitive audit records. |
| 2 | CRITICAL | TR-AUTH-010 |
AUTH | POST /api/v2/records/export/dispatch |
RecordController.kt:33 |
24.11 ms | Unauthenticated state-modifying POST succeeded without auth guard. |
| 3 | HIGH | TR-BFLA-001 |
BFLA | GET /api/v1/organizations/{orgId}/projects/{projectId} |
lib/api/routes.dart:14 |
11.51 ms | Regular user accessed administrative tenant project data without 403. |
| 4 | HIGH | TR-BFLA-002 |
BFLA | POST /api/v1/admin/users/{userId}/role |
lib/api/routes.dart:18 |
32.18 ms | Non-admin caller elevated role to SUPER_ADMIN with HTTP 200. |
| 5 | HIGH | TR-BFLA-004 |
BFLA | POST /api/v1/integrations/webhooks/preview |
lib/api/routes.dart:22 |
16.64 ms | Non-admin identity dispatched integration webhook actions. |
| 6 | HIGH | TR-BFLA-005 |
BFLA | POST /api/v1/billing/invoices/{invoiceId}/refund |
lib/api/routes.dart:26 |
15.97 ms | Financial refund executed under standard user credentials without 403. |
| 7 | HIGH | TR-BFLA-006 |
BFLA | PUT /api/v1/users/profile |
lib/api/routes.dart:30 |
21.40 ms | Privileged role parameter modification allowed under standard token. |
| 8 | HIGH | TR-SSRF-009 |
SSRF | GET /api/v2/records/{recordId} |
RecordController.kt:15 |
22.10 ms | Primary key direct lookup returned confidential audit records. |
| 9 | HIGH | TR-SSRF-011 |
SSRF | POST /api/v2/records/export/dispatch |
RecordController.kt:33 |
30.04 ms | RestTemplate dispatched outbound HTTP request without IP filtering. |
| 10 | MEDIUM | TR-MASS-003 |
MASS | POST /api/v1/admin/users/{userId}/role |
lib/api/routes.dart:18 |
14.96 ms | Bound administrative parameters (is_admin, role) directly. |
| 11 | MEDIUM | TR-MASS-007 |
MASS | PUT /api/v1/users/profile |
lib/api/routes.dart:30 |
21.90 ms | Raw payload binding injected elevated role into database model. |
4. Production Repository Audit (TDC-Main)¶
- Repository Path:
D:\Startup\TDC-Main - Technology Stack: React Router / Remix, TypeScript, Node.js, Prisma/SQL
- Total Files Scanned: 229 files
- Language Distribution:
- TypeScript / TSX: 142 files
- JavaScript / JSX: 24 files
- CSS / Styling: 18 files
- HTML: 6 files
- JSON / Configs: 27 files
- SQL: 12 files
- Endpoints Extracted: 83 endpoints
- Attack-Path Model Complexity:
- Graph Nodes: 1,202 nodes
- Directed Edges: 1,283 edges
- Attack Hypotheses: 78 hypotheses
- Correlated Findings: 77 findings
- Critical (P0): 13 findings
- High (P1): 64 findings
- Medium (P2): 0 findings
- Low (P3): 0 findings
- AI Analysis Duration: 0.94 seconds total (batched across 83 endpoints)
- Overall Security Score: 0 / 100 (Grade F - Critical Exposure)
5. Test Suite & Validation Metrics¶
Running the full pytest suite verifies zero regressions across all framework adapters, intelligence batchers, edge cases, and CLI workflows:
Module Breakdown:¶
| Test File | Tests Passed | Duration | Focus Area |
|---|---|---|---|
tests/test_adapters.py |
7 / 7 | 0.85s | Dart Shelf, Spring Boot Kotlin, Go Mux/Gin, Next.js, React Router |
tests/test_bugs_and_edge_cases.py |
7 / 7 | 0.42s | ScopeGuard limits, Correlator edge cases, empty AST handling |
tests/test_intelligence.py |
3 / 3 | 5.20s | SecureBERT 2.0 batch inference, Laya System 1 decisions |
tests/test_trace.py |
6 / 6 | 1.58s | Full CLI workflow, ingestion, graph construction, hypothesis generation |
| Total Test Suite | 23 / 23 | 8.05s | 100% Pass Rate |
6. CLI Reproduction & Verification Commands¶
All metrics and findings in this report can be verified locally using the following commands:
# 1. Run unit & integration test suite
.\.venv\Scripts\pytest tests\
# 2. Execute comprehensive audit on enterprise test repository
.\trace.bat test-all "D:\Startup\TRACE\test-repository" --target http://127.0.0.1:18082
# 3. Execute comprehensive audit on full-stack production repo
.\trace.bat test-all "D:\Startup\TDC-Main"
# 4. Deep-dive into root cause analysis for any finding
.\trace.bat explain TR-BFLA-002 "D:\Startup\TRACE\test-repository"
# 5. Replay exact HTTP exploitation proof
.\trace.bat replay TR-BFLA-002 "D:\Startup\TRACE\test-repository"