Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Mixed Workload Stress Tests

Stress tests that exercise all Neumann engines simultaneously with realistic workload patterns.

Test Suite

TestScaleDescription
stress_all_engines_concurrent25K ops/thread, 12 threadsAll engines under concurrent load
stress_realistic_workload30s durationMixed OLTP + search + traversal

Results

TestKey MetricResult
All enginesCombined throughput841 ops/sec
All enginesRelational p5012ms
All enginesGraph p505us
All enginesVector p50< 1us
Realistic workloadMixed throughput232 ops/sec
Realistic workloadRead rate91 reads/sec
Realistic workloadWrite rate68 writes/sec
Realistic workloadSearch rate72 searches/sec

Running

# Run all mixed workload stress tests
cargo test --release -p stress_tests --test mixed_workload_stress -- --ignored --nocapture

# Run specific test
cargo test --release -p stress_tests stress_all_engines_concurrent -- --ignored --nocapture

All Engines Concurrent

Tests all engines (relational, graph, vector) under simultaneous heavy load from 12 threads.

What it validates:

  • Cross-engine concurrency safety
  • Shared TensorStore contention handling
  • No deadlocks or livelocks
  • Correct results under maximum stress

Workload distribution per thread:

  • Relational: INSERT, SELECT, UPDATE
  • Graph: NODE, EDGE, NEIGHBORS
  • Vector: EMBED, SIMILAR

Expected behavior:

  • No panics or assertion failures
  • All operations complete (no hangs)
  • Data consistency verified post-test

Realistic Workload

Simulates a production-like mixed workload over 30 seconds.

What it validates:

  • Sustained throughput over time
  • Memory stability (no leaks)
  • Latency consistency

Workload pattern:

  • 40% Reads (SELECT, GET, NEIGHBORS)
  • 30% Writes (INSERT, UPDATE, NODE)
  • 30% Searches (SIMILAR, PATH)

Expected behavior:

  • Throughput variance < 20%
  • Memory usage stable
  • No degradation over time

Engine Latency Breakdown

EngineOperationTypical p50Notes
RelationalSELECT1-10msSchema lookup overhead
RelationalINSERT3-15msIndex maintenance
GraphNEIGHBORS5-50usAdjacency list lookup
GraphPATH100us-5msScales with path length
VectorEMBED1-5usHash insert
VectorSIMILAR1-100msScales with corpus size

Bottleneck Identification

When mixed workload throughput is lower than expected:

  1. Vector search dominates: Use HNSW index for SIMILAR queries
  2. Relational scans: Add hash/B-tree indexes on filter columns
  3. Graph traversals: Add LIMIT to NEIGHBORS/PATH queries
  4. Contention: Check hot shards with instrumentation

Scaling Considerations

BottleneckSolution
CPU-boundAdd more cores, enable rayon parallelism
Memory-boundEnable tiered storage, use sparse vectors
I/O-boundUse NVMe storage, increase buffer sizes
Network-boundBatch operations, use local cache