Package ai.attackframework.tools.burp.utils

Class ExportStats

java.lang.Object
ai.attackframework.tools.burp.utils.ExportStats
public final class ExportStats extends Object
Thread-safe per-index export stats for OpenSearch pushes.

Session-scoped: counts and last error/duration are not persisted. Used by StatsPanel and by the Exporter-index stats snapshot. Index keys align with short names: traffic, exporter, settings, sitemap, findings.

  • Field Details

    • SKIP_REASON_SCOPE

      public static final String SKIP_REASON_SCOPE
      Skip reason: document was dropped by the user's Burp scope filter.
      See Also:

    • SKIP_REASON_TOOL_DISABLED

      public static final String SKIP_REASON_TOOL_DISABLED
      Skip reason: document originated from a Burp tool source the user did not enable for export.
      See Also:

    • SKIP_REASON_SELF_OPENSEARCH

      public static final String SKIP_REASON_SELF_OPENSEARCH
      Skip reason: request targets the configured OpenSearch destination (self-export guard).
      See Also:

  • Method Details

    • getIndexKeys

      public static List<String> getIndexKeys()
      Returns the list of index keys (traffic, exporter, settings, sitemap, findings).

    • getTrafficSourceKeys

      public static List<String> getTrafficSourceKeys()
      Returns known traffic source keys used for source-level traffic stats.

    • getTrafficToolTypeKeys

      public static List<String> getTrafficToolTypeKeys()
      Returns known traffic tool type keys shown in Config > Traffic.

    • getRepeaterMetadataSourceKeys

      public static List<String> getRepeaterMetadataSourceKeys()
      Returns known live-Repeater metadata source keys used in trace and stats summaries.

    • recordSuccess

      public static void recordSuccess(String indexKey, long count)
      Records successful document push(es) for the given index.
      Parameters:
      indexKey - index key (e.g. "traffic")
      count - number of documents; ignored if <= 0

    • recordFailure

      public static void recordFailure(String indexKey, long count)
      Records failed document push(es) for the given index.
      Parameters:
      indexKey - index key (e.g. "traffic")
      count - number of failures; ignored if <= 0

    • recordTrafficSourceSuccess

      public static void recordTrafficSourceSuccess(String sourceKey, long count)
      Records successful traffic pushes for a specific traffic source.
      Parameters:
      sourceKey - source key (for example "proxy_live_http")
      count - number of successful documents; ignored if <= 0

    • recordTrafficSourceFailure

      public static void recordTrafficSourceFailure(String sourceKey, long count)
      Records failed traffic pushes for a specific traffic source.
      Parameters:
      sourceKey - source key (for example "proxy_live_http")
      count - number of failed documents; ignored if <= 0

    • recordLastPush

      public static void recordLastPush(String indexKey, long durationMs)
      Records the duration in ms of the last push for the given index.
      Parameters:
      indexKey - index key
      durationMs - duration in milliseconds, or -1 if unknown

    • recordLastError

      public static void recordLastError(String indexKey, String message)
      Records the last error for the given index.
      Parameters:
      indexKey - index key
      message - error message; null or empty clears the stored error

    • getSuccessCount

      public static long getSuccessCount(String indexKey)
      Returns the session total of documents successfully pushed for the given index.

    • getFailureCount

      public static long getFailureCount(String indexKey)
      Returns the session total of failed push attempts for the given index.

    • getExportedBytes

      public static long getExportedBytes(String indexKey)
      Returns estimated successful payload bytes pushed for the given index this session.

    • recordExportedBytes

      public static void recordExportedBytes(String indexKey, long bytes)
      Records estimated successful payload bytes for the given index.
      Parameters:
      indexKey - index key (e.g. "traffic")
      bytes - estimated successful payload bytes; ignored if <= 0

    • getTrafficSourceSuccessCount

      public static long getTrafficSourceSuccessCount(String sourceKey)
      Returns successful traffic pushes for a specific traffic source.

    • getTrafficSourceFailureCount

      public static long getTrafficSourceFailureCount(String sourceKey)
      Returns failed traffic pushes for a specific traffic source.

    • recordTrafficToolTypeSuccess

      public static void recordTrafficToolTypeSuccess(String toolTypeKey, long count)
      Records successful OpenSearch traffic pushes for a specific tool type.

    • recordTrafficToolTypeFailure

      public static void recordTrafficToolTypeFailure(String toolTypeKey, long count)
      Records failed OpenSearch traffic pushes for a specific tool type.

    • getTrafficToolTypeSuccessCount

      public static long getTrafficToolTypeSuccessCount(String toolTypeKey)
      Returns successful OpenSearch traffic push count for a specific tool type.

    • getTrafficToolTypeFailureCount

      public static long getTrafficToolTypeFailureCount(String toolTypeKey)
      Returns failed OpenSearch traffic push count for a specific tool type.

    • recordRepeaterMetadataSource

      public static void recordRepeaterMetadataSource(String metadataSource)
      Records one live-Repeater metadata source decision.

      These counters mirror the trace-level metadataSource vocabulary so the Stats panel and exporter stats snapshots can summarize which correlation paths are doing the work during a run.

    • getRepeaterMetadataSourceCount

      public static long getRepeaterMetadataSourceCount(String metadataSource)
      Returns the session total for one live-Repeater metadata source label.

    • describeRepeaterMetadataSourceCounts

      public static String describeRepeaterMetadataSourceCounts()
      Formats live-Repeater metadata source counters for compact UI and stats summaries.

    • getLastPushDurationMs

      public static long getLastPushDurationMs(String indexKey)
      Returns last push duration in ms, or -1 if not set.

    • getLastError

      public static String getLastError(String indexKey)
      Returns the last recorded error message for the given index, or null if none.

    • getQueueSize

      public static int getQueueSize(String indexKey)
      Returns the number of documents currently queued for retry for the given index (0 when no retry coordinator or queue empty).

    • recordTrafficQueueDrop

      public static void recordTrafficQueueDrop(long count)
      Records that one or more documents were dropped from the traffic queue (queue full, drop oldest).
      Parameters:
      count - number of documents dropped; ignored if <= 0

    • getTrafficQueueDrops

      public static long getTrafficQueueDrops()
      Returns the session total of documents dropped from the traffic queue because it was full.
      Returns:
      total drop count (0 or positive)

    • recordTrafficSpillEnqueued

      public static void recordTrafficSpillEnqueued(long count)
      Records one or more traffic documents persisted to spill storage.
      Parameters:
      count - number of spilled documents; ignored if <= 0

    • getTrafficSpillEnqueued

      public static long getTrafficSpillEnqueued()
      Returns total spilled traffic documents persisted this session.

    • recordTrafficSpillDequeued

      public static void recordTrafficSpillDequeued(long count)
      Records one or more traffic documents drained from spill storage.
      Parameters:
      count - number of drained spilled documents; ignored if <= 0

    • getTrafficSpillDequeued

      public static long getTrafficSpillDequeued()
      Returns total spilled traffic documents drained back into memory this session.

    • recordTrafficSpillDrop

      public static void recordTrafficSpillDrop(long count)
      Records one or more traffic documents dropped because spill storage was unavailable/full.
      Parameters:
      count - number of dropped spill documents; ignored if <= 0

    • getTrafficSpillDrops

      public static long getTrafficSpillDrops()
      Returns total traffic documents dropped due to spill rejection this session.

    • recordTrafficSpillRecovered

      public static void recordTrafficSpillRecovered(long count)
      Records one or more spill documents recovered on startup.
      Parameters:
      count - recovered spill document count; ignored if <= 0

    • getTrafficSpillRecovered

      public static long getTrafficSpillRecovered()
      Returns total spill documents recovered on startup this session.

    • recordTrafficSpillExpiredPruned

      public static void recordTrafficSpillExpiredPruned(long count)
      Records one or more spill files removed by retention cleanup.
      Parameters:
      count - pruned spill file count; ignored if <= 0

    • getTrafficSpillExpiredPruned

      public static long getTrafficSpillExpiredPruned()
      Returns total spill files pruned by retention this session.

    • recordTrafficDropReason

      public static void recordTrafficDropReason(String reason, long count)
      Records a reason-coded traffic drop event.
      Parameters:
      reason - non-blank reason key
      count - number of dropped documents for that reason

    • getTrafficDropReasonCount

      public static long getTrafficDropReasonCount(String reason)
      Returns the total for one reason-coded traffic drop key (0 when absent).

    • recordTrafficToolSourceFallback

      public static void recordTrafficToolSourceFallback()
      Records a response-path export decision that used request-side tool-type fallback.

      This is used for observability when response tool source is absent but request-side correlation by message id still allows correct traffic export gating.

    • getTrafficToolSourceFallbacks

      public static long getTrafficToolSourceFallbacks()
      Returns total response-path tool-source fallbacks recorded this session.

    • recordSynthesizedBodyParamsDropped

      public static void recordSynthesizedBodyParamsDropped(long count)
      Records synthesized BODY-typed parameters that were filtered out on binary request bodies.

      Feeds the Synthesized Body Params Dropped telemetry counter; ignored when count <= 0.

    • getSynthesizedBodyParamsDropped

      public static long getSynthesizedBodyParamsDropped()
      Returns the session total of synthesized BODY parameters dropped across all exports.

    • recordDocsOverParamsThreshold

      public static void recordDocsOverParamsThreshold()
      Records one document whose retained or dropped-synthesized parameter count tripped the high-cardinality WARN threshold.

    • getDocsOverParamsThreshold

      public static long getDocsOverParamsThreshold()
      Returns the session total of documents that tripped the parameter-cardinality threshold.

    • recordSkippedBodyParameterEnumeration

      public static void recordSkippedBodyParameterEnumeration()
      Records one document where the typed-accessor fast path was taken to avoid Burp's synthetic BODY enumeration. Bumped exactly once per qualifying document in RequestResponseDocBuilder.collectParameters(burp.api.montoya.http.message.requests.HttpRequest, boolean).

    • getDocsWithSkippedBodyEnumeration

      public static long getDocsWithSkippedBodyEnumeration()
      Returns the session total of documents that took the typed-accessor parameter fast path.

    • recordOpenSearchSuccess

      public static void recordOpenSearchSuccess()
      Records a successful OpenSearch push. Updates the connection-health timestamp and resets the consecutive-failure counter so the panel can surface live destination health.

    • recordOpenSearchFailure

      public static void recordOpenSearchFailure()
      Records a failed OpenSearch push. Increments the consecutive-failure counter without disturbing the last-success timestamp so the panel shows both values side by side.

    • getOpenSearchLastSuccessAtMs

      public static long getOpenSearchLastSuccessAtMs()
      Returns the epoch-ms timestamp of the most recent successful OpenSearch push, or -1.

    • getOpenSearchConsecutiveFailures

      public static long getOpenSearchConsecutiveFailures()
      Returns the count of consecutive OpenSearch push failures since the last success.

    • recordSkipReason

      public static void recordSkipReason(String reason, long count)
      Records a silent skip of a document at an exporter filter (scope, tool-source, self-export, etc.).

      Surfaces in the Misc Stats Skips by Reason row so the UI can show where coverage gaps come from. Ignores null, blank, or non-positive counts.

    • getSkipReasonCount

      public static long getSkipReasonCount(String reason)
      Returns the session total for one skip-reason key (0 when absent).

    • getSkipReasonCounts

      public static Map<String,Long> getSkipReasonCounts()
      Returns a live copy of the skip-reason counters.

      Keys are the reason labels ("scope", "tool_disabled", "self_opensearch", etc.); values are the counts. The returned map is a snapshot and safe to iterate without synchronization.

    • getTotalSkipCount

      public static long getTotalSkipCount()
      Returns the session total across all skip reasons.

    • recordBulkStart

      public static void recordBulkStart()
      Marks the start of a bulk request. Pair with recordBulkEnd() in a finally block. Increments the live Bulk Requests In-Flight counter surfaced on the Misc Stats card.

      Prefer openBulk() in new code so the increment / decrement cannot drift apart on early-return or exceptional paths.

    • recordBulkEnd

      public static void recordBulkEnd()
      Marks the end of a bulk request. Never drops below zero even if callers decrement more than they increment, so misuse cannot produce misleading negative readings.

    • openBulk

      public static ExportStats.BulkInFlightTicket openBulk()
      Opens a ExportStats.BulkInFlightTicket that increments the in-flight counter immediately and decrements it on AutoCloseable.close(). Use with try-with-resources: 
      
 try (ExportStats.BulkInFlightTicket ignored = ExportStats.openBulk()) {
     return executeRequest(...);
 }

      Equivalent to recordBulkStart() + recordBulkEnd() in a finally block, but enforces pairing at the type level so an early return or unhandled exception cannot leave the counter elevated.

    • getBulkInFlight

      public static int getBulkInFlight()
      Returns the current count of bulk requests in flight.

    • getOldestQueuedAgeMs

      public static long getOldestQueuedAgeMs(String indexKey)
      Returns the age (ms) of the oldest document currently queued for the given index, or -1 when the queue is empty. Surfaces in the Misc Stats Oldest Queued Age row.

    • getTotalSuccessCount

      public static long getTotalSuccessCount()
      Session total: sum of docs pushed across all indexes.

    • getTotalFailureCount

      public static long getTotalFailureCount()
      Session total: sum of failures across all indexes.

    • getTotalExportedBytes

      public static long getTotalExportedBytes()
      Session total: sum of estimated successful payload bytes across all indexes.

    • recordRetryQueueDrop

      public static void recordRetryQueueDrop(String indexKey, long count)
      Per-index retry queue: count of documents dropped when the retry queue was full.

    • getRetryQueueDrops

      public static long getRetryQueueDrops(String indexKey)
      Returns the session total of documents dropped from the retry queue for the given index.

    • getTotalRetryQueueDrops

      public static long getTotalRetryQueueDrops()
      Returns the session total of documents dropped from retry queues across all indexes.

    • recordPermanentDrop

      public static void recordPermanentDrop(String indexKey, long count)
      Per-index permanent-failure drops: count of documents rejected by OpenSearch with a mapping/parse/validation error (classified permanent) and therefore not re-queued for retry.

      Distinct from recordRetryQueueDrop(String, long) which counts capacity-driven drops. The poison-pill path in IndexingRetryCoordinator short-circuits permanently rejected items and records them here.

    • getPermanentDrops

      public static long getPermanentDrops(String indexKey)
      Returns the session total of permanently dropped documents for the given index.

    • getTotalPermanentDrops

      public static long getTotalPermanentDrops()
      Returns the session total of permanently dropped documents across all indexes.

    • getThroughputDocsPerSecLast60s

      public static double getThroughputDocsPerSecLast60s()
      Returns documents per second over the last 60 seconds (rolling throughput). Prunes entries older than the window. Thread-safe.
      Returns:
      docs/sec (0.0 or positive)

    • getThroughputDocsPerSecLast10s

      public static double getThroughputDocsPerSecLast10s()
      Returns documents per second over the last 10 seconds (rolling throughput). Prunes entries older than the window. Thread-safe.
      Returns:
      docs/sec (0.0 or positive)

    • recordExportStartRequested

      public static void recordExportStartRequested()
      Records that a new export start was requested.

      Resets startup latency tracking so the next successful traffic push can produce an updated start-to-first-traffic metric.

    • getStartToFirstTrafficMs

      public static long getStartToFirstTrafficMs()
      Returns milliseconds from the latest export Start request to the first successful traffic push, or -1 when not available yet.

    • getExportStartRequestedAtMs

      public static long getExportStartRequestedAtMs()
      Returns timestamp (epoch ms) for latest export Start request, or -1.

    • resetForTests

      public static void resetForTests()
      Resets process-local export stats. Intended for test teardown only.

    • recordProxyHistorySnapshot

      public static void recordProxyHistorySnapshot(int attempted, int success, long durationMs, int finalChunkTarget)
      Records summary metrics for the latest proxy-history snapshot push.
      Parameters:
      attempted - attempted document count
      success - successful document count
      durationMs - wall-clock duration in milliseconds
      finalChunkTarget - final chunk target doc count at end of run

    • getLastProxyHistorySnapshot

      public static ExportStats.ProxyHistorySnapshotStats getLastProxyHistorySnapshot()
      Returns the latest proxy-history snapshot stats, or null when none recorded.