Sentova MTD

What it is

Sentova MTD is a new anti-spyware / Mobile Threat Defense microservice (Go package mtd, module github.com/sentova/mtd) inside the larger Sentova platform of Go microservices behind an HMAC-authenticated gateway. It does server-side forensic analysis of uploaded mobile artifacts — iOS sysdiagnose/backup, Android bugreport — matched against a STIX 2.1 spyware-IOC feed, producing a deterministic verdict. Following the platform's DB-per-service rule it owns an isolated sentova_mtd database, and it mirrors the sibling surface service's models/store/service/api/authz layout and the fieldcrypt.WithCipher boot pattern.

Collection	Holds	Sensitivity
mtd_indicators	STIX 2.1 indicators + lifted observables array	Curated intel (clear)
mtd_feeds	IOC feed registry + monotonic snapshotVersion	Metadata (clear)
mtd_artifacts	Uploaded forensic bundle record	High — PII, field-encrypted
mtd_verdicts	Analysis result + matches	Mixed — threat names clear, device values encrypted

The core idea: index seek, not pattern parse

A STIX 2.1 pattern is a grammar, not a key — matching thousands of raw patterns per observed value is O(N) in the feed. So at ingest, MTD lifts every concrete {kind, value} comparison out of the STIX pattern once into a denormalized, indexed observables array (the raw pattern is retained only for provenance). The match hot path becomes a seek over a multikey index:

// The match hot path: multikey over the embedded observables array. value
// leads (high-cardinality selector); kind narrows collisions.
{Keys: bson.D{
  {Key: "observables.value", Value: 1},
  {Key: "observables.kind", Value: 1},
}},

The observable kinds are normalized categories — file hashes, network indicators (domain / ipv4 / url), process names, provisioning-profile ids — mapped from STIX object paths at ingest. The mapping is intentionally asymmetric in one place: a STIX field is left unmapped where the artifact side cannot supply a comparable value, because mapping it would silently read a real infection as clean. Family resolves to pegasus | predator | reign | other, and mercenary-spyware IOCs default to critical.

Two-phase match, and the bug it avoids

Matching is a batched DB pre-filter followed by an in-memory confirm, and its correctness turns on a subtle multikey trap:

• DB pre-filter (one query, not one-per-observable). A single scoped query pre-filters on value using the multikey index; the $elemMatch binds the value match to a single array element, and kind is deliberately not constrained here.

cur, err := s.indicators.Find(ctx, bson.M{
    "accountId":   bson.M{"$in": accountIDs},
    "observables": bson.M{"$elemMatch": bson.M{"value": bson.M{"$in": values}}},
})

• In-memory confirm. The analyzer keeps only indicators at the feed's active snapshot version and confirms an exact per-element (value, kind) hit. The naive combined-key form {"observables.value": v, "observables.kind": k} is explicitly wrong on a multikey index: it can match v in one array element and k in a different one — a silent false positive — so the exact pairing is done in memory, not in the query.

// Confirm an EXACT per-element (value,kind) hit: the batched query pre-filtered
// on value only, so a value seen under a different kind must NOT match.
for _, iob := range ind.Observables {
    if !extracted[iob.Kind+"\x00"+iob.Value] { continue }
    // ...record Match...
}

This IXSCAN behaviour is a structural property, not a runtime assertion: because observables is multikey-indexed, the value pre-filter plans as an index seek rather than the collection scan a per-request STIX-pattern parse would force. The data-model contract documents an explain-should-be-IXSCAN target, but there is no mechanical executionStats acceptance gate in the repo — the guarantee comes from the index, not from a test.

Both sides of the match are normalized identically; otherwise a match silently misses. Feed ingest writes at active + 1 and bumps the version pointer in one update, so a half-written feed is never matched and a rollback is a snapshotVersion decrement.

Verdicts, privacy, tenancy

Verdict derivation is pure and deterministic — no Mongo, no bus:

Condition	Verdict	Confidence / Severity
0 matches, healthy parse	CLEAN	1.0 / low
0 matches, degraded / zero-observable parse	INCONCLUSIVE	0.0 / low — a real state, never a CLEAN fallback
≥ 1 match	COMPROMISED	strongest matched kind / max matched severity

Confidence is derived from the strongest matched observable kind (a hash or a profile id is a near-certain hit; a network indicator is strong; a process name is the weakest single signal), and matches sort highest-severity-first with a deterministic stixId tie-break so output is stable across runs. The severity vocabulary low | medium | high | critical is shared platform-wide; family names (the Pegasus / Predator / Reign mercenary class) are safe to name — the concrete indicator values behind them are not.

Isolation and erasure are construction rules, because subjects may be under state-level threat:

• Every document carries an accountId and every query filters on it. Match scope is the union of the caller's account and the shared platform account — a tenant sees its own private indicators plus the curated platform set, never another tenant's data.
• Identity is taken only from the verified service-to-service context, never from a request body, making body-tampering structurally impossible. Writing or enumerating the curated platform intel is gated to operator (platform-account) callers.
• Sensitive forensic fields (storageRef, deviceIdentifier, deviceName, extractedRecords, matches[].matchedValue, the inline rawArtifact) are AES-256-GCM field-encrypted and never indexed; structural fields stay clear so indexes and TTL keep working. The polled list endpoint projects out every encrypted field, so a poll fetches and decrypts nothing. Threat names (stixId/indicatorName/family) stay clear — they name the threat without revealing device contents.
• Retention is first-class: per-document expiresAt TTL indexes (expireAfterSeconds: 0) on artifacts and verdicts, plus scoped DeleteMany({accountId, principalId}) erasure. The rawArtifact is encrypted inline in-document rather than as an external blob, so a doc delete plus the TTL fully cover the raw data with no external cleanup path to miss.

Testing and security posture

Unit tests cover store encryption, matching, STIX parsing, verdicts, normalization and feeds, driven by a hermetic offline STIX fixture in the Amnesty MVT indicator format. Idempotent re-ingest is guaranteed by a unique {accountId, stixId, feedVersion} index with an unordered bulk upsert. The platform's standing security review is explicitly scoped to an older service set and does not cover MTD — but the conventions MTD inherits (tenant isolation by verified account, the HMAC s2s trust boundary, a default-deny PDP, typed BSON queries, AES-256-GCM field encryption) are the reviewed-good platform baseline.