Files
sst/docs/superpowers/plans/2026-10-01-app-1-engine-hooks.md
T

42 KiB
Raw Blame History

App fase 1 — Engine-raakpunten Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: De engine krijgt vier additieve uitbreidingen (woordenlijst voor de transcriber, niveau-uitvoer, resultaat-hook met correctie, instelbare sneltoets) zonder dat bestaand gedrag verandert.

Architecture: Alle wijzigingen zijn additief met standaardwaarden die het huidige gedrag behouden. Nieuwe pure logica (PromptEcho, AudioLevel, HotkeyBinding, ModifierKeyTracker, promptTokens) staat los van hardware en wordt met unit-tests gedekt; de hardware-afhankelijke stukken (AudioRecorder, HotkeyMonitor, WhisperKit) zijn dun en worden handmatig gecontroleerd.

Tech Stack: Swift 6.4 (language mode 5), Swift Testing, AVFoundation, CoreGraphics, WhisperKit 0.18.

Spec: docs/superpowers/specs/2026-10-01-macos-app-design.md (secties "Engine-raakpunten", "Woordenboek — mechanisme 1")

Global Constraints

  • Bestaande tests (13 stuks) blijven ongewijzigd groen; bestaande testbestanden worden alleen aangevuld (Fakes.swift) of krijgen er een nieuw bestand naast.
  • Nieuwe parameters hebben standaardwaarden die het huidige gedrag behouden.
  • Vocabulaire: maximaal 64 tokens in de prompt; prompt alleen bij opnames met RMS ≥ 0,015; stiltefilter blijft 0,005.
  • Prompt-echo: resultaat dat uitsluitend uit de eerste k ≥ 2 termen van de prompt bestaat wordt verworpen; één term alleen niet.
  • Niveau: onLevel levert RMS in dBFS; 0 VU = −20 dBFS.
  • Toegestane sneltoetsen (exact): Fn (keycode 63), Rechter Option (61), Rechter Command (54), Rechter Control (62), Linker Control (59).
  • Elke commit eindigt met de trailer Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> (tweede -m).

Bestandsstructuur

Sources/SstCore/Models.swift             (wijzig)  Transcriber + AppliedCorrection, CorrectionResult, TextCorrecting, DictationResult
Sources/SstCore/PromptEcho.swift         (nieuw)
Sources/SstCore/DictationController.swift(vervang) corrector, onResult, vocabularyProvider
Sources/SstCore/HotkeyBinding.swift      (nieuw)
Sources/SstCore/ModifierKeyTracker.swift (nieuw)
Sources/SstCore/HotkeyMonitor.swift      (vervang) binding
Sources/SstCore/AudioLevel.swift         (nieuw)
Sources/SstCore/AudioRecorder.swift      (wijzig)  onLevel
Sources/SstWhisper/WhisperKitTranscriber.swift (wijzig) vocabulary → promptTokens
Sources/SstProbe/RecordProbe.swift, TranscribeProbe.swift (wijzig) handmatige controle
Package.swift                            (wijzig)  SstWhisperTests
Tests/SstCoreTests/Fakes.swift           (aanvullen)
Tests/SstCoreTests/TranscriberVocabularyTests.swift, PromptEchoTests.swift,
                   DictationControllerHooksTests.swift, HotkeyBindingTests.swift,
                   ModifierKeyTrackerTests.swift, AudioLevelTests.swift   (nieuw)
Tests/SstWhisperTests/PromptTokensTests.swift (nieuw)

Task 1: Gedeelde typen en woordenlijst op Transcriber

Files:

  • Modify: Sources/SstCore/Models.swift
  • Test: Tests/SstCoreTests/TranscriberVocabularyTests.swift

Interfaces:

  • Produces (gebruikt door alle latere fasen):

    • Transcriber.transcribe(_ samples: [Float], vocabulary: [String]) async throws -> String (protocol-eis, met standaardimplementatie die transcribe(_:) aanroept)
    • struct AppliedCorrection: Equatable, Codable, Sendable { heard, wrote, rule: String; start, length: Int } met publieke init init(heard:wrote:rule:start:length:)
    • struct CorrectionResult: Equatable, Sendable { text: String; corrections: [AppliedCorrection] } met publieke init init(text:corrections:)
    • protocol TextCorrecting { func correct(_ text: String) -> CorrectionResult }
    • struct DictationResult: Equatable, Sendable { date: Date; duration: TimeInterval; rawText: String; finalText: String; corrections: [AppliedCorrection] } met publieke init in die volgorde
  • Step 1: Schrijf de falende test

Tests/SstCoreTests/TranscriberVocabularyTests.swift:

import Foundation
import Testing
@testable import SstCore

private struct PlainTranscriber: Transcriber {
    func transcribe(_ samples: [Float]) async throws -> String { "plain \(samples.count)" }
}

struct TranscriberVocabularyTests {
    @Test func vocabularyVariantFallsBackToPlainTranscribe() async throws {
        let result = try await PlainTranscriber().transcribe([0, 0, 0], vocabulary: ["Anthropic"])
        #expect(result == "plain 3")
    }

    @Test func appliedCorrectionRoundTripsThroughJSON() throws {
        let original = AppliedCorrection(heard: "cloud code", wrote: "Claude Code", rule: "cloud code", start: 12, length: 11)
        let data = try JSONEncoder().encode(original)
        let decoded = try JSONDecoder().decode(AppliedCorrection.self, from: data)
        #expect(decoded == original)
    }
}
  • Step 2: Draai de test, moet falen

Run: swift test --filter TranscriberVocabularyTests Expected: compilatiefout extra argument 'vocabulary' in call / cannot find 'AppliedCorrection' in scope.

  • Step 3: Implementatie

In Sources/SstCore/Models.swift vervang het blok

public protocol Transcriber {
    func transcribe(_ samples: [Float]) async throws -> String
}

door:

public protocol Transcriber {
    func transcribe(_ samples: [Float]) async throws -> String
    /// Biases decoding towards `vocabulary`. Engines without bias support ignore the list.
    func transcribe(_ samples: [Float], vocabulary: [String]) async throws -> String
}

public extension Transcriber {
    func transcribe(_ samples: [Float], vocabulary: [String]) async throws -> String {
        try await transcribe(samples)
    }
}

/// One replacement made by the dictionary correction pass.
/// `start`/`length` are UTF-16 offsets into the corrected text.
public struct AppliedCorrection: Equatable, Codable, Sendable {
    public var heard: String
    public var wrote: String
    public var rule: String
    public var start: Int
    public var length: Int

    public init(heard: String, wrote: String, rule: String, start: Int, length: Int) {
        self.heard = heard
        self.wrote = wrote
        self.rule = rule
        self.start = start
        self.length = length
    }
}

public struct CorrectionResult: Equatable, Sendable {
    public var text: String
    public var corrections: [AppliedCorrection]

    public init(text: String, corrections: [AppliedCorrection]) {
        self.text = text
        self.corrections = corrections
    }
}

public protocol TextCorrecting {
    func correct(_ text: String) -> CorrectionResult
}

/// A finished dictation, emitted just before the text is inserted.
public struct DictationResult: Equatable, Sendable {
    public var date: Date
    public var duration: TimeInterval
    public var rawText: String
    public var finalText: String
    public var corrections: [AppliedCorrection]

    public init(date: Date, duration: TimeInterval, rawText: String, finalText: String, corrections: [AppliedCorrection]) {
        self.date = date
        self.duration = duration
        self.rawText = rawText
        self.finalText = finalText
        self.corrections = corrections
    }
}
  • Step 4: Draai alle tests

Run: swift test Expected: alle tests PASS (13 bestaande + 2 nieuwe).

  • Step 5: Commit
git add Sources/SstCore/Models.swift Tests/SstCoreTests/TranscriberVocabularyTests.swift
git commit -m "feat: add vocabulary-aware Transcriber and shared correction/result types" -m "Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"

Task 2: DictationController — voorrang, echo-filter, correctie en resultaat-hook

Files:

  • Create: Sources/SstCore/PromptEcho.swift
  • Replace: Sources/SstCore/DictationController.swift
  • Modify: Tests/SstCoreTests/Fakes.swift (aanvullen onderaan)
  • Test: Tests/SstCoreTests/PromptEchoTests.swift, Tests/SstCoreTests/DictationControllerHooksTests.swift

Interfaces:

  • Consumes: Task 1 (Transcriber.transcribe(_:vocabulary:), TextCorrecting, CorrectionResult, DictationResult).

  • Produces:

    • PromptEcho.isEcho(_ text: String, vocabulary: [String]) -> Bool
    • DictationController.init(recorder:transcriber:cleaner:inserter:minimumSamples:silenceThreshold:corrector:vocabularyProvider:biasMinimumRMS:) — de nieuwe parameters zijn corrector: TextCorrecting? = nil, vocabularyProvider: @escaping () -> [String] = { [] }, biasMinimumRMS: Float = 0.015
    • DictationController.onResult: ((DictationResult) -> Void)?
  • Step 1: Schrijf de falende tests

Tests/SstCoreTests/PromptEchoTests.swift:

import Testing
@testable import SstCore

struct PromptEchoTests {
    let vocabulary = ["Anthropic", "Vercel", "Supabase"]

    @Test func firstTwoTermsAreAnEcho() {
        #expect(PromptEcho.isEcho("Anthropic, Vercel.", vocabulary: vocabulary))
    }

    @Test func wholeListIsAnEcho() {
        #expect(PromptEcho.isEcho("anthropic vercel supabase", vocabulary: vocabulary))
    }

    @Test func singleTermIsRealSpeech() {
        #expect(!PromptEcho.isEcho("Anthropic", vocabulary: vocabulary))
    }

    @Test func termsOutOfOrderAreNotAnEcho() {
        #expect(!PromptEcho.isEcho("Vercel, Anthropic", vocabulary: vocabulary))
    }

    @Test func sentenceContainingTermsIsNotAnEcho() {
        #expect(!PromptEcho.isEcho("Ik gebruik Anthropic en Vercel", vocabulary: vocabulary))
    }

    @Test func tooShortVocabularyNeverEchoes() {
        #expect(!PromptEcho.isEcho("Anthropic", vocabulary: ["Anthropic"]))
        #expect(!PromptEcho.isEcho("", vocabulary: vocabulary))
    }
}

Voeg onderaan Tests/SstCoreTests/Fakes.swift toe:


final class FakeVocabularyTranscriber: Transcriber {
    var result = "hallo wereld"
    private(set) var vocabularies: [[String]] = []

    func transcribe(_ samples: [Float]) async throws -> String { result }

    func transcribe(_ samples: [Float], vocabulary: [String]) async throws -> String {
        vocabularies.append(vocabulary)
        return result
    }
}

struct UppercaseCorrector: TextCorrecting {
    func correct(_ text: String) -> CorrectionResult {
        let upper = text.uppercased()
        return CorrectionResult(
            text: upper,
            corrections: [AppliedCorrection(heard: text, wrote: upper, rule: "upper", start: 0, length: upper.utf16.count)]
        )
    }
}

Tests/SstCoreTests/DictationControllerHooksTests.swift:

import Foundation
import Testing
@testable import SstCore

@MainActor
struct DictationControllerHooksTests {
    let recorder = FakeRecorder()
    let transcriber = FakeVocabularyTranscriber()
    let inserter = FakeInserter()
    static let loud = [Float](repeating: 0.1, count: 16_000)
    static let quiet = [Float](repeating: 0.01, count: 16_000)

    func makeController(
        cleaner: Cleaner = FakeCleaner(),
        corrector: TextCorrecting? = nil,
        vocabulary: [String] = []
    ) -> DictationController {
        DictationController(
            recorder: recorder,
            transcriber: transcriber,
            cleaner: cleaner,
            inserter: inserter,
            corrector: corrector,
            vocabularyProvider: { vocabulary }
        )
    }

    func dictate(_ controller: DictationController) async {
        await controller.handle(.pressed)
        await controller.handle(.released)
    }

    @Test func loudRecordingPassesVocabularyToTranscriber() async {
        recorder.samples = Self.loud
        await dictate(makeController(vocabulary: ["Anthropic", "Vercel"]))
        #expect(transcriber.vocabularies == [["Anthropic", "Vercel"]])
    }

    @Test func quietRecordingSkipsVocabularyButStillTranscribes() async {
        recorder.samples = Self.quiet   // RMS 0.01: above silence (0.005), below bias minimum (0.015)
        await dictate(makeController(vocabulary: ["Anthropic", "Vercel"]))
        #expect(transcriber.vocabularies == [[]])
        #expect(inserter.inserted == ["[hallo wereld]"])
    }

    @Test func promptEchoIsDiscarded() async {
        recorder.samples = Self.loud
        transcriber.result = "Anthropic, Vercel."
        let controller = makeController(vocabulary: ["Anthropic", "Vercel", "Supabase"])
        var results: [DictationResult] = []
        controller.onResult = { results.append($0) }

        await dictate(controller)

        #expect(inserter.inserted.isEmpty)
        #expect(results.isEmpty)
        #expect(controller.state == .idle)
    }

    @Test func singleVocabularyTermIsKept() async {
        recorder.samples = Self.loud
        transcriber.result = "Anthropic"
        await dictate(makeController(vocabulary: ["Anthropic", "Vercel", "Supabase"]))
        #expect(inserter.inserted == ["[Anthropic]"])
    }

    @Test func correctorRunsBeforeCleanerAndCorrectionsSurviveAPassthroughCleaner() async {
        recorder.samples = Self.loud
        transcriber.result = "hallo"
        let controller = makeController(cleaner: PassthroughCleaner(), corrector: UppercaseCorrector())
        var results: [DictationResult] = []
        controller.onResult = { results.append($0) }

        await dictate(controller)

        #expect(inserter.inserted == ["HALLO"])
        #expect(results.count == 1)
        #expect(results[0].rawText == "hallo")
        #expect(results[0].finalText == "HALLO")
        #expect(results[0].corrections.count == 1)
    }

    @Test func correctionsAreDroppedWhenTheCleanerChangesTheText() async {
        recorder.samples = Self.loud
        transcriber.result = "hallo"
        let controller = makeController(cleaner: FakeCleaner(), corrector: UppercaseCorrector())
        var results: [DictationResult] = []
        controller.onResult = { results.append($0) }

        await dictate(controller)

        #expect(results[0].finalText == "[HALLO]")
        #expect(results[0].corrections.isEmpty)
    }

    @Test func resultIsReportedEvenWhenInsertionFails() async {
        recorder.samples = Self.loud
        inserter.error = TestError()
        let controller = makeController()
        var results: [DictationResult] = []
        var errors: [String] = []
        controller.onResult = { results.append($0) }
        controller.onError = { errors.append($0) }

        await dictate(controller)

        #expect(results.count == 1)
        #expect(errors.count == 1)
    }

    @Test func resultCarriesDurationAndDate() async {
        recorder.samples = Self.loud   // 16 000 samples = 1 s
        let controller = makeController()
        var results: [DictationResult] = []
        controller.onResult = { results.append($0) }
        let before = Date()

        await dictate(controller)

        #expect(results[0].duration == 1.0)
        #expect(results[0].date >= before)
        #expect(results[0].corrections.isEmpty)
    }
}
  • Step 2: Draai de tests, moeten falen

Run: swift test --filter "PromptEchoTests|DictationControllerHooksTests" Expected: compilatiefouten (cannot find 'PromptEcho', extra argument 'corrector').

  • Step 3: Implementatie

Sources/SstCore/PromptEcho.swift:

import Foundation

/// Whisper sometimes answers a vocabulary prompt with the prompt itself when there is little speech.
public enum PromptEcho {
    static func normalize(_ text: String) -> String {
        String(text.lowercased().map { $0.isLetter || $0.isNumber ? $0 : " " })
            .split(separator: " ")
            .joined(separator: " ")
    }

    /// True when `text` is nothing but the first k ≥ 2 terms of the prompt.
    /// A single term on its own is valid speech ("Anthropic").
    public static func isEcho(_ text: String, vocabulary: [String]) -> Bool {
        guard vocabulary.count >= 2 else { return false }
        let normalized = normalize(text)
        guard !normalized.isEmpty else { return false }
        return (2...vocabulary.count).contains { k in
            normalize(vocabulary.prefix(k).joined(separator: " ")) == normalized
        }
    }
}

Vervang Sources/SstCore/DictationController.swift volledig door:

import Foundation

@MainActor
public final class DictationController {
    public private(set) var state: DictationState = .idle {
        didSet { onStateChange?(state) }
    }
    public var onStateChange: ((DictationState) -> Void)?
    public var onError: ((String) -> Void)?
    /// Called with every finished transcription, just before it is inserted.
    public var onResult: ((DictationResult) -> Void)?

    private let recorder: AudioRecording
    private let transcriber: Transcriber
    private let cleaner: Cleaner
    private let inserter: TextInserting
    private let minimumSamples: Int
    private let silenceThreshold: Float
    private let corrector: TextCorrecting?
    private let vocabularyProvider: () -> [String]
    private let biasMinimumRMS: Float

    /// - Parameter minimumSamples: 16 kHz samples; default is 0.3 s.
    /// - Parameter silenceThreshold: RMS below which a recording counts as silence and is skipped.
    /// - Parameter corrector: optional dictionary correction applied to the transcript.
    /// - Parameter vocabularyProvider: words to bias the transcriber towards.
    /// - Parameter biasMinimumRMS: below this RMS no vocabulary is passed (guards against invented text on quiet audio).
    public init(
        recorder: AudioRecording,
        transcriber: Transcriber,
        cleaner: Cleaner,
        inserter: TextInserting,
        minimumSamples: Int = 4_800,
        silenceThreshold: Float = 0.005,
        corrector: TextCorrecting? = nil,
        vocabularyProvider: @escaping () -> [String] = { [] },
        biasMinimumRMS: Float = 0.015
    ) {
        self.recorder = recorder
        self.transcriber = transcriber
        self.cleaner = cleaner
        self.inserter = inserter
        self.minimumSamples = minimumSamples
        self.silenceThreshold = silenceThreshold
        self.corrector = corrector
        self.vocabularyProvider = vocabularyProvider
        self.biasMinimumRMS = biasMinimumRMS
    }

    public func handle(_ event: HotkeyEvent) async {
        switch (event, state) {
        case (.pressed, .idle):
            do {
                try recorder.start()
                state = .recording
            } catch {
                onError?(error.localizedDescription)
            }
        case (.released, .recording):
            let samples = recorder.stop()
            guard samples.count >= minimumSamples else {
                state = .idle
                return
            }
            let energy = samples.reduce(0) { $0 + $1 * $1 }
            let rms = (energy / Float(samples.count)).squareRoot()
            guard rms >= silenceThreshold else {
                state = .idle
                return
            }
            await process(samples, rms: rms)
        default:
            break
        }
    }

    private func process(_ samples: [Float], rms: Float) async {
        defer { state = .idle }
        do {
            state = .transcribing
            let vocabulary = rms >= biasMinimumRMS ? vocabularyProvider() : []
            let raw = try await transcriber.transcribe(samples, vocabulary: vocabulary)
            let text = raw.trimmingCharacters(in: .whitespacesAndNewlines)
            guard !text.isEmpty, !PromptEcho.isEcho(text, vocabulary: vocabulary) else { return }
            let corrected = corrector?.correct(text) ?? CorrectionResult(text: text, corrections: [])
            let cleaned = try await cleaner.clean(corrected.text, context: CleaningContext())
            onResult?(DictationResult(
                date: Date(),
                duration: Double(samples.count) / 16_000,
                rawText: text,
                finalText: cleaned,
                // Offsets only stay valid if the cleaner left the corrected text alone.
                corrections: cleaned == corrected.text ? corrected.corrections : []
            ))
            state = .inserting
            try await inserter.insert(cleaned)
        } catch {
            onError?(error.localizedDescription)
        }
    }
}
  • Step 4: Draai alle tests

Run: swift test Expected: alle tests PASS (de 13 + 2 + 6 + 8 nieuwe = 29).

  • Step 5: Commit
git add Sources/SstCore Tests/SstCoreTests
git commit -m "feat: controller supports vocabulary bias, prompt-echo guard, corrector and result hook" -m "Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"

Task 3: Instelbare sneltoets — HotkeyBinding, ModifierKeyTracker, HotkeyMonitor

Files:

  • Create: Sources/SstCore/HotkeyBinding.swift, Sources/SstCore/ModifierKeyTracker.swift
  • Replace: Sources/SstCore/HotkeyMonitor.swift
  • Test: Tests/SstCoreTests/HotkeyBindingTests.swift, Tests/SstCoreTests/ModifierKeyTrackerTests.swift

Interfaces:

  • Consumes: HotkeyEvent uit Models.swift.

  • Produces:

    • struct HotkeyBinding: Hashable, Sendable, Identifiable met id: String, displayName: String, keyCode: Int, flagRawValue: UInt64, flag: CGEventFlags; statics .fn, .rightOption, .rightCommand, .rightControl, .leftControl, allowed: [HotkeyBinding], binding(id:), binding(keyCode:)
    • struct ModifierKeyTracker { init(); mutating func process(isDown: Bool) -> HotkeyEvent? }
    • HotkeyMonitor.binding: HotkeyBinding (standaard .fn; wisselen herstelt de tracker)
  • Step 1: Schrijf de falende tests

Tests/SstCoreTests/HotkeyBindingTests.swift:

import Testing
@testable import SstCore

struct HotkeyBindingTests {
    @Test func allowedBindingsMatchTheSpec() {
        #expect(HotkeyBinding.allowed.map(\.id) == ["fn", "rightOption", "rightCommand", "rightControl", "leftControl"])
        #expect(HotkeyBinding.allowed.map(\.keyCode) == [63, 61, 54, 62, 59])
    }

    @Test func idsAndKeyCodesAreUnique() {
        #expect(Set(HotkeyBinding.allowed.map(\.id)).count == HotkeyBinding.allowed.count)
        #expect(Set(HotkeyBinding.allowed.map(\.keyCode)).count == HotkeyBinding.allowed.count)
    }

    @Test func lookupByKeyCodeAndId() {
        #expect(HotkeyBinding.binding(keyCode: 61) == .rightOption)
        #expect(HotkeyBinding.binding(id: "leftControl") == .leftControl)
    }

    @Test func unsupportedKeysAreRejected() {
        #expect(HotkeyBinding.binding(keyCode: 56) == nil)   // left shift
        #expect(HotkeyBinding.binding(keyCode: 57) == nil)   // caps lock
        #expect(HotkeyBinding.binding(id: "nope") == nil)
    }

    @Test func everyBindingHasAFlagAndName() {
        for binding in HotkeyBinding.allowed {
            #expect(binding.flagRawValue != 0)
            #expect(!binding.displayName.isEmpty)
        }
    }
}

Tests/SstCoreTests/ModifierKeyTrackerTests.swift:

import Testing
@testable import SstCore

struct ModifierKeyTrackerTests {
    @Test func emitsPressedThenReleasedOnlyOnTransitions() {
        var tracker = ModifierKeyTracker()
        #expect(tracker.process(isDown: false) == nil)
        #expect(tracker.process(isDown: true) == .pressed)
        #expect(tracker.process(isDown: true) == nil)
        #expect(tracker.process(isDown: false) == .released)
        #expect(tracker.process(isDown: false) == nil)
    }

    @Test func freshTrackerStartsReleased() {
        var tracker = ModifierKeyTracker()
        #expect(tracker.process(isDown: true) == .pressed)
    }
}
  • Step 2: Draai de tests, moeten falen

Run: swift test --filter "HotkeyBindingTests|ModifierKeyTrackerTests" Expected: compilatiefout cannot find 'HotkeyBinding' in scope.

  • Step 3: Implementatie

Sources/SstCore/ModifierKeyTracker.swift:

public struct ModifierKeyTracker {
    private var isDown = false

    public init() {}

    /// Feed the current state of the bound modifier flag; returns an event only on a transition.
    public mutating func process(isDown: Bool) -> HotkeyEvent? {
        guard isDown != self.isDown else { return nil }
        self.isDown = isDown
        return isDown ? .pressed : .released
    }
}

Sources/SstCore/HotkeyBinding.swift:

import CoreGraphics

/// A modifier key that can be held to dictate. Left/right keys share a flag, so the key code decides.
public struct HotkeyBinding: Hashable, Sendable, Identifiable {
    public let id: String
    public let displayName: String
    public let keyCode: Int
    public let flagRawValue: UInt64

    public var flag: CGEventFlags { CGEventFlags(rawValue: flagRawValue) }

    public static let fn = HotkeyBinding(
        id: "fn", displayName: "Fn", keyCode: 63, flagRawValue: CGEventFlags.maskSecondaryFn.rawValue)
    public static let rightOption = HotkeyBinding(
        id: "rightOption", displayName: "Rechter Option", keyCode: 61, flagRawValue: CGEventFlags.maskAlternate.rawValue)
    public static let rightCommand = HotkeyBinding(
        id: "rightCommand", displayName: "Rechter Command", keyCode: 54, flagRawValue: CGEventFlags.maskCommand.rawValue)
    public static let rightControl = HotkeyBinding(
        id: "rightControl", displayName: "Rechter Control", keyCode: 62, flagRawValue: CGEventFlags.maskControl.rawValue)
    public static let leftControl = HotkeyBinding(
        id: "leftControl", displayName: "Linker Control", keyCode: 59, flagRawValue: CGEventFlags.maskControl.rawValue)

    public static let allowed: [HotkeyBinding] = [.fn, .rightOption, .rightCommand, .rightControl, .leftControl]

    public static func binding(id: String) -> HotkeyBinding? { allowed.first { $0.id == id } }
    public static func binding(keyCode: Int) -> HotkeyBinding? { allowed.first { $0.keyCode == keyCode } }
}

Vervang Sources/SstCore/HotkeyMonitor.swift volledig door:

import ApplicationServices
import CoreGraphics

public final class HotkeyMonitor {
    public enum HotkeyError: Error, LocalizedError {
        case tapUnavailable

        public var errorDescription: String? {
            "Kan de sneltoets niet volgen: geef Input Monitoring-rechten in Systeeminstellingen en probeer opnieuw (soms is een herstart van de app nodig)."
        }
    }

    public var onEvent: ((HotkeyEvent) -> Void)?

    /// The key that has to be held. Changing it resets the press tracking.
    public var binding: HotkeyBinding = .fn {
        didSet { tracker = ModifierKeyTracker() }
    }

    private var tracker = ModifierKeyTracker()
    private var tap: CFMachPort?
    private var source: CFRunLoopSource?

    public init() {}

    public func start() throws {
        guard tap == nil else { return }
        let mask = CGEventMask(1 << CGEventType.flagsChanged.rawValue)
        let callback: CGEventTapCallBack = { _, type, event, refcon in
            if let refcon {
                let monitor = Unmanaged<HotkeyMonitor>.fromOpaque(refcon).takeUnretainedValue()
                monitor.handle(type: type, event: event)
            }
            return Unmanaged.passUnretained(event)
        }
        guard let tap = CGEvent.tapCreate(
            tap: .cgSessionEventTap,
            place: .headInsertEventTap,
            options: .listenOnly,
            eventsOfInterest: mask,
            callback: callback,
            userInfo: Unmanaged.passUnretained(self).toOpaque()
        ) else { throw HotkeyError.tapUnavailable }

        let source = CFMachPortCreateRunLoopSource(kCFAllocatorDefault, tap, 0)
        CFRunLoopAddSource(CFRunLoopGetMain(), source, .commonModes)
        CGEvent.tapEnable(tap: tap, enable: true)
        self.tap = tap
        self.source = source
    }

    public func stop() {
        if let source { CFRunLoopRemoveSource(CFRunLoopGetMain(), source, .commonModes) }
        if let tap { CGEvent.tapEnable(tap: tap, enable: false) }
        tap = nil
        source = nil
    }

    private func handle(type: CGEventType, event: CGEvent) {
        switch type {
        case .tapDisabledByTimeout, .tapDisabledByUserInput:
            if let tap { CGEvent.tapEnable(tap: tap, enable: true) }
        case .flagsChanged:
            // Flags are shared between left and right keys and with arrows/F-keys: only react to the bound key itself.
            guard event.getIntegerValueField(.keyboardEventKeycode) == Int64(binding.keyCode) else { return }
            let isDown = event.flags.contains(binding.flag)
            if let hotkeyEvent = tracker.process(isDown: isDown) {
                onEvent?(hotkeyEvent)
            }
        default:
            break
        }
    }
}

FnKeyTracker en zijn test blijven ongewijzigd bestaan; HotkeyMonitor gebruikt hem niet meer.

  • Step 4: Draai alle tests en bouw

Run: swift test && swift build Expected: alle tests PASS; build slaagt (de app en SstProbe gebruiken HotkeyMonitor() zonder argumenten en blijven dus compileren).

  • Step 5: Handmatige controle (met de gebruiker, na Task 5)

Wordt gebundeld in Task 5, stap 6.

  • Step 6: Commit
git add Sources/SstCore Tests/SstCoreTests
git commit -m "feat: configurable hotkey binding and ModifierKeyTracker" -m "Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"

Task 4: Niveau-uitvoer — AudioLevel en AudioRecorder.onLevel

Files:

  • Create: Sources/SstCore/AudioLevel.swift
  • Modify: Sources/SstCore/AudioRecorder.swift
  • Modify: Sources/SstProbe/RecordProbe.swift
  • Test: Tests/SstCoreTests/AudioLevelTests.swift

Interfaces:

  • Produces:

    • AudioLevel.rmsDbfs(_ samples: [Float]) -> Float en rmsDbfs(_ samples: UnsafeBufferPointer<Float>) -> Float (−120 voor stilte of leeg)
    • AudioLevel.vu(fromDbfs:referenceDbfs:) -> Float (standaard referentie −20)
    • AudioRecorder.onLevel: ((Float) -> Void)? — RMS in dBFS per aangeleverde buffer, aangeroepen op de audio-thread; vóór start() instellen.
  • Step 1: Schrijf de falende test

Tests/SstCoreTests/AudioLevelTests.swift:

import Testing
@testable import SstCore

struct AudioLevelTests {
    @Test func constantTenthIsMinusTwentyDbfs() {
        let level = AudioLevel.rmsDbfs([Float](repeating: 0.1, count: 1_000))
        #expect(abs(level - (-20)) < 0.01)
    }

    @Test func fullScaleIsZeroDbfs() {
        let level = AudioLevel.rmsDbfs([Float](repeating: 1.0, count: 100))
        #expect(abs(level) < 0.01)
    }

    @Test func negativeSamplesCountLikePositiveOnes() {
        let level = AudioLevel.rmsDbfs([Float](repeating: -0.1, count: 1_000))
        #expect(abs(level - (-20)) < 0.01)
    }

    @Test func emptyAndSilentBuffersHitTheFloor() {
        #expect(AudioLevel.rmsDbfs([]) == AudioLevel.silenceFloorDbfs)
        #expect(AudioLevel.rmsDbfs([Float](repeating: 0, count: 64)) == AudioLevel.silenceFloorDbfs)
    }

    @Test func vuIsRelativeToTheReference() {
        #expect(AudioLevel.vu(fromDbfs: -20) == 0)
        #expect(AudioLevel.vu(fromDbfs: -40) == -20)
        #expect(AudioLevel.vu(fromDbfs: -17) == 3)
        #expect(AudioLevel.vu(fromDbfs: -30, referenceDbfs: -18) == -12)
    }
}
  • Step 2: Draai de test, moet falen

Run: swift test --filter AudioLevelTests Expected: compilatiefout cannot find 'AudioLevel' in scope.

  • Step 3: Implementatie

Sources/SstCore/AudioLevel.swift:

import Foundation

public enum AudioLevel {
    /// Reported for empty or silent buffers (log of zero would be -infinity).
    public static let silenceFloorDbfs: Float = -120

    public static func rmsDbfs(_ samples: [Float]) -> Float {
        samples.withUnsafeBufferPointer { rmsDbfs($0) }
    }

    public static func rmsDbfs(_ samples: UnsafeBufferPointer<Float>) -> Float {
        guard !samples.isEmpty else { return silenceFloorDbfs }
        var sum: Float = 0
        for sample in samples { sum += sample * sample }
        let rms = (sum / Float(samples.count)).squareRoot()
        guard rms > 0 else { return silenceFloorDbfs }
        return max(silenceFloorDbfs, 20 * log10(rms))
    }

    /// VU reading for a dBFS level, where `referenceDbfs` is 0 VU.
    public static func vu(fromDbfs dbfs: Float, referenceDbfs: Float = -20) -> Float {
        dbfs - referenceDbfs
    }
}

In Sources/SstCore/AudioRecorder.swift:

  1. Voeg onder private var buffer: [Float] = [] toe:

    /// RMS level (dBFS) of every delivered input buffer. Called on the audio thread; set before `start()`.
    public var onLevel: ((Float) -> Void)?
  1. Vervang in de tap-closure de regel guard let self else { return } door:
            guard let self else { return }
            if let onLevel = self.onLevel, let channel = pcm.floatChannelData?[0] {
                onLevel(AudioLevel.rmsDbfs(UnsafeBufferPointer(start: channel, count: Int(pcm.frameLength))))
            }
  1. Vervang bufferSize: 4096 door bufferSize: 1024 (macOS mag een grotere buffer leveren; de meter werkt met elke grootte).

Sources/SstProbe/RecordProbe.swift — vervang volledig door:

import AVFoundation
import SstCore

/// Records `seconds` of microphone audio; returns nil if access is denied or recording fails.
func recordSamples(seconds: Double, onLevel: ((Float) -> Void)? = nil) async -> [Float]? {
    guard await AVCaptureDevice.requestAccess(for: .audio) else {
        print("Microfoon-toegang geweigerd (Systeeminstellingen → Privacy → Microfoon).")
        return nil
    }
    let recorder = AudioRecorder()
    recorder.onLevel = onLevel
    do { try recorder.start() } catch {
        print("start mislukt:", error.localizedDescription)
        return nil
    }
    print("Neem \(Int(seconds)) s op — praat nu")
    try? await Task.sleep(for: .seconds(seconds))
    return recorder.stop()
}

func runRecordProbe() async {
    final class Levels: @unchecked Sendable {
        let lock = NSLock()
        var values: [Float] = []
        func add(_ v: Float) { lock.withLock { values.append(v) } }
    }
    let levels = Levels()
    guard let samples = await recordSamples(seconds: 3, onLevel: { levels.add($0) }) else { return }
    let peak = samples.map { abs($0) }.max() ?? 0
    print("samples: \(samples.count) (verwacht ≈ 48000), piek: \(peak)")
    let values = levels.lock.withLock { levels.values }
    if let lo = values.min(), let hi = values.max() {
        print(String(format: "niveau-meldingen: %d (≈ %.0f per s), bereik %.1f … %.1f dBFS", values.count, Double(values.count) / 3, lo, hi))
    } else {
        print("geen niveau-meldingen ontvangen")
    }
}
  • Step 4: Draai alle tests en bouw

Run: swift test && swift build Expected: alle tests PASS; build slaagt.

  • Step 5: Handmatige controle (met de gebruiker, in Task 5, stap 6)

Hier alleen: de code compileert.

  • Step 6: Commit
git add Sources Tests
git commit -m "feat: AudioLevel helper and AudioRecorder.onLevel" -m "Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"

Task 5: WhisperKit-voorrang — woordenlijst als promptTokens

Files:

  • Modify: Package.swift (testtarget SstWhisperTests)
  • Modify: Sources/SstWhisper/WhisperKitTranscriber.swift
  • Modify: Sources/SstProbe/TranscribeProbe.swift
  • Test: Tests/SstWhisperTests/PromptTokensTests.swift

Interfaces:

  • Consumes: Task 1 (Transcriber.transcribe(_:vocabulary:)).

  • Produces: WhisperKitTranscriber.transcribe(_:vocabulary:); intern static func promptTokens(vocabulary:maxTokens:specialTokenBegin:encode:) -> [Int]?.

  • Step 1: Testtarget toevoegen

In Package.swift voeg onder de bestaande .testTarget(name: "SstCoreTests", …) toe:

        .testTarget(name: "SstWhisperTests", dependencies: ["SstWhisper"]),
  • Step 2: Schrijf de falende test

Tests/SstWhisperTests/PromptTokensTests.swift:

import Testing
@testable import SstWhisper

struct PromptTokensTests {
    /// Fake tokenizer: one token (id 7) per word, plus one special token (id 2000) that must be filtered out.
    let encode: (String) -> [Int] = { text in
        text.split(whereSeparator: { !$0.isLetter }).map { _ in 7 } + [2000]
    }

    @Test func allTermsFitAndSpecialTokensAreFiltered() {
        let tokens = WhisperKitTranscriber.promptTokens(
            vocabulary: ["Anthropic", "Vercel", "Supabase"], maxTokens: 64, specialTokenBegin: 1000, encode: encode)
        #expect(tokens == [7, 7, 7])
    }

    @Test func termsThatDoNotFitAreDroppedFromTheEnd() {
        let vocab = ["a", "b", "c", "d", "e", "f", "g", "h", "i", "j"]
        let tokens = WhisperKitTranscriber.promptTokens(
            vocabulary: vocab, maxTokens: 4, specialTokenBegin: 1000, encode: encode)
        #expect(tokens?.count == 4)
    }

    @Test func emptyVocabularyGivesNoPrompt() {
        #expect(WhisperKitTranscriber.promptTokens(
            vocabulary: [], maxTokens: 64, specialTokenBegin: 1000, encode: encode) == nil)
    }

    @Test func firstTermAloneTooLongGivesNoPrompt() {
        #expect(WhisperKitTranscriber.promptTokens(
            vocabulary: ["one two three"], maxTokens: 2, specialTokenBegin: 1000, encode: encode) == nil)
    }
}
  • Step 3: Draai de test, moet falen

Run: swift test --filter PromptTokensTests Expected: compilatiefout type 'WhisperKitTranscriber' has no member 'promptTokens'.

  • Step 4: Implementatie

In Sources/SstWhisper/WhisperKitTranscriber.swift vervang de hele methode public func transcribe(_ samples: [Float]) async throws -> String { … } (onderaan het bestand) door:

    public func transcribe(_ samples: [Float]) async throws -> String {
        try await transcribe(samples, vocabulary: [])
    }

    /// Terms are added one by one while the encoded prompt stays within `maxTokens`.
    /// Returns nil when nothing fits or the list is empty.
    static func promptTokens(
        vocabulary: [String],
        maxTokens: Int = 64,
        specialTokenBegin: Int,
        encode: (String) -> [Int]
    ) -> [Int]? {
        var kept: [String] = []
        var tokens: [Int] = []
        for term in vocabulary {
            let candidate = kept + [term]
            let encoded = encode(" " + candidate.joined(separator: ", ")).filter { $0 < specialTokenBegin }
            if encoded.count > maxTokens { break }
            kept = candidate
            tokens = encoded
        }
        return tokens.isEmpty ? nil : tokens
    }

    public func transcribe(_ samples: [Float], vocabulary: [String]) async throws -> String {
        guard let pipeline else { throw TranscriberError.notLoaded }
        // language nil + detectLanguage: Whisper picks NL or EN per utterance.
        var options = DecodingOptions(
            task: .transcribe,
            language: nil,
            detectLanguage: true,
            skipSpecialTokens: true,
            withoutTimestamps: true
        )
        if !vocabulary.isEmpty, let tokenizer = pipeline.tokenizer {
            options.promptTokens = Self.promptTokens(
                vocabulary: vocabulary,
                specialTokenBegin: tokenizer.specialTokens.specialTokenBegin,
                encode: { tokenizer.encode(text: $0) }
            )
        }
        let results = try await pipeline.transcribe(audioArray: samples, decodeOptions: options)
        return results
            .map(\.text)
            .joined(separator: " ")
            .trimmingCharacters(in: .whitespacesAndNewlines)
    }

Wijkt de WhisperKit-API licht af (bijvoorbeeld de plaats van specialTokens), pas dan alleen dit bestand aan en behoud de signatuur van promptTokens zodat de test geldig blijft.

Sources/SstProbe/TranscribeProbe.swift — vervang regel let text = try await transcriber.transcribe(samples) door:

        let vocabulary = (ProcessInfo.processInfo.environment["SST_VOCAB"] ?? "")
            .split(separator: ",").map { $0.trimmingCharacters(in: .whitespaces) }.filter { !$0.isEmpty }
        if !vocabulary.isEmpty { print("voorrang: \(vocabulary.joined(separator: ", "))") }
        let text = try await transcriber.transcribe(samples, vocabulary: vocabulary)
  • Step 5: Draai alle tests en bouw

Run: swift test && swift build Expected: alle tests PASS (incl. 4 nieuwe in SstWhisperTests); build slaagt.

  • Step 6: Handmatige controle samen met de gebruiker (hardware)

Doe in deze volgorde en noteer de uitkomsten in het commit-bericht van stap 7:

  1. swift run SstProbe record → er verschijnt een regel niveau-meldingen: N (≈ X per s), bereik … dBFS. Verwacht: meer dan 5 meldingen per seconde en een bereik dat bij praten duidelijk hoger ligt dan bij stilte. Noteer X (macOS levert soms ~10 per s in plaats van ~47).
  2. swift run SstProbe transcribe — zeg een zin, noteer de tijd zonder voorrang.
  3. SST_VOCAB="Anthropic,Vercel,Supabase,Claude Code" swift run SstProbe transcribe — zeg dezelfde zin met een of twee van die woorden erin ("Ik werk met Supabase en Claude Code"); noteer de tijd en of de spelling beter is. Verwacht: tijd onder 1,5 s voor 8 s spraak. Is het trager, noteer hoeveel; dat is geen blokkade, wel informatie voor de spec.
  4. Zeg niets (8 s stilte) met SST_VOCAB gezet: het resultaat mag geen woorden uit de lijst bevatten.
  5. Hotkey-controle: swift run SstProbe hotkey toont nog steeds pressed/released voor Fn.
  • Step 7: Commit
git add Package.swift Sources Tests
git commit -m "feat: WhisperKit transcriber biases decoding with vocabulary prompt tokens" -m "Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"

Self-review

  • Spec-dekking: raakpunt 1 (woordenlijst → promptTokens, Task 1+5), 2 (niveau, Task 4), 3 (corrector + onResult, Task 1+2), 4 (instelbare toets, Task 3). Voorrang-regels: RMS-grens 0,015 (Task 2 test quietRecordingSkipsVocabularyButStillTranscribes), echo-filter (Task 2), 64 tokens (Task 5). Niet in deze fase, wel in fase 2: het opbouwen van de woordenlijst (VocabularyBuilder) en de correctie zelf.
  • Placeholders: geen; de waarschuwing in Task 5 stap 4 is een gerichte terugval, geen open eind.
  • Typeconsistentie: AppliedCorrection(heard:wrote:rule:start:length:), CorrectionResult(text:corrections:) en DictationResult(date:duration:rawText:finalText:corrections:) worden in Task 2 precies zo gebruikt als in Task 1 gedefinieerd. HotkeyMonitor.binding gebruikt HotkeyBinding.keyCode: Int en flag.
  • Risico's: de bufferlevering van installTap (Task 4, stap 6.1) en de snelheid met prompt (stap 6.3) zijn de enige onbekenden; beide worden gemeten.