# App fase 1 — Engine-raakpunten Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** De engine krijgt vier additieve uitbreidingen (woordenlijst voor de transcriber, niveau-uitvoer, resultaat-hook met correctie, instelbare sneltoets) zonder dat bestaand gedrag verandert. **Architecture:** Alle wijzigingen zijn additief met standaardwaarden die het huidige gedrag behouden. Nieuwe pure logica (`PromptEcho`, `AudioLevel`, `HotkeyBinding`, `ModifierKeyTracker`, `promptTokens`) staat los van hardware en wordt met unit-tests gedekt; de hardware-afhankelijke stukken (`AudioRecorder`, `HotkeyMonitor`, WhisperKit) zijn dun en worden handmatig gecontroleerd. **Tech Stack:** Swift 6.4 (language mode 5), Swift Testing, AVFoundation, CoreGraphics, WhisperKit 0.18. **Spec:** `docs/superpowers/specs/2026-10-01-macos-app-design.md` (secties "Engine-raakpunten", "Woordenboek — mechanisme 1") ## Global Constraints - Bestaande tests (13 stuks) blijven ongewijzigd groen; bestaande testbestanden worden alleen *aangevuld* (`Fakes.swift`) of krijgen er een nieuw bestand naast. - Nieuwe parameters hebben standaardwaarden die het huidige gedrag behouden. - Vocabulaire: maximaal 64 tokens in de prompt; prompt alleen bij opnames met RMS ≥ 0,015; stiltefilter blijft 0,005. - Prompt-echo: resultaat dat uitsluitend uit de eerste *k* ≥ 2 termen van de prompt bestaat wordt verworpen; één term alleen niet. - Niveau: `onLevel` levert RMS in dBFS; 0 VU = −20 dBFS. - Toegestane sneltoetsen (exact): Fn (keycode 63), Rechter Option (61), Rechter Command (54), Rechter Control (62), Linker Control (59). - Elke commit eindigt met de trailer `Co-Authored-By: Claude Sonnet 5.5 ` (tweede `-m`). ## Bestandsstructuur ``` Sources/SstCore/Models.swift (wijzig) Transcriber + AppliedCorrection, CorrectionResult, TextCorrecting, DictationResult Sources/SstCore/PromptEcho.swift (nieuw) Sources/SstCore/DictationController.swift(vervang) corrector, onResult, vocabularyProvider Sources/SstCore/HotkeyBinding.swift (nieuw) Sources/SstCore/ModifierKeyTracker.swift (nieuw) Sources/SstCore/HotkeyMonitor.swift (vervang) binding Sources/SstCore/AudioLevel.swift (nieuw) Sources/SstCore/AudioRecorder.swift (wijzig) onLevel Sources/SstWhisper/WhisperKitTranscriber.swift (wijzig) vocabulary → promptTokens Sources/SstProbe/RecordProbe.swift, TranscribeProbe.swift (wijzig) handmatige controle Package.swift (wijzig) SstWhisperTests Tests/SstCoreTests/Fakes.swift (aanvullen) Tests/SstCoreTests/TranscriberVocabularyTests.swift, PromptEchoTests.swift, DictationControllerHooksTests.swift, HotkeyBindingTests.swift, ModifierKeyTrackerTests.swift, AudioLevelTests.swift (nieuw) Tests/SstWhisperTests/PromptTokensTests.swift (nieuw) ``` --- ### Task 1: Gedeelde typen en woordenlijst op `Transcriber` **Files:** - Modify: `Sources/SstCore/Models.swift` - Test: `Tests/SstCoreTests/TranscriberVocabularyTests.swift` **Interfaces:** - Produces (gebruikt door alle latere fasen): - `Transcriber.transcribe(_ samples: [Float], vocabulary: [String]) async throws -> String` (protocol-eis, met standaardimplementatie die `transcribe(_:)` aanroept) - `struct AppliedCorrection: Equatable, Codable, Sendable { heard, wrote, rule: String; start, length: Int }` met publieke init `init(heard:wrote:rule:start:length:)` - `struct CorrectionResult: Equatable, Sendable { text: String; corrections: [AppliedCorrection] }` met publieke init `init(text:corrections:)` - `protocol TextCorrecting { func correct(_ text: String) -> CorrectionResult }` - `struct DictationResult: Equatable, Sendable { date: Date; duration: TimeInterval; rawText: String; finalText: String; corrections: [AppliedCorrection] }` met publieke init in die volgorde - [ ] **Step 1: Schrijf de falende test** `Tests/SstCoreTests/TranscriberVocabularyTests.swift`: ```swift import Foundation import Testing @testable import SstCore private struct PlainTranscriber: Transcriber { func transcribe(_ samples: [Float]) async throws -> String { "plain \(samples.count)" } } struct TranscriberVocabularyTests { @Test func vocabularyVariantFallsBackToPlainTranscribe() async throws { let result = try await PlainTranscriber().transcribe([0, 0, 0], vocabulary: ["Anthropic"]) #expect(result == "plain 3") } @Test func appliedCorrectionRoundTripsThroughJSON() throws { let original = AppliedCorrection(heard: "cloud code", wrote: "Claude Code", rule: "cloud code", start: 12, length: 11) let data = try JSONEncoder().encode(original) let decoded = try JSONDecoder().decode(AppliedCorrection.self, from: data) #expect(decoded == original) } } ``` - [ ] **Step 2: Draai de test, moet falen** Run: `swift test --filter TranscriberVocabularyTests` Expected: compilatiefout `extra argument 'vocabulary' in call` / `cannot find 'AppliedCorrection' in scope`. - [ ] **Step 3: Implementatie** In `Sources/SstCore/Models.swift` vervang het blok ```swift public protocol Transcriber { func transcribe(_ samples: [Float]) async throws -> String } ``` door: ```swift public protocol Transcriber { func transcribe(_ samples: [Float]) async throws -> String /// Biases decoding towards `vocabulary`. Engines without bias support ignore the list. func transcribe(_ samples: [Float], vocabulary: [String]) async throws -> String } public extension Transcriber { func transcribe(_ samples: [Float], vocabulary: [String]) async throws -> String { try await transcribe(samples) } } /// One replacement made by the dictionary correction pass. /// `start`/`length` are UTF-16 offsets into the corrected text. public struct AppliedCorrection: Equatable, Codable, Sendable { public var heard: String public var wrote: String public var rule: String public var start: Int public var length: Int public init(heard: String, wrote: String, rule: String, start: Int, length: Int) { self.heard = heard self.wrote = wrote self.rule = rule self.start = start self.length = length } } public struct CorrectionResult: Equatable, Sendable { public var text: String public var corrections: [AppliedCorrection] public init(text: String, corrections: [AppliedCorrection]) { self.text = text self.corrections = corrections } } public protocol TextCorrecting { func correct(_ text: String) -> CorrectionResult } /// A finished dictation, emitted just before the text is inserted. public struct DictationResult: Equatable, Sendable { public var date: Date public var duration: TimeInterval public var rawText: String public var finalText: String public var corrections: [AppliedCorrection] public init(date: Date, duration: TimeInterval, rawText: String, finalText: String, corrections: [AppliedCorrection]) { self.date = date self.duration = duration self.rawText = rawText self.finalText = finalText self.corrections = corrections } } ``` - [ ] **Step 4: Draai alle tests** Run: `swift test` Expected: alle tests PASS (13 bestaande + 2 nieuwe). - [ ] **Step 5: Commit** ```bash git add Sources/SstCore/Models.swift Tests/SstCoreTests/TranscriberVocabularyTests.swift git commit -m "feat: add vocabulary-aware Transcriber and shared correction/result types" -m "Co-Authored-By: Claude Sonnet 5.5 " ``` --- ### Task 2: `DictationController` — voorrang, echo-filter, correctie en resultaat-hook **Files:** - Create: `Sources/SstCore/PromptEcho.swift` - Replace: `Sources/SstCore/DictationController.swift` - Modify: `Tests/SstCoreTests/Fakes.swift` (aanvullen onderaan) - Test: `Tests/SstCoreTests/PromptEchoTests.swift`, `Tests/SstCoreTests/DictationControllerHooksTests.swift` **Interfaces:** - Consumes: Task 1 (`Transcriber.transcribe(_:vocabulary:)`, `TextCorrecting`, `CorrectionResult`, `DictationResult`). - Produces: - `PromptEcho.isEcho(_ text: String, vocabulary: [String]) -> Bool` - `DictationController.init(recorder:transcriber:cleaner:inserter:minimumSamples:silenceThreshold:corrector:vocabularyProvider:biasMinimumRMS:)` — de nieuwe parameters zijn `corrector: TextCorrecting? = nil`, `vocabularyProvider: @escaping () -> [String] = { [] }`, `biasMinimumRMS: Float = 0.015` - `DictationController.onResult: ((DictationResult) -> Void)?` - [ ] **Step 1: Schrijf de falende tests** `Tests/SstCoreTests/PromptEchoTests.swift`: ```swift import Testing @testable import SstCore struct PromptEchoTests { let vocabulary = ["Anthropic", "Vercel", "Supabase"] @Test func firstTwoTermsAreAnEcho() { #expect(PromptEcho.isEcho("Anthropic, Vercel.", vocabulary: vocabulary)) } @Test func wholeListIsAnEcho() { #expect(PromptEcho.isEcho("anthropic vercel supabase", vocabulary: vocabulary)) } @Test func singleTermIsRealSpeech() { #expect(!PromptEcho.isEcho("Anthropic", vocabulary: vocabulary)) } @Test func termsOutOfOrderAreNotAnEcho() { #expect(!PromptEcho.isEcho("Vercel, Anthropic", vocabulary: vocabulary)) } @Test func sentenceContainingTermsIsNotAnEcho() { #expect(!PromptEcho.isEcho("Ik gebruik Anthropic en Vercel", vocabulary: vocabulary)) } @Test func tooShortVocabularyNeverEchoes() { #expect(!PromptEcho.isEcho("Anthropic", vocabulary: ["Anthropic"])) #expect(!PromptEcho.isEcho("", vocabulary: vocabulary)) } } ``` Voeg onderaan `Tests/SstCoreTests/Fakes.swift` toe: ```swift final class FakeVocabularyTranscriber: Transcriber { var result = "hallo wereld" private(set) var vocabularies: [[String]] = [] func transcribe(_ samples: [Float]) async throws -> String { result } func transcribe(_ samples: [Float], vocabulary: [String]) async throws -> String { vocabularies.append(vocabulary) return result } } struct UppercaseCorrector: TextCorrecting { func correct(_ text: String) -> CorrectionResult { let upper = text.uppercased() return CorrectionResult( text: upper, corrections: [AppliedCorrection(heard: text, wrote: upper, rule: "upper", start: 0, length: upper.utf16.count)] ) } } ``` `Tests/SstCoreTests/DictationControllerHooksTests.swift`: ```swift import Foundation import Testing @testable import SstCore @MainActor struct DictationControllerHooksTests { let recorder = FakeRecorder() let transcriber = FakeVocabularyTranscriber() let inserter = FakeInserter() static let loud = [Float](repeating: 0.1, count: 16_000) static let quiet = [Float](repeating: 0.01, count: 16_000) func makeController( cleaner: Cleaner = FakeCleaner(), corrector: TextCorrecting? = nil, vocabulary: [String] = [] ) -> DictationController { DictationController( recorder: recorder, transcriber: transcriber, cleaner: cleaner, inserter: inserter, corrector: corrector, vocabularyProvider: { vocabulary } ) } func dictate(_ controller: DictationController) async { await controller.handle(.pressed) await controller.handle(.released) } @Test func loudRecordingPassesVocabularyToTranscriber() async { recorder.samples = Self.loud await dictate(makeController(vocabulary: ["Anthropic", "Vercel"])) #expect(transcriber.vocabularies == [["Anthropic", "Vercel"]]) } @Test func quietRecordingSkipsVocabularyButStillTranscribes() async { recorder.samples = Self.quiet // RMS 0.01: above silence (0.005), below bias minimum (0.015) await dictate(makeController(vocabulary: ["Anthropic", "Vercel"])) #expect(transcriber.vocabularies == [[]]) #expect(inserter.inserted == ["[hallo wereld]"]) } @Test func promptEchoIsDiscarded() async { recorder.samples = Self.loud transcriber.result = "Anthropic, Vercel." let controller = makeController(vocabulary: ["Anthropic", "Vercel", "Supabase"]) var results: [DictationResult] = [] controller.onResult = { results.append($0) } await dictate(controller) #expect(inserter.inserted.isEmpty) #expect(results.isEmpty) #expect(controller.state == .idle) } @Test func singleVocabularyTermIsKept() async { recorder.samples = Self.loud transcriber.result = "Anthropic" await dictate(makeController(vocabulary: ["Anthropic", "Vercel", "Supabase"])) #expect(inserter.inserted == ["[Anthropic]"]) } @Test func correctorRunsBeforeCleanerAndCorrectionsSurviveAPassthroughCleaner() async { recorder.samples = Self.loud transcriber.result = "hallo" let controller = makeController(cleaner: PassthroughCleaner(), corrector: UppercaseCorrector()) var results: [DictationResult] = [] controller.onResult = { results.append($0) } await dictate(controller) #expect(inserter.inserted == ["HALLO"]) #expect(results.count == 1) #expect(results[0].rawText == "hallo") #expect(results[0].finalText == "HALLO") #expect(results[0].corrections.count == 1) } @Test func correctionsAreDroppedWhenTheCleanerChangesTheText() async { recorder.samples = Self.loud transcriber.result = "hallo" let controller = makeController(cleaner: FakeCleaner(), corrector: UppercaseCorrector()) var results: [DictationResult] = [] controller.onResult = { results.append($0) } await dictate(controller) #expect(results[0].finalText == "[HALLO]") #expect(results[0].corrections.isEmpty) } @Test func resultIsReportedEvenWhenInsertionFails() async { recorder.samples = Self.loud inserter.error = TestError() let controller = makeController() var results: [DictationResult] = [] var errors: [String] = [] controller.onResult = { results.append($0) } controller.onError = { errors.append($0) } await dictate(controller) #expect(results.count == 1) #expect(errors.count == 1) } @Test func resultCarriesDurationAndDate() async { recorder.samples = Self.loud // 16 000 samples = 1 s let controller = makeController() var results: [DictationResult] = [] controller.onResult = { results.append($0) } let before = Date() await dictate(controller) #expect(results[0].duration == 1.0) #expect(results[0].date >= before) #expect(results[0].corrections.isEmpty) } } ``` - [ ] **Step 2: Draai de tests, moeten falen** Run: `swift test --filter "PromptEchoTests|DictationControllerHooksTests"` Expected: compilatiefouten (`cannot find 'PromptEcho'`, `extra argument 'corrector'`). - [ ] **Step 3: Implementatie** `Sources/SstCore/PromptEcho.swift`: ```swift import Foundation /// Whisper sometimes answers a vocabulary prompt with the prompt itself when there is little speech. public enum PromptEcho { static func normalize(_ text: String) -> String { String(text.lowercased().map { $0.isLetter || $0.isNumber ? $0 : " " }) .split(separator: " ") .joined(separator: " ") } /// True when `text` is nothing but the first k ≥ 2 terms of the prompt. /// A single term on its own is valid speech ("Anthropic"). public static func isEcho(_ text: String, vocabulary: [String]) -> Bool { guard vocabulary.count >= 2 else { return false } let normalized = normalize(text) guard !normalized.isEmpty else { return false } return (2...vocabulary.count).contains { k in normalize(vocabulary.prefix(k).joined(separator: " ")) == normalized } } } ``` Vervang `Sources/SstCore/DictationController.swift` volledig door: ```swift import Foundation @MainActor public final class DictationController { public private(set) var state: DictationState = .idle { didSet { onStateChange?(state) } } public var onStateChange: ((DictationState) -> Void)? public var onError: ((String) -> Void)? /// Called with every finished transcription, just before it is inserted. public var onResult: ((DictationResult) -> Void)? private let recorder: AudioRecording private let transcriber: Transcriber private let cleaner: Cleaner private let inserter: TextInserting private let minimumSamples: Int private let silenceThreshold: Float private let corrector: TextCorrecting? private let vocabularyProvider: () -> [String] private let biasMinimumRMS: Float /// - Parameter minimumSamples: 16 kHz samples; default is 0.3 s. /// - Parameter silenceThreshold: RMS below which a recording counts as silence and is skipped. /// - Parameter corrector: optional dictionary correction applied to the transcript. /// - Parameter vocabularyProvider: words to bias the transcriber towards. /// - Parameter biasMinimumRMS: below this RMS no vocabulary is passed (guards against invented text on quiet audio). public init( recorder: AudioRecording, transcriber: Transcriber, cleaner: Cleaner, inserter: TextInserting, minimumSamples: Int = 4_800, silenceThreshold: Float = 0.005, corrector: TextCorrecting? = nil, vocabularyProvider: @escaping () -> [String] = { [] }, biasMinimumRMS: Float = 0.015 ) { self.recorder = recorder self.transcriber = transcriber self.cleaner = cleaner self.inserter = inserter self.minimumSamples = minimumSamples self.silenceThreshold = silenceThreshold self.corrector = corrector self.vocabularyProvider = vocabularyProvider self.biasMinimumRMS = biasMinimumRMS } public func handle(_ event: HotkeyEvent) async { switch (event, state) { case (.pressed, .idle): do { try recorder.start() state = .recording } catch { onError?(error.localizedDescription) } case (.released, .recording): let samples = recorder.stop() guard samples.count >= minimumSamples else { state = .idle return } let energy = samples.reduce(0) { $0 + $1 * $1 } let rms = (energy / Float(samples.count)).squareRoot() guard rms >= silenceThreshold else { state = .idle return } await process(samples, rms: rms) default: break } } private func process(_ samples: [Float], rms: Float) async { defer { state = .idle } do { state = .transcribing let vocabulary = rms >= biasMinimumRMS ? vocabularyProvider() : [] let raw = try await transcriber.transcribe(samples, vocabulary: vocabulary) let text = raw.trimmingCharacters(in: .whitespacesAndNewlines) guard !text.isEmpty, !PromptEcho.isEcho(text, vocabulary: vocabulary) else { return } let corrected = corrector?.correct(text) ?? CorrectionResult(text: text, corrections: []) let cleaned = try await cleaner.clean(corrected.text, context: CleaningContext()) onResult?(DictationResult( date: Date(), duration: Double(samples.count) / 16_000, rawText: text, finalText: cleaned, // Offsets only stay valid if the cleaner left the corrected text alone. corrections: cleaned == corrected.text ? corrected.corrections : [] )) state = .inserting try await inserter.insert(cleaned) } catch { onError?(error.localizedDescription) } } } ``` - [ ] **Step 4: Draai alle tests** Run: `swift test` Expected: alle tests PASS (de 13 + 2 + 6 + 8 nieuwe = 29). - [ ] **Step 5: Commit** ```bash git add Sources/SstCore Tests/SstCoreTests git commit -m "feat: controller supports vocabulary bias, prompt-echo guard, corrector and result hook" -m "Co-Authored-By: Claude Sonnet 5.5 " ``` --- ### Task 3: Instelbare sneltoets — `HotkeyBinding`, `ModifierKeyTracker`, `HotkeyMonitor` **Files:** - Create: `Sources/SstCore/HotkeyBinding.swift`, `Sources/SstCore/ModifierKeyTracker.swift` - Replace: `Sources/SstCore/HotkeyMonitor.swift` - Test: `Tests/SstCoreTests/HotkeyBindingTests.swift`, `Tests/SstCoreTests/ModifierKeyTrackerTests.swift` **Interfaces:** - Consumes: `HotkeyEvent` uit Models.swift. - Produces: - `struct HotkeyBinding: Hashable, Sendable, Identifiable` met `id: String`, `displayName: String`, `keyCode: Int`, `flagRawValue: UInt64`, `flag: CGEventFlags`; statics `.fn`, `.rightOption`, `.rightCommand`, `.rightControl`, `.leftControl`, `allowed: [HotkeyBinding]`, `binding(id:)`, `binding(keyCode:)` - `struct ModifierKeyTracker { init(); mutating func process(isDown: Bool) -> HotkeyEvent? }` - `HotkeyMonitor.binding: HotkeyBinding` (standaard `.fn`; wisselen herstelt de tracker) - [ ] **Step 1: Schrijf de falende tests** `Tests/SstCoreTests/HotkeyBindingTests.swift`: ```swift import Testing @testable import SstCore struct HotkeyBindingTests { @Test func allowedBindingsMatchTheSpec() { #expect(HotkeyBinding.allowed.map(\.id) == ["fn", "rightOption", "rightCommand", "rightControl", "leftControl"]) #expect(HotkeyBinding.allowed.map(\.keyCode) == [63, 61, 54, 62, 59]) } @Test func idsAndKeyCodesAreUnique() { #expect(Set(HotkeyBinding.allowed.map(\.id)).count == HotkeyBinding.allowed.count) #expect(Set(HotkeyBinding.allowed.map(\.keyCode)).count == HotkeyBinding.allowed.count) } @Test func lookupByKeyCodeAndId() { #expect(HotkeyBinding.binding(keyCode: 61) == .rightOption) #expect(HotkeyBinding.binding(id: "leftControl") == .leftControl) } @Test func unsupportedKeysAreRejected() { #expect(HotkeyBinding.binding(keyCode: 56) == nil) // left shift #expect(HotkeyBinding.binding(keyCode: 57) == nil) // caps lock #expect(HotkeyBinding.binding(id: "nope") == nil) } @Test func everyBindingHasAFlagAndName() { for binding in HotkeyBinding.allowed { #expect(binding.flagRawValue != 0) #expect(!binding.displayName.isEmpty) } } } ``` `Tests/SstCoreTests/ModifierKeyTrackerTests.swift`: ```swift import Testing @testable import SstCore struct ModifierKeyTrackerTests { @Test func emitsPressedThenReleasedOnlyOnTransitions() { var tracker = ModifierKeyTracker() #expect(tracker.process(isDown: false) == nil) #expect(tracker.process(isDown: true) == .pressed) #expect(tracker.process(isDown: true) == nil) #expect(tracker.process(isDown: false) == .released) #expect(tracker.process(isDown: false) == nil) } @Test func freshTrackerStartsReleased() { var tracker = ModifierKeyTracker() #expect(tracker.process(isDown: true) == .pressed) } } ``` - [ ] **Step 2: Draai de tests, moeten falen** Run: `swift test --filter "HotkeyBindingTests|ModifierKeyTrackerTests"` Expected: compilatiefout `cannot find 'HotkeyBinding' in scope`. - [ ] **Step 3: Implementatie** `Sources/SstCore/ModifierKeyTracker.swift`: ```swift public struct ModifierKeyTracker { private var isDown = false public init() {} /// Feed the current state of the bound modifier flag; returns an event only on a transition. public mutating func process(isDown: Bool) -> HotkeyEvent? { guard isDown != self.isDown else { return nil } self.isDown = isDown return isDown ? .pressed : .released } } ``` `Sources/SstCore/HotkeyBinding.swift`: ```swift import CoreGraphics /// A modifier key that can be held to dictate. Left/right keys share a flag, so the key code decides. public struct HotkeyBinding: Hashable, Sendable, Identifiable { public let id: String public let displayName: String public let keyCode: Int public let flagRawValue: UInt64 public var flag: CGEventFlags { CGEventFlags(rawValue: flagRawValue) } public static let fn = HotkeyBinding( id: "fn", displayName: "Fn", keyCode: 63, flagRawValue: CGEventFlags.maskSecondaryFn.rawValue) public static let rightOption = HotkeyBinding( id: "rightOption", displayName: "Rechter Option", keyCode: 61, flagRawValue: CGEventFlags.maskAlternate.rawValue) public static let rightCommand = HotkeyBinding( id: "rightCommand", displayName: "Rechter Command", keyCode: 54, flagRawValue: CGEventFlags.maskCommand.rawValue) public static let rightControl = HotkeyBinding( id: "rightControl", displayName: "Rechter Control", keyCode: 62, flagRawValue: CGEventFlags.maskControl.rawValue) public static let leftControl = HotkeyBinding( id: "leftControl", displayName: "Linker Control", keyCode: 59, flagRawValue: CGEventFlags.maskControl.rawValue) public static let allowed: [HotkeyBinding] = [.fn, .rightOption, .rightCommand, .rightControl, .leftControl] public static func binding(id: String) -> HotkeyBinding? { allowed.first { $0.id == id } } public static func binding(keyCode: Int) -> HotkeyBinding? { allowed.first { $0.keyCode == keyCode } } } ``` Vervang `Sources/SstCore/HotkeyMonitor.swift` volledig door: ```swift import ApplicationServices import CoreGraphics public final class HotkeyMonitor { public enum HotkeyError: Error, LocalizedError { case tapUnavailable public var errorDescription: String? { "Kan de sneltoets niet volgen: geef Input Monitoring-rechten in Systeeminstellingen en probeer opnieuw (soms is een herstart van de app nodig)." } } public var onEvent: ((HotkeyEvent) -> Void)? /// The key that has to be held. Changing it resets the press tracking. public var binding: HotkeyBinding = .fn { didSet { tracker = ModifierKeyTracker() } } private var tracker = ModifierKeyTracker() private var tap: CFMachPort? private var source: CFRunLoopSource? public init() {} public func start() throws { guard tap == nil else { return } let mask = CGEventMask(1 << CGEventType.flagsChanged.rawValue) let callback: CGEventTapCallBack = { _, type, event, refcon in if let refcon { let monitor = Unmanaged.fromOpaque(refcon).takeUnretainedValue() monitor.handle(type: type, event: event) } return Unmanaged.passUnretained(event) } guard let tap = CGEvent.tapCreate( tap: .cgSessionEventTap, place: .headInsertEventTap, options: .listenOnly, eventsOfInterest: mask, callback: callback, userInfo: Unmanaged.passUnretained(self).toOpaque() ) else { throw HotkeyError.tapUnavailable } let source = CFMachPortCreateRunLoopSource(kCFAllocatorDefault, tap, 0) CFRunLoopAddSource(CFRunLoopGetMain(), source, .commonModes) CGEvent.tapEnable(tap: tap, enable: true) self.tap = tap self.source = source } public func stop() { if let source { CFRunLoopRemoveSource(CFRunLoopGetMain(), source, .commonModes) } if let tap { CGEvent.tapEnable(tap: tap, enable: false) } tap = nil source = nil } private func handle(type: CGEventType, event: CGEvent) { switch type { case .tapDisabledByTimeout, .tapDisabledByUserInput: if let tap { CGEvent.tapEnable(tap: tap, enable: true) } case .flagsChanged: // Flags are shared between left and right keys and with arrows/F-keys: only react to the bound key itself. guard event.getIntegerValueField(.keyboardEventKeycode) == Int64(binding.keyCode) else { return } let isDown = event.flags.contains(binding.flag) if let hotkeyEvent = tracker.process(isDown: isDown) { onEvent?(hotkeyEvent) } default: break } } } ``` > `FnKeyTracker` en zijn test blijven ongewijzigd bestaan; `HotkeyMonitor` gebruikt hem niet meer. - [ ] **Step 4: Draai alle tests en bouw** Run: `swift test && swift build` Expected: alle tests PASS; build slaagt (de app en `SstProbe` gebruiken `HotkeyMonitor()` zonder argumenten en blijven dus compileren). - [ ] **Step 5: Handmatige controle (met de gebruiker, na Task 5)** Wordt gebundeld in Task 5, stap 6. - [ ] **Step 6: Commit** ```bash git add Sources/SstCore Tests/SstCoreTests git commit -m "feat: configurable hotkey binding and ModifierKeyTracker" -m "Co-Authored-By: Claude Sonnet 5.5 " ``` --- ### Task 4: Niveau-uitvoer — `AudioLevel` en `AudioRecorder.onLevel` **Files:** - Create: `Sources/SstCore/AudioLevel.swift` - Modify: `Sources/SstCore/AudioRecorder.swift` - Modify: `Sources/SstProbe/RecordProbe.swift` - Test: `Tests/SstCoreTests/AudioLevelTests.swift` **Interfaces:** - Produces: - `AudioLevel.rmsDbfs(_ samples: [Float]) -> Float` en `rmsDbfs(_ samples: UnsafeBufferPointer) -> Float` (−120 voor stilte of leeg) - `AudioLevel.vu(fromDbfs:referenceDbfs:) -> Float` (standaard referentie −20) - `AudioRecorder.onLevel: ((Float) -> Void)?` — RMS in dBFS per aangeleverde buffer, aangeroepen op de audio-thread; vóór `start()` instellen. - [ ] **Step 1: Schrijf de falende test** `Tests/SstCoreTests/AudioLevelTests.swift`: ```swift import Testing @testable import SstCore struct AudioLevelTests { @Test func constantTenthIsMinusTwentyDbfs() { let level = AudioLevel.rmsDbfs([Float](repeating: 0.1, count: 1_000)) #expect(abs(level - (-20)) < 0.01) } @Test func fullScaleIsZeroDbfs() { let level = AudioLevel.rmsDbfs([Float](repeating: 1.0, count: 100)) #expect(abs(level) < 0.01) } @Test func negativeSamplesCountLikePositiveOnes() { let level = AudioLevel.rmsDbfs([Float](repeating: -0.1, count: 1_000)) #expect(abs(level - (-20)) < 0.01) } @Test func emptyAndSilentBuffersHitTheFloor() { #expect(AudioLevel.rmsDbfs([]) == AudioLevel.silenceFloorDbfs) #expect(AudioLevel.rmsDbfs([Float](repeating: 0, count: 64)) == AudioLevel.silenceFloorDbfs) } @Test func vuIsRelativeToTheReference() { #expect(AudioLevel.vu(fromDbfs: -20) == 0) #expect(AudioLevel.vu(fromDbfs: -40) == -20) #expect(AudioLevel.vu(fromDbfs: -17) == 3) #expect(AudioLevel.vu(fromDbfs: -30, referenceDbfs: -18) == -12) } } ``` - [ ] **Step 2: Draai de test, moet falen** Run: `swift test --filter AudioLevelTests` Expected: compilatiefout `cannot find 'AudioLevel' in scope`. - [ ] **Step 3: Implementatie** `Sources/SstCore/AudioLevel.swift`: ```swift import Foundation public enum AudioLevel { /// Reported for empty or silent buffers (log of zero would be -infinity). public static let silenceFloorDbfs: Float = -120 public static func rmsDbfs(_ samples: [Float]) -> Float { samples.withUnsafeBufferPointer { rmsDbfs($0) } } public static func rmsDbfs(_ samples: UnsafeBufferPointer) -> Float { guard !samples.isEmpty else { return silenceFloorDbfs } var sum: Float = 0 for sample in samples { sum += sample * sample } let rms = (sum / Float(samples.count)).squareRoot() guard rms > 0 else { return silenceFloorDbfs } return max(silenceFloorDbfs, 20 * log10(rms)) } /// VU reading for a dBFS level, where `referenceDbfs` is 0 VU. public static func vu(fromDbfs dbfs: Float, referenceDbfs: Float = -20) -> Float { dbfs - referenceDbfs } } ``` In `Sources/SstCore/AudioRecorder.swift`: 1. Voeg onder `private var buffer: [Float] = []` toe: ```swift /// RMS level (dBFS) of every delivered input buffer. Called on the audio thread; set before `start()`. public var onLevel: ((Float) -> Void)? ``` 2. Vervang in de tap-closure de regel `guard let self else { return }` door: ```swift guard let self else { return } if let onLevel = self.onLevel, let channel = pcm.floatChannelData?[0] { onLevel(AudioLevel.rmsDbfs(UnsafeBufferPointer(start: channel, count: Int(pcm.frameLength)))) } ``` 3. Vervang `bufferSize: 4096` door `bufferSize: 1024` (macOS mag een grotere buffer leveren; de meter werkt met elke grootte). `Sources/SstProbe/RecordProbe.swift` — vervang volledig door: ```swift import AVFoundation import SstCore /// Records `seconds` of microphone audio; returns nil if access is denied or recording fails. func recordSamples(seconds: Double, onLevel: ((Float) -> Void)? = nil) async -> [Float]? { guard await AVCaptureDevice.requestAccess(for: .audio) else { print("Microfoon-toegang geweigerd (Systeeminstellingen → Privacy → Microfoon).") return nil } let recorder = AudioRecorder() recorder.onLevel = onLevel do { try recorder.start() } catch { print("start mislukt:", error.localizedDescription) return nil } print("Neem \(Int(seconds)) s op — praat nu") try? await Task.sleep(for: .seconds(seconds)) return recorder.stop() } func runRecordProbe() async { final class Levels: @unchecked Sendable { let lock = NSLock() var values: [Float] = [] func add(_ v: Float) { lock.withLock { values.append(v) } } } let levels = Levels() guard let samples = await recordSamples(seconds: 3, onLevel: { levels.add($0) }) else { return } let peak = samples.map { abs($0) }.max() ?? 0 print("samples: \(samples.count) (verwacht ≈ 48000), piek: \(peak)") let values = levels.lock.withLock { levels.values } if let lo = values.min(), let hi = values.max() { print(String(format: "niveau-meldingen: %d (≈ %.0f per s), bereik %.1f … %.1f dBFS", values.count, Double(values.count) / 3, lo, hi)) } else { print("geen niveau-meldingen ontvangen") } } ``` - [ ] **Step 4: Draai alle tests en bouw** Run: `swift test && swift build` Expected: alle tests PASS; build slaagt. - [ ] **Step 5: Handmatige controle (met de gebruiker, in Task 5, stap 6)** Hier alleen: de code compileert. - [ ] **Step 6: Commit** ```bash git add Sources Tests git commit -m "feat: AudioLevel helper and AudioRecorder.onLevel" -m "Co-Authored-By: Claude Sonnet 5.5 " ``` --- ### Task 5: WhisperKit-voorrang — woordenlijst als `promptTokens` **Files:** - Modify: `Package.swift` (testtarget `SstWhisperTests`) - Modify: `Sources/SstWhisper/WhisperKitTranscriber.swift` - Modify: `Sources/SstProbe/TranscribeProbe.swift` - Test: `Tests/SstWhisperTests/PromptTokensTests.swift` **Interfaces:** - Consumes: Task 1 (`Transcriber.transcribe(_:vocabulary:)`). - Produces: `WhisperKitTranscriber.transcribe(_:vocabulary:)`; intern `static func promptTokens(vocabulary:maxTokens:specialTokenBegin:encode:) -> [Int]?`. - [ ] **Step 1: Testtarget toevoegen** In `Package.swift` voeg onder de bestaande `.testTarget(name: "SstCoreTests", …)` toe: ```swift .testTarget(name: "SstWhisperTests", dependencies: ["SstWhisper"]), ``` - [ ] **Step 2: Schrijf de falende test** `Tests/SstWhisperTests/PromptTokensTests.swift`: ```swift import Testing @testable import SstWhisper struct PromptTokensTests { /// Fake tokenizer: one token (id 7) per word, plus one special token (id 2000) that must be filtered out. let encode: (String) -> [Int] = { text in text.split(whereSeparator: { !$0.isLetter }).map { _ in 7 } + [2000] } @Test func allTermsFitAndSpecialTokensAreFiltered() { let tokens = WhisperKitTranscriber.promptTokens( vocabulary: ["Anthropic", "Vercel", "Supabase"], maxTokens: 64, specialTokenBegin: 1000, encode: encode) #expect(tokens == [7, 7, 7]) } @Test func termsThatDoNotFitAreDroppedFromTheEnd() { let vocab = ["a", "b", "c", "d", "e", "f", "g", "h", "i", "j"] let tokens = WhisperKitTranscriber.promptTokens( vocabulary: vocab, maxTokens: 4, specialTokenBegin: 1000, encode: encode) #expect(tokens?.count == 4) } @Test func emptyVocabularyGivesNoPrompt() { #expect(WhisperKitTranscriber.promptTokens( vocabulary: [], maxTokens: 64, specialTokenBegin: 1000, encode: encode) == nil) } @Test func firstTermAloneTooLongGivesNoPrompt() { #expect(WhisperKitTranscriber.promptTokens( vocabulary: ["one two three"], maxTokens: 2, specialTokenBegin: 1000, encode: encode) == nil) } } ``` - [ ] **Step 3: Draai de test, moet falen** Run: `swift test --filter PromptTokensTests` Expected: compilatiefout `type 'WhisperKitTranscriber' has no member 'promptTokens'`. - [ ] **Step 4: Implementatie** In `Sources/SstWhisper/WhisperKitTranscriber.swift` vervang de hele methode `public func transcribe(_ samples: [Float]) async throws -> String { … }` (onderaan het bestand) door: ```swift public func transcribe(_ samples: [Float]) async throws -> String { try await transcribe(samples, vocabulary: []) } /// Terms are added one by one while the encoded prompt stays within `maxTokens`. /// Returns nil when nothing fits or the list is empty. static func promptTokens( vocabulary: [String], maxTokens: Int = 64, specialTokenBegin: Int, encode: (String) -> [Int] ) -> [Int]? { var kept: [String] = [] var tokens: [Int] = [] for term in vocabulary { let candidate = kept + [term] let encoded = encode(" " + candidate.joined(separator: ", ")).filter { $0 < specialTokenBegin } if encoded.count > maxTokens { break } kept = candidate tokens = encoded } return tokens.isEmpty ? nil : tokens } public func transcribe(_ samples: [Float], vocabulary: [String]) async throws -> String { guard let pipeline else { throw TranscriberError.notLoaded } // language nil + detectLanguage: Whisper picks NL or EN per utterance. var options = DecodingOptions( task: .transcribe, language: nil, detectLanguage: true, skipSpecialTokens: true, withoutTimestamps: true ) if !vocabulary.isEmpty, let tokenizer = pipeline.tokenizer { options.promptTokens = Self.promptTokens( vocabulary: vocabulary, specialTokenBegin: tokenizer.specialTokens.specialTokenBegin, encode: { tokenizer.encode(text: $0) } ) } let results = try await pipeline.transcribe(audioArray: samples, decodeOptions: options) return results .map(\.text) .joined(separator: " ") .trimmingCharacters(in: .whitespacesAndNewlines) } ``` > Wijkt de WhisperKit-API licht af (bijvoorbeeld de plaats van `specialTokens`), pas dan alleen dit bestand aan en behoud de signatuur van `promptTokens` zodat de test geldig blijft. `Sources/SstProbe/TranscribeProbe.swift` — vervang regel `let text = try await transcriber.transcribe(samples)` door: ```swift let vocabulary = (ProcessInfo.processInfo.environment["SST_VOCAB"] ?? "") .split(separator: ",").map { $0.trimmingCharacters(in: .whitespaces) }.filter { !$0.isEmpty } if !vocabulary.isEmpty { print("voorrang: \(vocabulary.joined(separator: ", "))") } let text = try await transcriber.transcribe(samples, vocabulary: vocabulary) ``` - [ ] **Step 5: Draai alle tests en bouw** Run: `swift test && swift build` Expected: alle tests PASS (incl. 4 nieuwe in `SstWhisperTests`); build slaagt. - [ ] **Step 6: Handmatige controle samen met de gebruiker (hardware)** Doe in deze volgorde en noteer de uitkomsten in het commit-bericht van stap 7: 1. `swift run SstProbe record` → er verschijnt een regel `niveau-meldingen: N (≈ X per s), bereik … dBFS`. Verwacht: meer dan 5 meldingen per seconde en een bereik dat bij praten duidelijk hoger ligt dan bij stilte. Noteer X (macOS levert soms ~10 per s in plaats van ~47). 2. `swift run SstProbe transcribe` — zeg een zin, noteer de tijd **zonder** voorrang. 3. `SST_VOCAB="Anthropic,Vercel,Supabase,Claude Code" swift run SstProbe transcribe` — zeg dezelfde zin met een of twee van die woorden erin ("Ik werk met Supabase en Claude Code"); noteer de tijd en of de spelling beter is. Verwacht: tijd onder 1,5 s voor 8 s spraak. Is het trager, noteer hoeveel; dat is geen blokkade, wel informatie voor de spec. 4. Zeg niets (8 s stilte) met `SST_VOCAB` gezet: het resultaat mag geen woorden uit de lijst bevatten. 5. Hotkey-controle: `swift run SstProbe hotkey` toont nog steeds `pressed`/`released` voor Fn. - [ ] **Step 7: Commit** ```bash git add Package.swift Sources Tests git commit -m "feat: WhisperKit transcriber biases decoding with vocabulary prompt tokens" -m "Co-Authored-By: Claude Sonnet 5.5 " ``` --- ## Self-review - **Spec-dekking:** raakpunt 1 (woordenlijst → `promptTokens`, Task 1+5), 2 (niveau, Task 4), 3 (corrector + `onResult`, Task 1+2), 4 (instelbare toets, Task 3). Voorrang-regels: RMS-grens 0,015 (Task 2 test `quietRecordingSkipsVocabularyButStillTranscribes`), echo-filter (Task 2), 64 tokens (Task 5). Niet in deze fase, wel in fase 2: het opbouwen van de woordenlijst (`VocabularyBuilder`) en de correctie zelf. - **Placeholders:** geen; de waarschuwing in Task 5 stap 4 is een gerichte terugval, geen open eind. - **Typeconsistentie:** `AppliedCorrection(heard:wrote:rule:start:length:)`, `CorrectionResult(text:corrections:)` en `DictationResult(date:duration:rawText:finalText:corrections:)` worden in Task 2 precies zo gebruikt als in Task 1 gedefinieerd. `HotkeyMonitor.binding` gebruikt `HotkeyBinding.keyCode: Int` en `flag`. - **Risico's:** de bufferlevering van `installTap` (Task 4, stap 6.1) en de snelheid met prompt (stap 6.3) zijn de enige onbekenden; beide worden gemeten.