Files
sst/docs/superpowers/plans/2026-10-01-app-1-engine-hooks.md
T

1102 lines
42 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# App fase 1 — Engine-raakpunten Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** De engine krijgt vier additieve uitbreidingen (woordenlijst voor de transcriber, niveau-uitvoer, resultaat-hook met correctie, instelbare sneltoets) zonder dat bestaand gedrag verandert.
**Architecture:** Alle wijzigingen zijn additief met standaardwaarden die het huidige gedrag behouden. Nieuwe pure logica (`PromptEcho`, `AudioLevel`, `HotkeyBinding`, `ModifierKeyTracker`, `promptTokens`) staat los van hardware en wordt met unit-tests gedekt; de hardware-afhankelijke stukken (`AudioRecorder`, `HotkeyMonitor`, WhisperKit) zijn dun en worden handmatig gecontroleerd.
**Tech Stack:** Swift 6.4 (language mode 5), Swift Testing, AVFoundation, CoreGraphics, WhisperKit 0.18.
**Spec:** `docs/superpowers/specs/2026-10-01-macos-app-design.md` (secties "Engine-raakpunten", "Woordenboek — mechanisme 1")
## Global Constraints
- Bestaande tests (13 stuks) blijven ongewijzigd groen; bestaande testbestanden worden alleen *aangevuld* (`Fakes.swift`) of krijgen er een nieuw bestand naast.
- Nieuwe parameters hebben standaardwaarden die het huidige gedrag behouden.
- Vocabulaire: maximaal 64 tokens in de prompt; prompt alleen bij opnames met RMS ≥ 0,015; stiltefilter blijft 0,005.
- Prompt-echo: resultaat dat uitsluitend uit de eerste *k* ≥ 2 termen van de prompt bestaat wordt verworpen; één term alleen niet.
- Niveau: `onLevel` levert RMS in dBFS; 0 VU = −20 dBFS.
- Toegestane sneltoetsen (exact): Fn (keycode 63), Rechter Option (61), Rechter Command (54), Rechter Control (62), Linker Control (59).
- Elke commit eindigt met de trailer `Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>` (tweede `-m`).
## Bestandsstructuur
```
Sources/SstCore/Models.swift (wijzig) Transcriber + AppliedCorrection, CorrectionResult, TextCorrecting, DictationResult
Sources/SstCore/PromptEcho.swift (nieuw)
Sources/SstCore/DictationController.swift(vervang) corrector, onResult, vocabularyProvider
Sources/SstCore/HotkeyBinding.swift (nieuw)
Sources/SstCore/ModifierKeyTracker.swift (nieuw)
Sources/SstCore/HotkeyMonitor.swift (vervang) binding
Sources/SstCore/AudioLevel.swift (nieuw)
Sources/SstCore/AudioRecorder.swift (wijzig) onLevel
Sources/SstWhisper/WhisperKitTranscriber.swift (wijzig) vocabulary → promptTokens
Sources/SstProbe/RecordProbe.swift, TranscribeProbe.swift (wijzig) handmatige controle
Package.swift (wijzig) SstWhisperTests
Tests/SstCoreTests/Fakes.swift (aanvullen)
Tests/SstCoreTests/TranscriberVocabularyTests.swift, PromptEchoTests.swift,
DictationControllerHooksTests.swift, HotkeyBindingTests.swift,
ModifierKeyTrackerTests.swift, AudioLevelTests.swift (nieuw)
Tests/SstWhisperTests/PromptTokensTests.swift (nieuw)
```
---
### Task 1: Gedeelde typen en woordenlijst op `Transcriber`
**Files:**
- Modify: `Sources/SstCore/Models.swift`
- Test: `Tests/SstCoreTests/TranscriberVocabularyTests.swift`
**Interfaces:**
- Produces (gebruikt door alle latere fasen):
- `Transcriber.transcribe(_ samples: [Float], vocabulary: [String]) async throws -> String` (protocol-eis, met standaardimplementatie die `transcribe(_:)` aanroept)
- `struct AppliedCorrection: Equatable, Codable, Sendable { heard, wrote, rule: String; start, length: Int }` met publieke init `init(heard:wrote:rule:start:length:)`
- `struct CorrectionResult: Equatable, Sendable { text: String; corrections: [AppliedCorrection] }` met publieke init `init(text:corrections:)`
- `protocol TextCorrecting { func correct(_ text: String) -> CorrectionResult }`
- `struct DictationResult: Equatable, Sendable { date: Date; duration: TimeInterval; rawText: String; finalText: String; corrections: [AppliedCorrection] }` met publieke init in die volgorde
- [ ] **Step 1: Schrijf de falende test**
`Tests/SstCoreTests/TranscriberVocabularyTests.swift`:
```swift
import Testing
@testable import SstCore
private struct PlainTranscriber: Transcriber {
func transcribe(_ samples: [Float]) async throws -> String { "plain \(samples.count)" }
}
struct TranscriberVocabularyTests {
@Test func vocabularyVariantFallsBackToPlainTranscribe() async throws {
let result = try await PlainTranscriber().transcribe([0, 0, 0], vocabulary: ["Anthropic"])
#expect(result == "plain 3")
}
@Test func appliedCorrectionRoundTripsThroughJSON() throws {
let original = AppliedCorrection(heard: "cloud code", wrote: "Claude Code", rule: "cloud code", start: 12, length: 11)
let data = try JSONEncoder().encode(original)
let decoded = try JSONDecoder().decode(AppliedCorrection.self, from: data)
#expect(decoded == original)
}
}
```
- [ ] **Step 2: Draai de test, moet falen**
Run: `swift test --filter TranscriberVocabularyTests`
Expected: compilatiefout `extra argument 'vocabulary' in call` / `cannot find 'AppliedCorrection' in scope`.
- [ ] **Step 3: Implementatie**
In `Sources/SstCore/Models.swift` vervang het blok
```swift
public protocol Transcriber {
func transcribe(_ samples: [Float]) async throws -> String
}
```
door:
```swift
public protocol Transcriber {
func transcribe(_ samples: [Float]) async throws -> String
/// Biases decoding towards `vocabulary`. Engines without bias support ignore the list.
func transcribe(_ samples: [Float], vocabulary: [String]) async throws -> String
}
public extension Transcriber {
func transcribe(_ samples: [Float], vocabulary: [String]) async throws -> String {
try await transcribe(samples)
}
}
/// One replacement made by the dictionary correction pass.
/// `start`/`length` are UTF-16 offsets into the corrected text.
public struct AppliedCorrection: Equatable, Codable, Sendable {
public var heard: String
public var wrote: String
public var rule: String
public var start: Int
public var length: Int
public init(heard: String, wrote: String, rule: String, start: Int, length: Int) {
self.heard = heard
self.wrote = wrote
self.rule = rule
self.start = start
self.length = length
}
}
public struct CorrectionResult: Equatable, Sendable {
public var text: String
public var corrections: [AppliedCorrection]
public init(text: String, corrections: [AppliedCorrection]) {
self.text = text
self.corrections = corrections
}
}
public protocol TextCorrecting {
func correct(_ text: String) -> CorrectionResult
}
/// A finished dictation, emitted just before the text is inserted.
public struct DictationResult: Equatable, Sendable {
public var date: Date
public var duration: TimeInterval
public var rawText: String
public var finalText: String
public var corrections: [AppliedCorrection]
public init(date: Date, duration: TimeInterval, rawText: String, finalText: String, corrections: [AppliedCorrection]) {
self.date = date
self.duration = duration
self.rawText = rawText
self.finalText = finalText
self.corrections = corrections
}
}
```
- [ ] **Step 4: Draai alle tests**
Run: `swift test`
Expected: alle tests PASS (13 bestaande + 2 nieuwe).
- [ ] **Step 5: Commit**
```bash
git add Sources/SstCore/Models.swift Tests/SstCoreTests/TranscriberVocabularyTests.swift
git commit -m "feat: add vocabulary-aware Transcriber and shared correction/result types" -m "Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"
```
---
### Task 2: `DictationController` — voorrang, echo-filter, correctie en resultaat-hook
**Files:**
- Create: `Sources/SstCore/PromptEcho.swift`
- Replace: `Sources/SstCore/DictationController.swift`
- Modify: `Tests/SstCoreTests/Fakes.swift` (aanvullen onderaan)
- Test: `Tests/SstCoreTests/PromptEchoTests.swift`, `Tests/SstCoreTests/DictationControllerHooksTests.swift`
**Interfaces:**
- Consumes: Task 1 (`Transcriber.transcribe(_:vocabulary:)`, `TextCorrecting`, `CorrectionResult`, `DictationResult`).
- Produces:
- `PromptEcho.isEcho(_ text: String, vocabulary: [String]) -> Bool`
- `DictationController.init(recorder:transcriber:cleaner:inserter:minimumSamples:silenceThreshold:corrector:vocabularyProvider:biasMinimumRMS:)` — de nieuwe parameters zijn `corrector: TextCorrecting? = nil`, `vocabularyProvider: @escaping () -> [String] = { [] }`, `biasMinimumRMS: Float = 0.015`
- `DictationController.onResult: ((DictationResult) -> Void)?`
- [ ] **Step 1: Schrijf de falende tests**
`Tests/SstCoreTests/PromptEchoTests.swift`:
```swift
import Testing
@testable import SstCore
struct PromptEchoTests {
let vocabulary = ["Anthropic", "Vercel", "Supabase"]
@Test func firstTwoTermsAreAnEcho() {
#expect(PromptEcho.isEcho("Anthropic, Vercel.", vocabulary: vocabulary))
}
@Test func wholeListIsAnEcho() {
#expect(PromptEcho.isEcho("anthropic vercel supabase", vocabulary: vocabulary))
}
@Test func singleTermIsRealSpeech() {
#expect(!PromptEcho.isEcho("Anthropic", vocabulary: vocabulary))
}
@Test func termsOutOfOrderAreNotAnEcho() {
#expect(!PromptEcho.isEcho("Vercel, Anthropic", vocabulary: vocabulary))
}
@Test func sentenceContainingTermsIsNotAnEcho() {
#expect(!PromptEcho.isEcho("Ik gebruik Anthropic en Vercel", vocabulary: vocabulary))
}
@Test func tooShortVocabularyNeverEchoes() {
#expect(!PromptEcho.isEcho("Anthropic", vocabulary: ["Anthropic"]))
#expect(!PromptEcho.isEcho("", vocabulary: vocabulary))
}
}
```
Voeg onderaan `Tests/SstCoreTests/Fakes.swift` toe:
```swift
final class FakeVocabularyTranscriber: Transcriber {
var result = "hallo wereld"
private(set) var vocabularies: [[String]] = []
func transcribe(_ samples: [Float]) async throws -> String { result }
func transcribe(_ samples: [Float], vocabulary: [String]) async throws -> String {
vocabularies.append(vocabulary)
return result
}
}
struct UppercaseCorrector: TextCorrecting {
func correct(_ text: String) -> CorrectionResult {
let upper = text.uppercased()
return CorrectionResult(
text: upper,
corrections: [AppliedCorrection(heard: text, wrote: upper, rule: "upper", start: 0, length: upper.utf16.count)]
)
}
}
```
`Tests/SstCoreTests/DictationControllerHooksTests.swift`:
```swift
import Foundation
import Testing
@testable import SstCore
@MainActor
struct DictationControllerHooksTests {
let recorder = FakeRecorder()
let transcriber = FakeVocabularyTranscriber()
let inserter = FakeInserter()
static let loud = [Float](repeating: 0.1, count: 16_000)
static let quiet = [Float](repeating: 0.01, count: 16_000)
func makeController(
cleaner: Cleaner = FakeCleaner(),
corrector: TextCorrecting? = nil,
vocabulary: [String] = []
) -> DictationController {
DictationController(
recorder: recorder,
transcriber: transcriber,
cleaner: cleaner,
inserter: inserter,
corrector: corrector,
vocabularyProvider: { vocabulary }
)
}
func dictate(_ controller: DictationController) async {
await controller.handle(.pressed)
await controller.handle(.released)
}
@Test func loudRecordingPassesVocabularyToTranscriber() async {
recorder.samples = Self.loud
await dictate(makeController(vocabulary: ["Anthropic", "Vercel"]))
#expect(transcriber.vocabularies == [["Anthropic", "Vercel"]])
}
@Test func quietRecordingSkipsVocabularyButStillTranscribes() async {
recorder.samples = Self.quiet // RMS 0.01: above silence (0.005), below bias minimum (0.015)
await dictate(makeController(vocabulary: ["Anthropic", "Vercel"]))
#expect(transcriber.vocabularies == [[]])
#expect(inserter.inserted == ["[hallo wereld]"])
}
@Test func promptEchoIsDiscarded() async {
recorder.samples = Self.loud
transcriber.result = "Anthropic, Vercel."
let controller = makeController(vocabulary: ["Anthropic", "Vercel", "Supabase"])
var results: [DictationResult] = []
controller.onResult = { results.append($0) }
await dictate(controller)
#expect(inserter.inserted.isEmpty)
#expect(results.isEmpty)
#expect(controller.state == .idle)
}
@Test func singleVocabularyTermIsKept() async {
recorder.samples = Self.loud
transcriber.result = "Anthropic"
await dictate(makeController(vocabulary: ["Anthropic", "Vercel", "Supabase"]))
#expect(inserter.inserted == ["[Anthropic]"])
}
@Test func correctorRunsBeforeCleanerAndCorrectionsSurviveAPassthroughCleaner() async {
recorder.samples = Self.loud
transcriber.result = "hallo"
let controller = makeController(cleaner: PassthroughCleaner(), corrector: UppercaseCorrector())
var results: [DictationResult] = []
controller.onResult = { results.append($0) }
await dictate(controller)
#expect(inserter.inserted == ["HALLO"])
#expect(results.count == 1)
#expect(results[0].rawText == "hallo")
#expect(results[0].finalText == "HALLO")
#expect(results[0].corrections.count == 1)
}
@Test func correctionsAreDroppedWhenTheCleanerChangesTheText() async {
recorder.samples = Self.loud
transcriber.result = "hallo"
let controller = makeController(cleaner: FakeCleaner(), corrector: UppercaseCorrector())
var results: [DictationResult] = []
controller.onResult = { results.append($0) }
await dictate(controller)
#expect(results[0].finalText == "[HALLO]")
#expect(results[0].corrections.isEmpty)
}
@Test func resultIsReportedEvenWhenInsertionFails() async {
recorder.samples = Self.loud
inserter.error = TestError()
let controller = makeController()
var results: [DictationResult] = []
var errors: [String] = []
controller.onResult = { results.append($0) }
controller.onError = { errors.append($0) }
await dictate(controller)
#expect(results.count == 1)
#expect(errors.count == 1)
}
@Test func resultCarriesDurationAndDate() async {
recorder.samples = Self.loud // 16 000 samples = 1 s
let controller = makeController()
var results: [DictationResult] = []
controller.onResult = { results.append($0) }
let before = Date()
await dictate(controller)
#expect(results[0].duration == 1.0)
#expect(results[0].date >= before)
#expect(results[0].corrections.isEmpty)
}
}
```
- [ ] **Step 2: Draai de tests, moeten falen**
Run: `swift test --filter "PromptEchoTests|DictationControllerHooksTests"`
Expected: compilatiefouten (`cannot find 'PromptEcho'`, `extra argument 'corrector'`).
- [ ] **Step 3: Implementatie**
`Sources/SstCore/PromptEcho.swift`:
```swift
import Foundation
/// Whisper sometimes answers a vocabulary prompt with the prompt itself when there is little speech.
public enum PromptEcho {
static func normalize(_ text: String) -> String {
String(text.lowercased().map { $0.isLetter || $0.isNumber ? $0 : " " })
.split(separator: " ")
.joined(separator: " ")
}
/// True when `text` is nothing but the first k ≥ 2 terms of the prompt.
/// A single term on its own is valid speech ("Anthropic").
public static func isEcho(_ text: String, vocabulary: [String]) -> Bool {
guard vocabulary.count >= 2 else { return false }
let normalized = normalize(text)
guard !normalized.isEmpty else { return false }
return (2...vocabulary.count).contains { k in
normalize(vocabulary.prefix(k).joined(separator: " ")) == normalized
}
}
}
```
Vervang `Sources/SstCore/DictationController.swift` volledig door:
```swift
import Foundation
@MainActor
public final class DictationController {
public private(set) var state: DictationState = .idle {
didSet { onStateChange?(state) }
}
public var onStateChange: ((DictationState) -> Void)?
public var onError: ((String) -> Void)?
/// Called with every finished transcription, just before it is inserted.
public var onResult: ((DictationResult) -> Void)?
private let recorder: AudioRecording
private let transcriber: Transcriber
private let cleaner: Cleaner
private let inserter: TextInserting
private let minimumSamples: Int
private let silenceThreshold: Float
private let corrector: TextCorrecting?
private let vocabularyProvider: () -> [String]
private let biasMinimumRMS: Float
/// - Parameter minimumSamples: 16 kHz samples; default is 0.3 s.
/// - Parameter silenceThreshold: RMS below which a recording counts as silence and is skipped.
/// - Parameter corrector: optional dictionary correction applied to the transcript.
/// - Parameter vocabularyProvider: words to bias the transcriber towards.
/// - Parameter biasMinimumRMS: below this RMS no vocabulary is passed (guards against invented text on quiet audio).
public init(
recorder: AudioRecording,
transcriber: Transcriber,
cleaner: Cleaner,
inserter: TextInserting,
minimumSamples: Int = 4_800,
silenceThreshold: Float = 0.005,
corrector: TextCorrecting? = nil,
vocabularyProvider: @escaping () -> [String] = { [] },
biasMinimumRMS: Float = 0.015
) {
self.recorder = recorder
self.transcriber = transcriber
self.cleaner = cleaner
self.inserter = inserter
self.minimumSamples = minimumSamples
self.silenceThreshold = silenceThreshold
self.corrector = corrector
self.vocabularyProvider = vocabularyProvider
self.biasMinimumRMS = biasMinimumRMS
}
public func handle(_ event: HotkeyEvent) async {
switch (event, state) {
case (.pressed, .idle):
do {
try recorder.start()
state = .recording
} catch {
onError?(error.localizedDescription)
}
case (.released, .recording):
let samples = recorder.stop()
guard samples.count >= minimumSamples else {
state = .idle
return
}
let energy = samples.reduce(0) { $0 + $1 * $1 }
let rms = (energy / Float(samples.count)).squareRoot()
guard rms >= silenceThreshold else {
state = .idle
return
}
await process(samples, rms: rms)
default:
break
}
}
private func process(_ samples: [Float], rms: Float) async {
defer { state = .idle }
do {
state = .transcribing
let vocabulary = rms >= biasMinimumRMS ? vocabularyProvider() : []
let raw = try await transcriber.transcribe(samples, vocabulary: vocabulary)
let text = raw.trimmingCharacters(in: .whitespacesAndNewlines)
guard !text.isEmpty, !PromptEcho.isEcho(text, vocabulary: vocabulary) else { return }
let corrected = corrector?.correct(text) ?? CorrectionResult(text: text, corrections: [])
let cleaned = try await cleaner.clean(corrected.text, context: CleaningContext())
onResult?(DictationResult(
date: Date(),
duration: Double(samples.count) / 16_000,
rawText: text,
finalText: cleaned,
// Offsets only stay valid if the cleaner left the corrected text alone.
corrections: cleaned == corrected.text ? corrected.corrections : []
))
state = .inserting
try await inserter.insert(cleaned)
} catch {
onError?(error.localizedDescription)
}
}
}
```
- [ ] **Step 4: Draai alle tests**
Run: `swift test`
Expected: alle tests PASS (de 13 + 2 + 6 + 8 nieuwe = 29).
- [ ] **Step 5: Commit**
```bash
git add Sources/SstCore Tests/SstCoreTests
git commit -m "feat: controller supports vocabulary bias, prompt-echo guard, corrector and result hook" -m "Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"
```
---
### Task 3: Instelbare sneltoets — `HotkeyBinding`, `ModifierKeyTracker`, `HotkeyMonitor`
**Files:**
- Create: `Sources/SstCore/HotkeyBinding.swift`, `Sources/SstCore/ModifierKeyTracker.swift`
- Replace: `Sources/SstCore/HotkeyMonitor.swift`
- Test: `Tests/SstCoreTests/HotkeyBindingTests.swift`, `Tests/SstCoreTests/ModifierKeyTrackerTests.swift`
**Interfaces:**
- Consumes: `HotkeyEvent` uit Models.swift.
- Produces:
- `struct HotkeyBinding: Hashable, Sendable, Identifiable` met `id: String`, `displayName: String`, `keyCode: Int`, `flagRawValue: UInt64`, `flag: CGEventFlags`; statics `.fn`, `.rightOption`, `.rightCommand`, `.rightControl`, `.leftControl`, `allowed: [HotkeyBinding]`, `binding(id:)`, `binding(keyCode:)`
- `struct ModifierKeyTracker { init(); mutating func process(isDown: Bool) -> HotkeyEvent? }`
- `HotkeyMonitor.binding: HotkeyBinding` (standaard `.fn`; wisselen herstelt de tracker)
- [ ] **Step 1: Schrijf de falende tests**
`Tests/SstCoreTests/HotkeyBindingTests.swift`:
```swift
import Testing
@testable import SstCore
struct HotkeyBindingTests {
@Test func allowedBindingsMatchTheSpec() {
#expect(HotkeyBinding.allowed.map(\.id) == ["fn", "rightOption", "rightCommand", "rightControl", "leftControl"])
#expect(HotkeyBinding.allowed.map(\.keyCode) == [63, 61, 54, 62, 59])
}
@Test func idsAndKeyCodesAreUnique() {
#expect(Set(HotkeyBinding.allowed.map(\.id)).count == HotkeyBinding.allowed.count)
#expect(Set(HotkeyBinding.allowed.map(\.keyCode)).count == HotkeyBinding.allowed.count)
}
@Test func lookupByKeyCodeAndId() {
#expect(HotkeyBinding.binding(keyCode: 61) == .rightOption)
#expect(HotkeyBinding.binding(id: "leftControl") == .leftControl)
}
@Test func unsupportedKeysAreRejected() {
#expect(HotkeyBinding.binding(keyCode: 56) == nil) // left shift
#expect(HotkeyBinding.binding(keyCode: 57) == nil) // caps lock
#expect(HotkeyBinding.binding(id: "nope") == nil)
}
@Test func everyBindingHasAFlagAndName() {
for binding in HotkeyBinding.allowed {
#expect(binding.flagRawValue != 0)
#expect(!binding.displayName.isEmpty)
}
}
}
```
`Tests/SstCoreTests/ModifierKeyTrackerTests.swift`:
```swift
import Testing
@testable import SstCore
struct ModifierKeyTrackerTests {
@Test func emitsPressedThenReleasedOnlyOnTransitions() {
var tracker = ModifierKeyTracker()
#expect(tracker.process(isDown: false) == nil)
#expect(tracker.process(isDown: true) == .pressed)
#expect(tracker.process(isDown: true) == nil)
#expect(tracker.process(isDown: false) == .released)
#expect(tracker.process(isDown: false) == nil)
}
@Test func freshTrackerStartsReleased() {
var tracker = ModifierKeyTracker()
#expect(tracker.process(isDown: true) == .pressed)
}
}
```
- [ ] **Step 2: Draai de tests, moeten falen**
Run: `swift test --filter "HotkeyBindingTests|ModifierKeyTrackerTests"`
Expected: compilatiefout `cannot find 'HotkeyBinding' in scope`.
- [ ] **Step 3: Implementatie**
`Sources/SstCore/ModifierKeyTracker.swift`:
```swift
public struct ModifierKeyTracker {
private var isDown = false
public init() {}
/// Feed the current state of the bound modifier flag; returns an event only on a transition.
public mutating func process(isDown: Bool) -> HotkeyEvent? {
guard isDown != self.isDown else { return nil }
self.isDown = isDown
return isDown ? .pressed : .released
}
}
```
`Sources/SstCore/HotkeyBinding.swift`:
```swift
import CoreGraphics
/// A modifier key that can be held to dictate. Left/right keys share a flag, so the key code decides.
public struct HotkeyBinding: Hashable, Sendable, Identifiable {
public let id: String
public let displayName: String
public let keyCode: Int
public let flagRawValue: UInt64
public var flag: CGEventFlags { CGEventFlags(rawValue: flagRawValue) }
public static let fn = HotkeyBinding(
id: "fn", displayName: "Fn", keyCode: 63, flagRawValue: CGEventFlags.maskSecondaryFn.rawValue)
public static let rightOption = HotkeyBinding(
id: "rightOption", displayName: "Rechter Option", keyCode: 61, flagRawValue: CGEventFlags.maskAlternate.rawValue)
public static let rightCommand = HotkeyBinding(
id: "rightCommand", displayName: "Rechter Command", keyCode: 54, flagRawValue: CGEventFlags.maskCommand.rawValue)
public static let rightControl = HotkeyBinding(
id: "rightControl", displayName: "Rechter Control", keyCode: 62, flagRawValue: CGEventFlags.maskControl.rawValue)
public static let leftControl = HotkeyBinding(
id: "leftControl", displayName: "Linker Control", keyCode: 59, flagRawValue: CGEventFlags.maskControl.rawValue)
public static let allowed: [HotkeyBinding] = [.fn, .rightOption, .rightCommand, .rightControl, .leftControl]
public static func binding(id: String) -> HotkeyBinding? { allowed.first { $0.id == id } }
public static func binding(keyCode: Int) -> HotkeyBinding? { allowed.first { $0.keyCode == keyCode } }
}
```
Vervang `Sources/SstCore/HotkeyMonitor.swift` volledig door:
```swift
import ApplicationServices
import CoreGraphics
public final class HotkeyMonitor {
public enum HotkeyError: Error, LocalizedError {
case tapUnavailable
public var errorDescription: String? {
"Kan de sneltoets niet volgen: geef Input Monitoring-rechten in Systeeminstellingen en probeer opnieuw (soms is een herstart van de app nodig)."
}
}
public var onEvent: ((HotkeyEvent) -> Void)?
/// The key that has to be held. Changing it resets the press tracking.
public var binding: HotkeyBinding = .fn {
didSet { tracker = ModifierKeyTracker() }
}
private var tracker = ModifierKeyTracker()
private var tap: CFMachPort?
private var source: CFRunLoopSource?
public init() {}
public func start() throws {
guard tap == nil else { return }
let mask = CGEventMask(1 << CGEventType.flagsChanged.rawValue)
let callback: CGEventTapCallBack = { _, type, event, refcon in
if let refcon {
let monitor = Unmanaged<HotkeyMonitor>.fromOpaque(refcon).takeUnretainedValue()
monitor.handle(type: type, event: event)
}
return Unmanaged.passUnretained(event)
}
guard let tap = CGEvent.tapCreate(
tap: .cgSessionEventTap,
place: .headInsertEventTap,
options: .listenOnly,
eventsOfInterest: mask,
callback: callback,
userInfo: Unmanaged.passUnretained(self).toOpaque()
) else { throw HotkeyError.tapUnavailable }
let source = CFMachPortCreateRunLoopSource(kCFAllocatorDefault, tap, 0)
CFRunLoopAddSource(CFRunLoopGetMain(), source, .commonModes)
CGEvent.tapEnable(tap: tap, enable: true)
self.tap = tap
self.source = source
}
public func stop() {
if let source { CFRunLoopRemoveSource(CFRunLoopGetMain(), source, .commonModes) }
if let tap { CGEvent.tapEnable(tap: tap, enable: false) }
tap = nil
source = nil
}
private func handle(type: CGEventType, event: CGEvent) {
switch type {
case .tapDisabledByTimeout, .tapDisabledByUserInput:
if let tap { CGEvent.tapEnable(tap: tap, enable: true) }
case .flagsChanged:
// Flags are shared between left and right keys and with arrows/F-keys: only react to the bound key itself.
guard event.getIntegerValueField(.keyboardEventKeycode) == Int64(binding.keyCode) else { return }
let isDown = event.flags.contains(binding.flag)
if let hotkeyEvent = tracker.process(isDown: isDown) {
onEvent?(hotkeyEvent)
}
default:
break
}
}
}
```
> `FnKeyTracker` en zijn test blijven ongewijzigd bestaan; `HotkeyMonitor` gebruikt hem niet meer.
- [ ] **Step 4: Draai alle tests en bouw**
Run: `swift test && swift build`
Expected: alle tests PASS; build slaagt (de app en `SstProbe` gebruiken `HotkeyMonitor()` zonder argumenten en blijven dus compileren).
- [ ] **Step 5: Handmatige controle (met de gebruiker, na Task 5)**
Wordt gebundeld in Task 5, stap 6.
- [ ] **Step 6: Commit**
```bash
git add Sources/SstCore Tests/SstCoreTests
git commit -m "feat: configurable hotkey binding and ModifierKeyTracker" -m "Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"
```
---
### Task 4: Niveau-uitvoer — `AudioLevel` en `AudioRecorder.onLevel`
**Files:**
- Create: `Sources/SstCore/AudioLevel.swift`
- Modify: `Sources/SstCore/AudioRecorder.swift`
- Modify: `Sources/SstProbe/RecordProbe.swift`
- Test: `Tests/SstCoreTests/AudioLevelTests.swift`
**Interfaces:**
- Produces:
- `AudioLevel.rmsDbfs(_ samples: [Float]) -> Float` en `rmsDbfs(_ samples: UnsafeBufferPointer<Float>) -> Float` (−120 voor stilte of leeg)
- `AudioLevel.vu(fromDbfs:referenceDbfs:) -> Float` (standaard referentie −20)
- `AudioRecorder.onLevel: ((Float) -> Void)?` — RMS in dBFS per aangeleverde buffer, aangeroepen op de audio-thread; vóór `start()` instellen.
- [ ] **Step 1: Schrijf de falende test**
`Tests/SstCoreTests/AudioLevelTests.swift`:
```swift
import Testing
@testable import SstCore
struct AudioLevelTests {
@Test func constantTenthIsMinusTwentyDbfs() {
let level = AudioLevel.rmsDbfs([Float](repeating: 0.1, count: 1_000))
#expect(abs(level - (-20)) < 0.01)
}
@Test func fullScaleIsZeroDbfs() {
let level = AudioLevel.rmsDbfs([Float](repeating: 1.0, count: 100))
#expect(abs(level) < 0.01)
}
@Test func negativeSamplesCountLikePositiveOnes() {
let level = AudioLevel.rmsDbfs([Float](repeating: -0.1, count: 1_000))
#expect(abs(level - (-20)) < 0.01)
}
@Test func emptyAndSilentBuffersHitTheFloor() {
#expect(AudioLevel.rmsDbfs([]) == AudioLevel.silenceFloorDbfs)
#expect(AudioLevel.rmsDbfs([Float](repeating: 0, count: 64)) == AudioLevel.silenceFloorDbfs)
}
@Test func vuIsRelativeToTheReference() {
#expect(AudioLevel.vu(fromDbfs: -20) == 0)
#expect(AudioLevel.vu(fromDbfs: -40) == -20)
#expect(AudioLevel.vu(fromDbfs: -17) == 3)
#expect(AudioLevel.vu(fromDbfs: -30, referenceDbfs: -18) == -12)
}
}
```
- [ ] **Step 2: Draai de test, moet falen**
Run: `swift test --filter AudioLevelTests`
Expected: compilatiefout `cannot find 'AudioLevel' in scope`.
- [ ] **Step 3: Implementatie**
`Sources/SstCore/AudioLevel.swift`:
```swift
import Foundation
public enum AudioLevel {
/// Reported for empty or silent buffers (log of zero would be -infinity).
public static let silenceFloorDbfs: Float = -120
public static func rmsDbfs(_ samples: [Float]) -> Float {
samples.withUnsafeBufferPointer { rmsDbfs($0) }
}
public static func rmsDbfs(_ samples: UnsafeBufferPointer<Float>) -> Float {
guard !samples.isEmpty else { return silenceFloorDbfs }
var sum: Float = 0
for sample in samples { sum += sample * sample }
let rms = (sum / Float(samples.count)).squareRoot()
guard rms > 0 else { return silenceFloorDbfs }
return max(silenceFloorDbfs, 20 * log10(rms))
}
/// VU reading for a dBFS level, where `referenceDbfs` is 0 VU.
public static func vu(fromDbfs dbfs: Float, referenceDbfs: Float = -20) -> Float {
dbfs - referenceDbfs
}
}
```
In `Sources/SstCore/AudioRecorder.swift`:
1. Voeg onder `private var buffer: [Float] = []` toe:
```swift
/// RMS level (dBFS) of every delivered input buffer. Called on the audio thread; set before `start()`.
public var onLevel: ((Float) -> Void)?
```
2. Vervang in de tap-closure de regel `guard let self else { return }` door:
```swift
guard let self else { return }
if let onLevel = self.onLevel, let channel = pcm.floatChannelData?[0] {
onLevel(AudioLevel.rmsDbfs(UnsafeBufferPointer(start: channel, count: Int(pcm.frameLength))))
}
```
3. Vervang `bufferSize: 4096` door `bufferSize: 1024` (macOS mag een grotere buffer leveren; de meter werkt met elke grootte).
`Sources/SstProbe/RecordProbe.swift` — vervang volledig door:
```swift
import AVFoundation
import SstCore
/// Records `seconds` of microphone audio; returns nil if access is denied or recording fails.
func recordSamples(seconds: Double, onLevel: ((Float) -> Void)? = nil) async -> [Float]? {
guard await AVCaptureDevice.requestAccess(for: .audio) else {
print("Microfoon-toegang geweigerd (Systeeminstellingen → Privacy → Microfoon).")
return nil
}
let recorder = AudioRecorder()
recorder.onLevel = onLevel
do { try recorder.start() } catch {
print("start mislukt:", error.localizedDescription)
return nil
}
print("Neem \(Int(seconds)) s op — praat nu")
try? await Task.sleep(for: .seconds(seconds))
return recorder.stop()
}
func runRecordProbe() async {
final class Levels: @unchecked Sendable {
let lock = NSLock()
var values: [Float] = []
func add(_ v: Float) { lock.withLock { values.append(v) } }
}
let levels = Levels()
guard let samples = await recordSamples(seconds: 3, onLevel: { levels.add($0) }) else { return }
let peak = samples.map { abs($0) }.max() ?? 0
print("samples: \(samples.count) (verwacht ≈ 48000), piek: \(peak)")
let values = levels.lock.withLock { levels.values }
if let lo = values.min(), let hi = values.max() {
print(String(format: "niveau-meldingen: %d (≈ %.0f per s), bereik %.1f … %.1f dBFS", values.count, Double(values.count) / 3, lo, hi))
} else {
print("geen niveau-meldingen ontvangen")
}
}
```
- [ ] **Step 4: Draai alle tests en bouw**
Run: `swift test && swift build`
Expected: alle tests PASS; build slaagt.
- [ ] **Step 5: Handmatige controle (met de gebruiker, in Task 5, stap 6)**
Hier alleen: de code compileert.
- [ ] **Step 6: Commit**
```bash
git add Sources Tests
git commit -m "feat: AudioLevel helper and AudioRecorder.onLevel" -m "Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"
```
---
### Task 5: WhisperKit-voorrang — woordenlijst als `promptTokens`
**Files:**
- Modify: `Package.swift` (testtarget `SstWhisperTests`)
- Modify: `Sources/SstWhisper/WhisperKitTranscriber.swift`
- Modify: `Sources/SstProbe/TranscribeProbe.swift`
- Test: `Tests/SstWhisperTests/PromptTokensTests.swift`
**Interfaces:**
- Consumes: Task 1 (`Transcriber.transcribe(_:vocabulary:)`).
- Produces: `WhisperKitTranscriber.transcribe(_:vocabulary:)`; intern `static func promptTokens(vocabulary:maxTokens:specialTokenBegin:encode:) -> [Int]?`.
- [ ] **Step 1: Testtarget toevoegen**
In `Package.swift` voeg onder de bestaande `.testTarget(name: "SstCoreTests", …)` toe:
```swift
.testTarget(name: "SstWhisperTests", dependencies: ["SstWhisper"]),
```
- [ ] **Step 2: Schrijf de falende test**
`Tests/SstWhisperTests/PromptTokensTests.swift`:
```swift
import Testing
@testable import SstWhisper
struct PromptTokensTests {
/// Fake tokenizer: one token (id 7) per word, plus one special token (id 2000) that must be filtered out.
let encode: (String) -> [Int] = { text in
text.split(whereSeparator: { !$0.isLetter }).map { _ in 7 } + [2000]
}
@Test func allTermsFitAndSpecialTokensAreFiltered() {
let tokens = WhisperKitTranscriber.promptTokens(
vocabulary: ["Anthropic", "Vercel", "Supabase"], maxTokens: 64, specialTokenBegin: 1000, encode: encode)
#expect(tokens == [7, 7, 7])
}
@Test func termsThatDoNotFitAreDroppedFromTheEnd() {
let vocab = ["a", "b", "c", "d", "e", "f", "g", "h", "i", "j"]
let tokens = WhisperKitTranscriber.promptTokens(
vocabulary: vocab, maxTokens: 4, specialTokenBegin: 1000, encode: encode)
#expect(tokens?.count == 4)
}
@Test func emptyVocabularyGivesNoPrompt() {
#expect(WhisperKitTranscriber.promptTokens(
vocabulary: [], maxTokens: 64, specialTokenBegin: 1000, encode: encode) == nil)
}
@Test func firstTermAloneTooLongGivesNoPrompt() {
#expect(WhisperKitTranscriber.promptTokens(
vocabulary: ["one two three"], maxTokens: 2, specialTokenBegin: 1000, encode: encode) == nil)
}
}
```
- [ ] **Step 3: Draai de test, moet falen**
Run: `swift test --filter PromptTokensTests`
Expected: compilatiefout `type 'WhisperKitTranscriber' has no member 'promptTokens'`.
- [ ] **Step 4: Implementatie**
In `Sources/SstWhisper/WhisperKitTranscriber.swift` vervang de hele methode `public func transcribe(_ samples: [Float]) async throws -> String { … }` (onderaan het bestand) door:
```swift
public func transcribe(_ samples: [Float]) async throws -> String {
try await transcribe(samples, vocabulary: [])
}
/// Terms are added one by one while the encoded prompt stays within `maxTokens`.
/// Returns nil when nothing fits or the list is empty.
static func promptTokens(
vocabulary: [String],
maxTokens: Int = 64,
specialTokenBegin: Int,
encode: (String) -> [Int]
) -> [Int]? {
var kept: [String] = []
var tokens: [Int] = []
for term in vocabulary {
let candidate = kept + [term]
let encoded = encode(" " + candidate.joined(separator: ", ")).filter { $0 < specialTokenBegin }
if encoded.count > maxTokens { break }
kept = candidate
tokens = encoded
}
return tokens.isEmpty ? nil : tokens
}
public func transcribe(_ samples: [Float], vocabulary: [String]) async throws -> String {
guard let pipeline else { throw TranscriberError.notLoaded }
// language nil + detectLanguage: Whisper picks NL or EN per utterance.
var options = DecodingOptions(
task: .transcribe,
language: nil,
detectLanguage: true,
skipSpecialTokens: true,
withoutTimestamps: true
)
if !vocabulary.isEmpty, let tokenizer = pipeline.tokenizer {
options.promptTokens = Self.promptTokens(
vocabulary: vocabulary,
specialTokenBegin: tokenizer.specialTokens.specialTokenBegin,
encode: { tokenizer.encode(text: $0) }
)
}
let results = try await pipeline.transcribe(audioArray: samples, decodeOptions: options)
return results
.map(\.text)
.joined(separator: " ")
.trimmingCharacters(in: .whitespacesAndNewlines)
}
```
> Wijkt de WhisperKit-API licht af (bijvoorbeeld de plaats van `specialTokens`), pas dan alleen dit bestand aan en behoud de signatuur van `promptTokens` zodat de test geldig blijft.
`Sources/SstProbe/TranscribeProbe.swift` — vervang regel `let text = try await transcriber.transcribe(samples)` door:
```swift
let vocabulary = (ProcessInfo.processInfo.environment["SST_VOCAB"] ?? "")
.split(separator: ",").map { $0.trimmingCharacters(in: .whitespaces) }.filter { !$0.isEmpty }
if !vocabulary.isEmpty { print("voorrang: \(vocabulary.joined(separator: ", "))") }
let text = try await transcriber.transcribe(samples, vocabulary: vocabulary)
```
- [ ] **Step 5: Draai alle tests en bouw**
Run: `swift test && swift build`
Expected: alle tests PASS (incl. 4 nieuwe in `SstWhisperTests`); build slaagt.
- [ ] **Step 6: Handmatige controle samen met de gebruiker (hardware)**
Doe in deze volgorde en noteer de uitkomsten in het commit-bericht van stap 7:
1. `swift run SstProbe record` → er verschijnt een regel `niveau-meldingen: N (≈ X per s), bereik … dBFS`. Verwacht: meer dan 5 meldingen per seconde en een bereik dat bij praten duidelijk hoger ligt dan bij stilte. Noteer X (macOS levert soms ~10 per s in plaats van ~47).
2. `swift run SstProbe transcribe` — zeg een zin, noteer de tijd **zonder** voorrang.
3. `SST_VOCAB="Anthropic,Vercel,Supabase,Claude Code" swift run SstProbe transcribe` — zeg dezelfde zin met een of twee van die woorden erin ("Ik werk met Supabase en Claude Code"); noteer de tijd en of de spelling beter is. Verwacht: tijd onder 1,5 s voor 8 s spraak. Is het trager, noteer hoeveel; dat is geen blokkade, wel informatie voor de spec.
4. Zeg niets (8 s stilte) met `SST_VOCAB` gezet: het resultaat mag geen woorden uit de lijst bevatten.
5. Hotkey-controle: `swift run SstProbe hotkey` toont nog steeds `pressed`/`released` voor Fn.
- [ ] **Step 7: Commit**
```bash
git add Package.swift Sources Tests
git commit -m "feat: WhisperKit transcriber biases decoding with vocabulary prompt tokens" -m "Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"
```
---
## Self-review
- **Spec-dekking:** raakpunt 1 (woordenlijst → `promptTokens`, Task 1+5), 2 (niveau, Task 4), 3 (corrector + `onResult`, Task 1+2), 4 (instelbare toets, Task 3). Voorrang-regels: RMS-grens 0,015 (Task 2 test `quietRecordingSkipsVocabularyButStillTranscribes`), echo-filter (Task 2), 64 tokens (Task 5). Niet in deze fase, wel in fase 2: het opbouwen van de woordenlijst (`VocabularyBuilder`) en de correctie zelf.
- **Placeholders:** geen; de waarschuwing in Task 5 stap 4 is een gerichte terugval, geen open eind.
- **Typeconsistentie:** `AppliedCorrection(heard:wrote:rule:start:length:)`, `CorrectionResult(text:corrections:)` en `DictationResult(date:duration:rawText:finalText:corrections:)` worden in Task 2 precies zo gebruikt als in Task 1 gedefinieerd. `HotkeyMonitor.binding` gebruikt `HotkeyBinding.keyCode: Int` en `flag`.
- **Risico's:** de bufferlevering van `installTap` (Task 4, stap 6.1) en de snelheid met prompt (stap 6.3) zijn de enige onbekenden; beide worden gemeten.