42 KiB
App fase 1 — Engine-raakpunten Implementation Plan
For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (
- [ ]) syntax for tracking.
Goal: De engine krijgt vier additieve uitbreidingen (woordenlijst voor de transcriber, niveau-uitvoer, resultaat-hook met correctie, instelbare sneltoets) zonder dat bestaand gedrag verandert.
Architecture: Alle wijzigingen zijn additief met standaardwaarden die het huidige gedrag behouden. Nieuwe pure logica (PromptEcho, AudioLevel, HotkeyBinding, ModifierKeyTracker, promptTokens) staat los van hardware en wordt met unit-tests gedekt; de hardware-afhankelijke stukken (AudioRecorder, HotkeyMonitor, WhisperKit) zijn dun en worden handmatig gecontroleerd.
Tech Stack: Swift 6.4 (language mode 5), Swift Testing, AVFoundation, CoreGraphics, WhisperKit 0.18.
Spec: docs/superpowers/specs/2026-10-01-macos-app-design.md (secties "Engine-raakpunten", "Woordenboek — mechanisme 1")
Global Constraints
- Bestaande tests (13 stuks) blijven ongewijzigd groen; bestaande testbestanden worden alleen aangevuld (
Fakes.swift) of krijgen er een nieuw bestand naast. - Nieuwe parameters hebben standaardwaarden die het huidige gedrag behouden.
- Vocabulaire: maximaal 64 tokens in de prompt; prompt alleen bij opnames met RMS ≥ 0,015; stiltefilter blijft 0,005.
- Prompt-echo: resultaat dat uitsluitend uit de eerste k ≥ 2 termen van de prompt bestaat wordt verworpen; één term alleen niet.
- Niveau:
onLevellevert RMS in dBFS; 0 VU = −20 dBFS. - Toegestane sneltoetsen (exact): Fn (keycode 63), Rechter Option (61), Rechter Command (54), Rechter Control (62), Linker Control (59).
- Elke commit eindigt met de trailer
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>(tweede-m).
Bestandsstructuur
Sources/SstCore/Models.swift (wijzig) Transcriber + AppliedCorrection, CorrectionResult, TextCorrecting, DictationResult
Sources/SstCore/PromptEcho.swift (nieuw)
Sources/SstCore/DictationController.swift(vervang) corrector, onResult, vocabularyProvider
Sources/SstCore/HotkeyBinding.swift (nieuw)
Sources/SstCore/ModifierKeyTracker.swift (nieuw)
Sources/SstCore/HotkeyMonitor.swift (vervang) binding
Sources/SstCore/AudioLevel.swift (nieuw)
Sources/SstCore/AudioRecorder.swift (wijzig) onLevel
Sources/SstWhisper/WhisperKitTranscriber.swift (wijzig) vocabulary → promptTokens
Sources/SstProbe/RecordProbe.swift, TranscribeProbe.swift (wijzig) handmatige controle
Package.swift (wijzig) SstWhisperTests
Tests/SstCoreTests/Fakes.swift (aanvullen)
Tests/SstCoreTests/TranscriberVocabularyTests.swift, PromptEchoTests.swift,
DictationControllerHooksTests.swift, HotkeyBindingTests.swift,
ModifierKeyTrackerTests.swift, AudioLevelTests.swift (nieuw)
Tests/SstWhisperTests/PromptTokensTests.swift (nieuw)
Task 1: Gedeelde typen en woordenlijst op Transcriber
Files:
- Modify:
Sources/SstCore/Models.swift - Test:
Tests/SstCoreTests/TranscriberVocabularyTests.swift
Interfaces:
-
Produces (gebruikt door alle latere fasen):
Transcriber.transcribe(_ samples: [Float], vocabulary: [String]) async throws -> String(protocol-eis, met standaardimplementatie dietranscribe(_:)aanroept)struct AppliedCorrection: Equatable, Codable, Sendable { heard, wrote, rule: String; start, length: Int }met publieke initinit(heard:wrote:rule:start:length:)struct CorrectionResult: Equatable, Sendable { text: String; corrections: [AppliedCorrection] }met publieke initinit(text:corrections:)protocol TextCorrecting { func correct(_ text: String) -> CorrectionResult }struct DictationResult: Equatable, Sendable { date: Date; duration: TimeInterval; rawText: String; finalText: String; corrections: [AppliedCorrection] }met publieke init in die volgorde
-
Step 1: Schrijf de falende test
Tests/SstCoreTests/TranscriberVocabularyTests.swift:
import Foundation
import Testing
@testable import SstCore
private struct PlainTranscriber: Transcriber {
func transcribe(_ samples: [Float]) async throws -> String { "plain \(samples.count)" }
}
struct TranscriberVocabularyTests {
@Test func vocabularyVariantFallsBackToPlainTranscribe() async throws {
let result = try await PlainTranscriber().transcribe([0, 0, 0], vocabulary: ["Anthropic"])
#expect(result == "plain 3")
}
@Test func appliedCorrectionRoundTripsThroughJSON() throws {
let original = AppliedCorrection(heard: "cloud code", wrote: "Claude Code", rule: "cloud code", start: 12, length: 11)
let data = try JSONEncoder().encode(original)
let decoded = try JSONDecoder().decode(AppliedCorrection.self, from: data)
#expect(decoded == original)
}
}
- Step 2: Draai de test, moet falen
Run: swift test --filter TranscriberVocabularyTests
Expected: compilatiefout extra argument 'vocabulary' in call / cannot find 'AppliedCorrection' in scope.
- Step 3: Implementatie
In Sources/SstCore/Models.swift vervang het blok
public protocol Transcriber {
func transcribe(_ samples: [Float]) async throws -> String
}
door:
public protocol Transcriber {
func transcribe(_ samples: [Float]) async throws -> String
/// Biases decoding towards `vocabulary`. Engines without bias support ignore the list.
func transcribe(_ samples: [Float], vocabulary: [String]) async throws -> String
}
public extension Transcriber {
func transcribe(_ samples: [Float], vocabulary: [String]) async throws -> String {
try await transcribe(samples)
}
}
/// One replacement made by the dictionary correction pass.
/// `start`/`length` are UTF-16 offsets into the corrected text.
public struct AppliedCorrection: Equatable, Codable, Sendable {
public var heard: String
public var wrote: String
public var rule: String
public var start: Int
public var length: Int
public init(heard: String, wrote: String, rule: String, start: Int, length: Int) {
self.heard = heard
self.wrote = wrote
self.rule = rule
self.start = start
self.length = length
}
}
public struct CorrectionResult: Equatable, Sendable {
public var text: String
public var corrections: [AppliedCorrection]
public init(text: String, corrections: [AppliedCorrection]) {
self.text = text
self.corrections = corrections
}
}
public protocol TextCorrecting {
func correct(_ text: String) -> CorrectionResult
}
/// A finished dictation, emitted just before the text is inserted.
public struct DictationResult: Equatable, Sendable {
public var date: Date
public var duration: TimeInterval
public var rawText: String
public var finalText: String
public var corrections: [AppliedCorrection]
public init(date: Date, duration: TimeInterval, rawText: String, finalText: String, corrections: [AppliedCorrection]) {
self.date = date
self.duration = duration
self.rawText = rawText
self.finalText = finalText
self.corrections = corrections
}
}
- Step 4: Draai alle tests
Run: swift test
Expected: alle tests PASS (13 bestaande + 2 nieuwe).
- Step 5: Commit
git add Sources/SstCore/Models.swift Tests/SstCoreTests/TranscriberVocabularyTests.swift
git commit -m "feat: add vocabulary-aware Transcriber and shared correction/result types" -m "Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"
Task 2: DictationController — voorrang, echo-filter, correctie en resultaat-hook
Files:
- Create:
Sources/SstCore/PromptEcho.swift - Replace:
Sources/SstCore/DictationController.swift - Modify:
Tests/SstCoreTests/Fakes.swift(aanvullen onderaan) - Test:
Tests/SstCoreTests/PromptEchoTests.swift,Tests/SstCoreTests/DictationControllerHooksTests.swift
Interfaces:
-
Consumes: Task 1 (
Transcriber.transcribe(_:vocabulary:),TextCorrecting,CorrectionResult,DictationResult). -
Produces:
PromptEcho.isEcho(_ text: String, vocabulary: [String]) -> BoolDictationController.init(recorder:transcriber:cleaner:inserter:minimumSamples:silenceThreshold:corrector:vocabularyProvider:biasMinimumRMS:)— de nieuwe parameters zijncorrector: TextCorrecting? = nil,vocabularyProvider: @escaping () -> [String] = { [] },biasMinimumRMS: Float = 0.015DictationController.onResult: ((DictationResult) -> Void)?
-
Step 1: Schrijf de falende tests
Tests/SstCoreTests/PromptEchoTests.swift:
import Testing
@testable import SstCore
struct PromptEchoTests {
let vocabulary = ["Anthropic", "Vercel", "Supabase"]
@Test func firstTwoTermsAreAnEcho() {
#expect(PromptEcho.isEcho("Anthropic, Vercel.", vocabulary: vocabulary))
}
@Test func wholeListIsAnEcho() {
#expect(PromptEcho.isEcho("anthropic vercel supabase", vocabulary: vocabulary))
}
@Test func singleTermIsRealSpeech() {
#expect(!PromptEcho.isEcho("Anthropic", vocabulary: vocabulary))
}
@Test func termsOutOfOrderAreNotAnEcho() {
#expect(!PromptEcho.isEcho("Vercel, Anthropic", vocabulary: vocabulary))
}
@Test func sentenceContainingTermsIsNotAnEcho() {
#expect(!PromptEcho.isEcho("Ik gebruik Anthropic en Vercel", vocabulary: vocabulary))
}
@Test func tooShortVocabularyNeverEchoes() {
#expect(!PromptEcho.isEcho("Anthropic", vocabulary: ["Anthropic"]))
#expect(!PromptEcho.isEcho("", vocabulary: vocabulary))
}
}
Voeg onderaan Tests/SstCoreTests/Fakes.swift toe:
final class FakeVocabularyTranscriber: Transcriber {
var result = "hallo wereld"
private(set) var vocabularies: [[String]] = []
func transcribe(_ samples: [Float]) async throws -> String { result }
func transcribe(_ samples: [Float], vocabulary: [String]) async throws -> String {
vocabularies.append(vocabulary)
return result
}
}
struct UppercaseCorrector: TextCorrecting {
func correct(_ text: String) -> CorrectionResult {
let upper = text.uppercased()
return CorrectionResult(
text: upper,
corrections: [AppliedCorrection(heard: text, wrote: upper, rule: "upper", start: 0, length: upper.utf16.count)]
)
}
}
Tests/SstCoreTests/DictationControllerHooksTests.swift:
import Foundation
import Testing
@testable import SstCore
@MainActor
struct DictationControllerHooksTests {
let recorder = FakeRecorder()
let transcriber = FakeVocabularyTranscriber()
let inserter = FakeInserter()
static let loud = [Float](repeating: 0.1, count: 16_000)
static let quiet = [Float](repeating: 0.01, count: 16_000)
func makeController(
cleaner: Cleaner = FakeCleaner(),
corrector: TextCorrecting? = nil,
vocabulary: [String] = []
) -> DictationController {
DictationController(
recorder: recorder,
transcriber: transcriber,
cleaner: cleaner,
inserter: inserter,
corrector: corrector,
vocabularyProvider: { vocabulary }
)
}
func dictate(_ controller: DictationController) async {
await controller.handle(.pressed)
await controller.handle(.released)
}
@Test func loudRecordingPassesVocabularyToTranscriber() async {
recorder.samples = Self.loud
await dictate(makeController(vocabulary: ["Anthropic", "Vercel"]))
#expect(transcriber.vocabularies == [["Anthropic", "Vercel"]])
}
@Test func quietRecordingSkipsVocabularyButStillTranscribes() async {
recorder.samples = Self.quiet // RMS 0.01: above silence (0.005), below bias minimum (0.015)
await dictate(makeController(vocabulary: ["Anthropic", "Vercel"]))
#expect(transcriber.vocabularies == [[]])
#expect(inserter.inserted == ["[hallo wereld]"])
}
@Test func promptEchoIsDiscarded() async {
recorder.samples = Self.loud
transcriber.result = "Anthropic, Vercel."
let controller = makeController(vocabulary: ["Anthropic", "Vercel", "Supabase"])
var results: [DictationResult] = []
controller.onResult = { results.append($0) }
await dictate(controller)
#expect(inserter.inserted.isEmpty)
#expect(results.isEmpty)
#expect(controller.state == .idle)
}
@Test func singleVocabularyTermIsKept() async {
recorder.samples = Self.loud
transcriber.result = "Anthropic"
await dictate(makeController(vocabulary: ["Anthropic", "Vercel", "Supabase"]))
#expect(inserter.inserted == ["[Anthropic]"])
}
@Test func correctorRunsBeforeCleanerAndCorrectionsSurviveAPassthroughCleaner() async {
recorder.samples = Self.loud
transcriber.result = "hallo"
let controller = makeController(cleaner: PassthroughCleaner(), corrector: UppercaseCorrector())
var results: [DictationResult] = []
controller.onResult = { results.append($0) }
await dictate(controller)
#expect(inserter.inserted == ["HALLO"])
#expect(results.count == 1)
#expect(results[0].rawText == "hallo")
#expect(results[0].finalText == "HALLO")
#expect(results[0].corrections.count == 1)
}
@Test func correctionsAreDroppedWhenTheCleanerChangesTheText() async {
recorder.samples = Self.loud
transcriber.result = "hallo"
let controller = makeController(cleaner: FakeCleaner(), corrector: UppercaseCorrector())
var results: [DictationResult] = []
controller.onResult = { results.append($0) }
await dictate(controller)
#expect(results[0].finalText == "[HALLO]")
#expect(results[0].corrections.isEmpty)
}
@Test func resultIsReportedEvenWhenInsertionFails() async {
recorder.samples = Self.loud
inserter.error = TestError()
let controller = makeController()
var results: [DictationResult] = []
var errors: [String] = []
controller.onResult = { results.append($0) }
controller.onError = { errors.append($0) }
await dictate(controller)
#expect(results.count == 1)
#expect(errors.count == 1)
}
@Test func resultCarriesDurationAndDate() async {
recorder.samples = Self.loud // 16 000 samples = 1 s
let controller = makeController()
var results: [DictationResult] = []
controller.onResult = { results.append($0) }
let before = Date()
await dictate(controller)
#expect(results[0].duration == 1.0)
#expect(results[0].date >= before)
#expect(results[0].corrections.isEmpty)
}
}
- Step 2: Draai de tests, moeten falen
Run: swift test --filter "PromptEchoTests|DictationControllerHooksTests"
Expected: compilatiefouten (cannot find 'PromptEcho', extra argument 'corrector').
- Step 3: Implementatie
Sources/SstCore/PromptEcho.swift:
import Foundation
/// Whisper sometimes answers a vocabulary prompt with the prompt itself when there is little speech.
public enum PromptEcho {
static func normalize(_ text: String) -> String {
String(text.lowercased().map { $0.isLetter || $0.isNumber ? $0 : " " })
.split(separator: " ")
.joined(separator: " ")
}
/// True when `text` is nothing but the first k ≥ 2 terms of the prompt.
/// A single term on its own is valid speech ("Anthropic").
public static func isEcho(_ text: String, vocabulary: [String]) -> Bool {
guard vocabulary.count >= 2 else { return false }
let normalized = normalize(text)
guard !normalized.isEmpty else { return false }
return (2...vocabulary.count).contains { k in
normalize(vocabulary.prefix(k).joined(separator: " ")) == normalized
}
}
}
Vervang Sources/SstCore/DictationController.swift volledig door:
import Foundation
@MainActor
public final class DictationController {
public private(set) var state: DictationState = .idle {
didSet { onStateChange?(state) }
}
public var onStateChange: ((DictationState) -> Void)?
public var onError: ((String) -> Void)?
/// Called with every finished transcription, just before it is inserted.
public var onResult: ((DictationResult) -> Void)?
private let recorder: AudioRecording
private let transcriber: Transcriber
private let cleaner: Cleaner
private let inserter: TextInserting
private let minimumSamples: Int
private let silenceThreshold: Float
private let corrector: TextCorrecting?
private let vocabularyProvider: () -> [String]
private let biasMinimumRMS: Float
/// - Parameter minimumSamples: 16 kHz samples; default is 0.3 s.
/// - Parameter silenceThreshold: RMS below which a recording counts as silence and is skipped.
/// - Parameter corrector: optional dictionary correction applied to the transcript.
/// - Parameter vocabularyProvider: words to bias the transcriber towards.
/// - Parameter biasMinimumRMS: below this RMS no vocabulary is passed (guards against invented text on quiet audio).
public init(
recorder: AudioRecording,
transcriber: Transcriber,
cleaner: Cleaner,
inserter: TextInserting,
minimumSamples: Int = 4_800,
silenceThreshold: Float = 0.005,
corrector: TextCorrecting? = nil,
vocabularyProvider: @escaping () -> [String] = { [] },
biasMinimumRMS: Float = 0.015
) {
self.recorder = recorder
self.transcriber = transcriber
self.cleaner = cleaner
self.inserter = inserter
self.minimumSamples = minimumSamples
self.silenceThreshold = silenceThreshold
self.corrector = corrector
self.vocabularyProvider = vocabularyProvider
self.biasMinimumRMS = biasMinimumRMS
}
public func handle(_ event: HotkeyEvent) async {
switch (event, state) {
case (.pressed, .idle):
do {
try recorder.start()
state = .recording
} catch {
onError?(error.localizedDescription)
}
case (.released, .recording):
let samples = recorder.stop()
guard samples.count >= minimumSamples else {
state = .idle
return
}
let energy = samples.reduce(0) { $0 + $1 * $1 }
let rms = (energy / Float(samples.count)).squareRoot()
guard rms >= silenceThreshold else {
state = .idle
return
}
await process(samples, rms: rms)
default:
break
}
}
private func process(_ samples: [Float], rms: Float) async {
defer { state = .idle }
do {
state = .transcribing
let vocabulary = rms >= biasMinimumRMS ? vocabularyProvider() : []
let raw = try await transcriber.transcribe(samples, vocabulary: vocabulary)
let text = raw.trimmingCharacters(in: .whitespacesAndNewlines)
guard !text.isEmpty, !PromptEcho.isEcho(text, vocabulary: vocabulary) else { return }
let corrected = corrector?.correct(text) ?? CorrectionResult(text: text, corrections: [])
let cleaned = try await cleaner.clean(corrected.text, context: CleaningContext())
onResult?(DictationResult(
date: Date(),
duration: Double(samples.count) / 16_000,
rawText: text,
finalText: cleaned,
// Offsets only stay valid if the cleaner left the corrected text alone.
corrections: cleaned == corrected.text ? corrected.corrections : []
))
state = .inserting
try await inserter.insert(cleaned)
} catch {
onError?(error.localizedDescription)
}
}
}
- Step 4: Draai alle tests
Run: swift test
Expected: alle tests PASS (de 13 + 2 + 6 + 8 nieuwe = 29).
- Step 5: Commit
git add Sources/SstCore Tests/SstCoreTests
git commit -m "feat: controller supports vocabulary bias, prompt-echo guard, corrector and result hook" -m "Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"
Task 3: Instelbare sneltoets — HotkeyBinding, ModifierKeyTracker, HotkeyMonitor
Files:
- Create:
Sources/SstCore/HotkeyBinding.swift,Sources/SstCore/ModifierKeyTracker.swift - Replace:
Sources/SstCore/HotkeyMonitor.swift - Test:
Tests/SstCoreTests/HotkeyBindingTests.swift,Tests/SstCoreTests/ModifierKeyTrackerTests.swift
Interfaces:
-
Consumes:
HotkeyEventuit Models.swift. -
Produces:
struct HotkeyBinding: Hashable, Sendable, Identifiablemetid: String,displayName: String,keyCode: Int,flagRawValue: UInt64,flag: CGEventFlags; statics.fn,.rightOption,.rightCommand,.rightControl,.leftControl,allowed: [HotkeyBinding],binding(id:),binding(keyCode:)struct ModifierKeyTracker { init(); mutating func process(isDown: Bool) -> HotkeyEvent? }HotkeyMonitor.binding: HotkeyBinding(standaard.fn; wisselen herstelt de tracker)
-
Step 1: Schrijf de falende tests
Tests/SstCoreTests/HotkeyBindingTests.swift:
import Testing
@testable import SstCore
struct HotkeyBindingTests {
@Test func allowedBindingsMatchTheSpec() {
#expect(HotkeyBinding.allowed.map(\.id) == ["fn", "rightOption", "rightCommand", "rightControl", "leftControl"])
#expect(HotkeyBinding.allowed.map(\.keyCode) == [63, 61, 54, 62, 59])
}
@Test func idsAndKeyCodesAreUnique() {
#expect(Set(HotkeyBinding.allowed.map(\.id)).count == HotkeyBinding.allowed.count)
#expect(Set(HotkeyBinding.allowed.map(\.keyCode)).count == HotkeyBinding.allowed.count)
}
@Test func lookupByKeyCodeAndId() {
#expect(HotkeyBinding.binding(keyCode: 61) == .rightOption)
#expect(HotkeyBinding.binding(id: "leftControl") == .leftControl)
}
@Test func unsupportedKeysAreRejected() {
#expect(HotkeyBinding.binding(keyCode: 56) == nil) // left shift
#expect(HotkeyBinding.binding(keyCode: 57) == nil) // caps lock
#expect(HotkeyBinding.binding(id: "nope") == nil)
}
@Test func everyBindingHasAFlagAndName() {
for binding in HotkeyBinding.allowed {
#expect(binding.flagRawValue != 0)
#expect(!binding.displayName.isEmpty)
}
}
}
Tests/SstCoreTests/ModifierKeyTrackerTests.swift:
import Testing
@testable import SstCore
struct ModifierKeyTrackerTests {
@Test func emitsPressedThenReleasedOnlyOnTransitions() {
var tracker = ModifierKeyTracker()
#expect(tracker.process(isDown: false) == nil)
#expect(tracker.process(isDown: true) == .pressed)
#expect(tracker.process(isDown: true) == nil)
#expect(tracker.process(isDown: false) == .released)
#expect(tracker.process(isDown: false) == nil)
}
@Test func freshTrackerStartsReleased() {
var tracker = ModifierKeyTracker()
#expect(tracker.process(isDown: true) == .pressed)
}
}
- Step 2: Draai de tests, moeten falen
Run: swift test --filter "HotkeyBindingTests|ModifierKeyTrackerTests"
Expected: compilatiefout cannot find 'HotkeyBinding' in scope.
- Step 3: Implementatie
Sources/SstCore/ModifierKeyTracker.swift:
public struct ModifierKeyTracker {
private var isDown = false
public init() {}
/// Feed the current state of the bound modifier flag; returns an event only on a transition.
public mutating func process(isDown: Bool) -> HotkeyEvent? {
guard isDown != self.isDown else { return nil }
self.isDown = isDown
return isDown ? .pressed : .released
}
}
Sources/SstCore/HotkeyBinding.swift:
import CoreGraphics
/// A modifier key that can be held to dictate. Left/right keys share a flag, so the key code decides.
public struct HotkeyBinding: Hashable, Sendable, Identifiable {
public let id: String
public let displayName: String
public let keyCode: Int
public let flagRawValue: UInt64
public var flag: CGEventFlags { CGEventFlags(rawValue: flagRawValue) }
public static let fn = HotkeyBinding(
id: "fn", displayName: "Fn", keyCode: 63, flagRawValue: CGEventFlags.maskSecondaryFn.rawValue)
public static let rightOption = HotkeyBinding(
id: "rightOption", displayName: "Rechter Option", keyCode: 61, flagRawValue: CGEventFlags.maskAlternate.rawValue)
public static let rightCommand = HotkeyBinding(
id: "rightCommand", displayName: "Rechter Command", keyCode: 54, flagRawValue: CGEventFlags.maskCommand.rawValue)
public static let rightControl = HotkeyBinding(
id: "rightControl", displayName: "Rechter Control", keyCode: 62, flagRawValue: CGEventFlags.maskControl.rawValue)
public static let leftControl = HotkeyBinding(
id: "leftControl", displayName: "Linker Control", keyCode: 59, flagRawValue: CGEventFlags.maskControl.rawValue)
public static let allowed: [HotkeyBinding] = [.fn, .rightOption, .rightCommand, .rightControl, .leftControl]
public static func binding(id: String) -> HotkeyBinding? { allowed.first { $0.id == id } }
public static func binding(keyCode: Int) -> HotkeyBinding? { allowed.first { $0.keyCode == keyCode } }
}
Vervang Sources/SstCore/HotkeyMonitor.swift volledig door:
import ApplicationServices
import CoreGraphics
public final class HotkeyMonitor {
public enum HotkeyError: Error, LocalizedError {
case tapUnavailable
public var errorDescription: String? {
"Kan de sneltoets niet volgen: geef Input Monitoring-rechten in Systeeminstellingen en probeer opnieuw (soms is een herstart van de app nodig)."
}
}
public var onEvent: ((HotkeyEvent) -> Void)?
/// The key that has to be held. Changing it resets the press tracking.
public var binding: HotkeyBinding = .fn {
didSet { tracker = ModifierKeyTracker() }
}
private var tracker = ModifierKeyTracker()
private var tap: CFMachPort?
private var source: CFRunLoopSource?
public init() {}
public func start() throws {
guard tap == nil else { return }
let mask = CGEventMask(1 << CGEventType.flagsChanged.rawValue)
let callback: CGEventTapCallBack = { _, type, event, refcon in
if let refcon {
let monitor = Unmanaged<HotkeyMonitor>.fromOpaque(refcon).takeUnretainedValue()
monitor.handle(type: type, event: event)
}
return Unmanaged.passUnretained(event)
}
guard let tap = CGEvent.tapCreate(
tap: .cgSessionEventTap,
place: .headInsertEventTap,
options: .listenOnly,
eventsOfInterest: mask,
callback: callback,
userInfo: Unmanaged.passUnretained(self).toOpaque()
) else { throw HotkeyError.tapUnavailable }
let source = CFMachPortCreateRunLoopSource(kCFAllocatorDefault, tap, 0)
CFRunLoopAddSource(CFRunLoopGetMain(), source, .commonModes)
CGEvent.tapEnable(tap: tap, enable: true)
self.tap = tap
self.source = source
}
public func stop() {
if let source { CFRunLoopRemoveSource(CFRunLoopGetMain(), source, .commonModes) }
if let tap { CGEvent.tapEnable(tap: tap, enable: false) }
tap = nil
source = nil
}
private func handle(type: CGEventType, event: CGEvent) {
switch type {
case .tapDisabledByTimeout, .tapDisabledByUserInput:
if let tap { CGEvent.tapEnable(tap: tap, enable: true) }
case .flagsChanged:
// Flags are shared between left and right keys and with arrows/F-keys: only react to the bound key itself.
guard event.getIntegerValueField(.keyboardEventKeycode) == Int64(binding.keyCode) else { return }
let isDown = event.flags.contains(binding.flag)
if let hotkeyEvent = tracker.process(isDown: isDown) {
onEvent?(hotkeyEvent)
}
default:
break
}
}
}
FnKeyTrackeren zijn test blijven ongewijzigd bestaan;HotkeyMonitorgebruikt hem niet meer.
- Step 4: Draai alle tests en bouw
Run: swift test && swift build
Expected: alle tests PASS; build slaagt (de app en SstProbe gebruiken HotkeyMonitor() zonder argumenten en blijven dus compileren).
- Step 5: Handmatige controle (met de gebruiker, na Task 5)
Wordt gebundeld in Task 5, stap 6.
- Step 6: Commit
git add Sources/SstCore Tests/SstCoreTests
git commit -m "feat: configurable hotkey binding and ModifierKeyTracker" -m "Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"
Task 4: Niveau-uitvoer — AudioLevel en AudioRecorder.onLevel
Files:
- Create:
Sources/SstCore/AudioLevel.swift - Modify:
Sources/SstCore/AudioRecorder.swift - Modify:
Sources/SstProbe/RecordProbe.swift - Test:
Tests/SstCoreTests/AudioLevelTests.swift
Interfaces:
-
Produces:
AudioLevel.rmsDbfs(_ samples: [Float]) -> FloatenrmsDbfs(_ samples: UnsafeBufferPointer<Float>) -> Float(−120 voor stilte of leeg)AudioLevel.vu(fromDbfs:referenceDbfs:) -> Float(standaard referentie −20)AudioRecorder.onLevel: ((Float) -> Void)?— RMS in dBFS per aangeleverde buffer, aangeroepen op de audio-thread; vóórstart()instellen.
-
Step 1: Schrijf de falende test
Tests/SstCoreTests/AudioLevelTests.swift:
import Testing
@testable import SstCore
struct AudioLevelTests {
@Test func constantTenthIsMinusTwentyDbfs() {
let level = AudioLevel.rmsDbfs([Float](repeating: 0.1, count: 1_000))
#expect(abs(level - (-20)) < 0.01)
}
@Test func fullScaleIsZeroDbfs() {
let level = AudioLevel.rmsDbfs([Float](repeating: 1.0, count: 100))
#expect(abs(level) < 0.01)
}
@Test func negativeSamplesCountLikePositiveOnes() {
let level = AudioLevel.rmsDbfs([Float](repeating: -0.1, count: 1_000))
#expect(abs(level - (-20)) < 0.01)
}
@Test func emptyAndSilentBuffersHitTheFloor() {
#expect(AudioLevel.rmsDbfs([]) == AudioLevel.silenceFloorDbfs)
#expect(AudioLevel.rmsDbfs([Float](repeating: 0, count: 64)) == AudioLevel.silenceFloorDbfs)
}
@Test func vuIsRelativeToTheReference() {
#expect(AudioLevel.vu(fromDbfs: -20) == 0)
#expect(AudioLevel.vu(fromDbfs: -40) == -20)
#expect(AudioLevel.vu(fromDbfs: -17) == 3)
#expect(AudioLevel.vu(fromDbfs: -30, referenceDbfs: -18) == -12)
}
}
- Step 2: Draai de test, moet falen
Run: swift test --filter AudioLevelTests
Expected: compilatiefout cannot find 'AudioLevel' in scope.
- Step 3: Implementatie
Sources/SstCore/AudioLevel.swift:
import Foundation
public enum AudioLevel {
/// Reported for empty or silent buffers (log of zero would be -infinity).
public static let silenceFloorDbfs: Float = -120
public static func rmsDbfs(_ samples: [Float]) -> Float {
samples.withUnsafeBufferPointer { rmsDbfs($0) }
}
public static func rmsDbfs(_ samples: UnsafeBufferPointer<Float>) -> Float {
guard !samples.isEmpty else { return silenceFloorDbfs }
var sum: Float = 0
for sample in samples { sum += sample * sample }
let rms = (sum / Float(samples.count)).squareRoot()
guard rms > 0 else { return silenceFloorDbfs }
return max(silenceFloorDbfs, 20 * log10(rms))
}
/// VU reading for a dBFS level, where `referenceDbfs` is 0 VU.
public static func vu(fromDbfs dbfs: Float, referenceDbfs: Float = -20) -> Float {
dbfs - referenceDbfs
}
}
In Sources/SstCore/AudioRecorder.swift:
- Voeg onder
private var buffer: [Float] = []toe:
/// RMS level (dBFS) of every delivered input buffer. Called on the audio thread; set before `start()`.
public var onLevel: ((Float) -> Void)?
- Vervang in de tap-closure de regel
guard let self else { return }door:
guard let self else { return }
if let onLevel = self.onLevel, let channel = pcm.floatChannelData?[0] {
onLevel(AudioLevel.rmsDbfs(UnsafeBufferPointer(start: channel, count: Int(pcm.frameLength))))
}
- Vervang
bufferSize: 4096doorbufferSize: 1024(macOS mag een grotere buffer leveren; de meter werkt met elke grootte).
Sources/SstProbe/RecordProbe.swift — vervang volledig door:
import AVFoundation
import SstCore
/// Records `seconds` of microphone audio; returns nil if access is denied or recording fails.
func recordSamples(seconds: Double, onLevel: ((Float) -> Void)? = nil) async -> [Float]? {
guard await AVCaptureDevice.requestAccess(for: .audio) else {
print("Microfoon-toegang geweigerd (Systeeminstellingen → Privacy → Microfoon).")
return nil
}
let recorder = AudioRecorder()
recorder.onLevel = onLevel
do { try recorder.start() } catch {
print("start mislukt:", error.localizedDescription)
return nil
}
print("Neem \(Int(seconds)) s op — praat nu")
try? await Task.sleep(for: .seconds(seconds))
return recorder.stop()
}
func runRecordProbe() async {
final class Levels: @unchecked Sendable {
let lock = NSLock()
var values: [Float] = []
func add(_ v: Float) { lock.withLock { values.append(v) } }
}
let levels = Levels()
guard let samples = await recordSamples(seconds: 3, onLevel: { levels.add($0) }) else { return }
let peak = samples.map { abs($0) }.max() ?? 0
print("samples: \(samples.count) (verwacht ≈ 48000), piek: \(peak)")
let values = levels.lock.withLock { levels.values }
if let lo = values.min(), let hi = values.max() {
print(String(format: "niveau-meldingen: %d (≈ %.0f per s), bereik %.1f … %.1f dBFS", values.count, Double(values.count) / 3, lo, hi))
} else {
print("geen niveau-meldingen ontvangen")
}
}
- Step 4: Draai alle tests en bouw
Run: swift test && swift build
Expected: alle tests PASS; build slaagt.
- Step 5: Handmatige controle (met de gebruiker, in Task 5, stap 6)
Hier alleen: de code compileert.
- Step 6: Commit
git add Sources Tests
git commit -m "feat: AudioLevel helper and AudioRecorder.onLevel" -m "Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"
Task 5: WhisperKit-voorrang — woordenlijst als promptTokens
Files:
- Modify:
Package.swift(testtargetSstWhisperTests) - Modify:
Sources/SstWhisper/WhisperKitTranscriber.swift - Modify:
Sources/SstProbe/TranscribeProbe.swift - Test:
Tests/SstWhisperTests/PromptTokensTests.swift
Interfaces:
-
Consumes: Task 1 (
Transcriber.transcribe(_:vocabulary:)). -
Produces:
WhisperKitTranscriber.transcribe(_:vocabulary:); internstatic func promptTokens(vocabulary:maxTokens:specialTokenBegin:encode:) -> [Int]?. -
Step 1: Testtarget toevoegen
In Package.swift voeg onder de bestaande .testTarget(name: "SstCoreTests", …) toe:
.testTarget(name: "SstWhisperTests", dependencies: ["SstWhisper"]),
- Step 2: Schrijf de falende test
Tests/SstWhisperTests/PromptTokensTests.swift:
import Testing
@testable import SstWhisper
struct PromptTokensTests {
/// Fake tokenizer: one token (id 7) per word, plus one special token (id 2000) that must be filtered out.
let encode: (String) -> [Int] = { text in
text.split(whereSeparator: { !$0.isLetter }).map { _ in 7 } + [2000]
}
@Test func allTermsFitAndSpecialTokensAreFiltered() {
let tokens = WhisperKitTranscriber.promptTokens(
vocabulary: ["Anthropic", "Vercel", "Supabase"], maxTokens: 64, specialTokenBegin: 1000, encode: encode)
#expect(tokens == [7, 7, 7])
}
@Test func termsThatDoNotFitAreDroppedFromTheEnd() {
let vocab = ["a", "b", "c", "d", "e", "f", "g", "h", "i", "j"]
let tokens = WhisperKitTranscriber.promptTokens(
vocabulary: vocab, maxTokens: 4, specialTokenBegin: 1000, encode: encode)
#expect(tokens?.count == 4)
}
@Test func emptyVocabularyGivesNoPrompt() {
#expect(WhisperKitTranscriber.promptTokens(
vocabulary: [], maxTokens: 64, specialTokenBegin: 1000, encode: encode) == nil)
}
@Test func firstTermAloneTooLongGivesNoPrompt() {
#expect(WhisperKitTranscriber.promptTokens(
vocabulary: ["one two three"], maxTokens: 2, specialTokenBegin: 1000, encode: encode) == nil)
}
}
- Step 3: Draai de test, moet falen
Run: swift test --filter PromptTokensTests
Expected: compilatiefout type 'WhisperKitTranscriber' has no member 'promptTokens'.
- Step 4: Implementatie
In Sources/SstWhisper/WhisperKitTranscriber.swift vervang de hele methode public func transcribe(_ samples: [Float]) async throws -> String { … } (onderaan het bestand) door:
public func transcribe(_ samples: [Float]) async throws -> String {
try await transcribe(samples, vocabulary: [])
}
/// Terms are added one by one while the encoded prompt stays within `maxTokens`.
/// Returns nil when nothing fits or the list is empty.
static func promptTokens(
vocabulary: [String],
maxTokens: Int = 64,
specialTokenBegin: Int,
encode: (String) -> [Int]
) -> [Int]? {
var kept: [String] = []
var tokens: [Int] = []
for term in vocabulary {
let candidate = kept + [term]
let encoded = encode(" " + candidate.joined(separator: ", ")).filter { $0 < specialTokenBegin }
if encoded.count > maxTokens { break }
kept = candidate
tokens = encoded
}
return tokens.isEmpty ? nil : tokens
}
public func transcribe(_ samples: [Float], vocabulary: [String]) async throws -> String {
guard let pipeline else { throw TranscriberError.notLoaded }
// language nil + detectLanguage: Whisper picks NL or EN per utterance.
var options = DecodingOptions(
task: .transcribe,
language: nil,
detectLanguage: true,
skipSpecialTokens: true,
withoutTimestamps: true
)
if !vocabulary.isEmpty, let tokenizer = pipeline.tokenizer {
options.promptTokens = Self.promptTokens(
vocabulary: vocabulary,
specialTokenBegin: tokenizer.specialTokens.specialTokenBegin,
encode: { tokenizer.encode(text: $0) }
)
}
let results = try await pipeline.transcribe(audioArray: samples, decodeOptions: options)
return results
.map(\.text)
.joined(separator: " ")
.trimmingCharacters(in: .whitespacesAndNewlines)
}
Wijkt de WhisperKit-API licht af (bijvoorbeeld de plaats van
specialTokens), pas dan alleen dit bestand aan en behoud de signatuur vanpromptTokenszodat de test geldig blijft.
Sources/SstProbe/TranscribeProbe.swift — vervang regel let text = try await transcriber.transcribe(samples) door:
let vocabulary = (ProcessInfo.processInfo.environment["SST_VOCAB"] ?? "")
.split(separator: ",").map { $0.trimmingCharacters(in: .whitespaces) }.filter { !$0.isEmpty }
if !vocabulary.isEmpty { print("voorrang: \(vocabulary.joined(separator: ", "))") }
let text = try await transcriber.transcribe(samples, vocabulary: vocabulary)
- Step 5: Draai alle tests en bouw
Run: swift test && swift build
Expected: alle tests PASS (incl. 4 nieuwe in SstWhisperTests); build slaagt.
- Step 6: Handmatige controle samen met de gebruiker (hardware)
Doe in deze volgorde en noteer de uitkomsten in het commit-bericht van stap 7:
swift run SstProbe record→ er verschijnt een regelniveau-meldingen: N (≈ X per s), bereik … dBFS. Verwacht: meer dan 5 meldingen per seconde en een bereik dat bij praten duidelijk hoger ligt dan bij stilte. Noteer X (macOS levert soms ~10 per s in plaats van ~47).swift run SstProbe transcribe— zeg een zin, noteer de tijd zonder voorrang.SST_VOCAB="Anthropic,Vercel,Supabase,Claude Code" swift run SstProbe transcribe— zeg dezelfde zin met een of twee van die woorden erin ("Ik werk met Supabase en Claude Code"); noteer de tijd en of de spelling beter is. Verwacht: tijd onder 1,5 s voor 8 s spraak. Is het trager, noteer hoeveel; dat is geen blokkade, wel informatie voor de spec.- Zeg niets (8 s stilte) met
SST_VOCABgezet: het resultaat mag geen woorden uit de lijst bevatten. - Hotkey-controle:
swift run SstProbe hotkeytoont nog steedspressed/releasedvoor Fn.
- Step 7: Commit
git add Package.swift Sources Tests
git commit -m "feat: WhisperKit transcriber biases decoding with vocabulary prompt tokens" -m "Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"
Self-review
- Spec-dekking: raakpunt 1 (woordenlijst →
promptTokens, Task 1+5), 2 (niveau, Task 4), 3 (corrector +onResult, Task 1+2), 4 (instelbare toets, Task 3). Voorrang-regels: RMS-grens 0,015 (Task 2 testquietRecordingSkipsVocabularyButStillTranscribes), echo-filter (Task 2), 64 tokens (Task 5). Niet in deze fase, wel in fase 2: het opbouwen van de woordenlijst (VocabularyBuilder) en de correctie zelf. - Placeholders: geen; de waarschuwing in Task 5 stap 4 is een gerichte terugval, geen open eind.
- Typeconsistentie:
AppliedCorrection(heard:wrote:rule:start:length:),CorrectionResult(text:corrections:)enDictationResult(date:duration:rawText:finalText:corrections:)worden in Task 2 precies zo gebruikt als in Task 1 gedefinieerd.HotkeyMonitor.bindinggebruiktHotkeyBinding.keyCode: Intenflag. - Risico's: de bufferlevering van
installTap(Task 4, stap 6.1) en de snelheid met prompt (stap 6.3) zijn de enige onbekenden; beide worden gemeten.