# Dictatie-skeleton Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Een werkende macOS-menubalk-app: Fn ingedrukt houden neemt op, loslaten transcribeert lokaal (NL/EN) en plakt de tekst op de cursor. **Architecture:** Een Swift Package. `SstCore` bevat de logica achter protocollen (testbaar met fakes), `SstWhisper` bevat de WhisperKit-implementatie, `Sst` is de SwiftUI-menubalk-app en `SstProbe` is een CLI om hotkey, opname en transcriptie los te testen. Een script bouwt de package tot een `.app`-bundle. **Tech Stack:** Swift 6.4 (language mode 5), SwiftUI `MenuBarExtra`, `AVAudioEngine`, `CGEventTap`, WhisperKit, Swift Testing. **Spec:** `docs/superpowers/specs/2026-10-01-push-to-talk-dictation-design.md` ## Global Constraints - Alleen Apple Silicon, macOS 14+ als deployment target (doelmachine: M4 Pro, macOS 27, Xcode 27). - Alles lokaal: geen netwerkverkeer behalve de eenmalige modeldownload; audio wordt nooit opgeslagen. - Audioformaat naar de transcriber: 16 kHz, mono, Float32. - Opnames korter dan 0,3 s (4800 samples) worden genegeerd. - Model: Whisper `large-v3-turbo` via WhisperKit, automatische taalherkenning (Nederlands en Engels). - Niet-gesandboxed; vereiste rechten: Microfoon, Accessibility, Input Monitoring. - Statemachine: `idle → recording → transcribing → inserting → idle`; fouten gaan naar `onError` en eindigen in `idle`. - Falend invoegen laat de tekst op het klembord staan. - Elke commit eindigt met de trailer `Co-Authored-By: Claude Sonnet 5.5 `. **Afwijkingen van de spec (bewust):** - De "dunne Xcode-app" is een app-bundle die `scripts/make-app.sh` uit de package assembleert. Een handgeschreven `.xcodeproj` is fragiel; Xcode kan `Package.swift` rechtstreeks openen. - Swift language mode 5, om strikte Sendable-regels buiten de skeleton-scope te houden. - De overlay-pill (bouwvolgorde-stap 5) valt buiten dit plan; de menubalk-icoon toont de status. ## Bestandsstructuur ``` Package.swift .gitignore scripts/make-app.sh Sources/SstCore/Models.swift HotkeyEvent, DictationState, CleaningContext, protocollen Sources/SstCore/DictationController.swift statemachine Sources/SstCore/PassthroughCleaner.swift Sources/SstCore/TextInserter.swift TextInserter + PasteboardProviding/KeystrokePosting Sources/SstCore/SystemPasteboard.swift NSPasteboard + CGEvent-implementaties Sources/SstCore/FnKeyTracker.swift pure Fn-logica Sources/SstCore/HotkeyMonitor.swift CGEventTap Sources/SstCore/AudioRecorder.swift AVAudioEngine → 16 kHz mono Sources/SstWhisper/WhisperKitTranscriber.swift Sources/SstProbe/main.swift, HotkeyProbe.swift, RecordProbe.swift Sources/Sst/SstApp.swift, AppModel.swift, MenuContent.swift Tests/SstCoreTests/Fakes.swift, DictationControllerTests.swift, TextInserterTests.swift, FnKeyTrackerTests.swift ``` --- ### Task 1: Package-scaffold en DictationController **Files:** - Create: `Package.swift`, `.gitignore` - Create: `Sources/SstCore/Models.swift`, `Sources/SstCore/DictationController.swift` - Test: `Tests/SstCoreTests/Fakes.swift`, `Tests/SstCoreTests/DictationControllerTests.swift` **Interfaces:** - Produces (gebruikt door alle latere taken): - `enum HotkeyEvent { case pressed, released }` - `enum DictationState: Equatable { case idle, recording, transcribing, inserting }` - `struct CleaningContext { init() }` - `protocol AudioRecording { func start() throws; func stop() -> [Float] }` - `protocol Transcriber { func transcribe(_ samples: [Float]) async throws -> String }` - `protocol Cleaner { func clean(_ text: String, context: CleaningContext) async throws -> String }` - `protocol TextInserting { func insert(_ text: String) async throws }` - `@MainActor final class DictationController` met `init(recorder:transcriber:cleaner:inserter:minimumSamples:)`, `var onStateChange: ((DictationState) -> Void)?`, `var onError: ((String) -> Void)?`, `func handle(_ event: HotkeyEvent) async` - [ ] **Step 1: Scaffold maken** `Package.swift`: ```swift // swift-tools-version: 6.0 import PackageDescription let package = Package( name: "Sst", platforms: [.macOS(.v14)], targets: [ .target(name: "SstCore"), .testTarget(name: "SstCoreTests", dependencies: ["SstCore"]), ], swiftLanguageModes: [.v5] ) ``` `.gitignore`: ``` .build/ build/ .DS_Store *.xcodeproj ``` `Sources/SstCore/Models.swift`: ```swift import Foundation public enum HotkeyEvent: Equatable, Sendable { case pressed case released } public enum DictationState: Equatable, Sendable { case idle case recording case transcribing case inserting } public struct CleaningContext: Sendable { public init() {} } public protocol AudioRecording { func start() throws /// Stops recording and returns all captured 16 kHz mono Float samples. func stop() -> [Float] } public protocol Transcriber { func transcribe(_ samples: [Float]) async throws -> String } public protocol Cleaner { func clean(_ text: String, context: CleaningContext) async throws -> String } public protocol TextInserting { func insert(_ text: String) async throws } ``` - [ ] **Step 2: Fakes en falende tests schrijven** `Tests/SstCoreTests/Fakes.swift`: ```swift import Foundation @testable import SstCore struct TestError: Error {} final class FakeRecorder: AudioRecording { var samples: [Float] = [] var startError: Error? private(set) var startCount = 0 func start() throws { startCount += 1 if let startError { throw startError } } func stop() -> [Float] { samples } } final class FakeTranscriber: Transcriber { var result: Result = .success("hallo wereld") private(set) var callCount = 0 func transcribe(_ samples: [Float]) async throws -> String { callCount += 1 return try result.get() } } final class FakeCleaner: Cleaner { func clean(_ text: String, context: CleaningContext) async throws -> String { "[\(text)]" } } final class FakeInserter: TextInserting { var error: Error? private(set) var inserted: [String] = [] func insert(_ text: String) async throws { if let error { throw error } inserted.append(text) } } ``` `Tests/SstCoreTests/DictationControllerTests.swift`: ```swift import Testing @testable import SstCore @MainActor struct DictationControllerTests { let recorder = FakeRecorder() let transcriber = FakeTranscriber() let cleaner = FakeCleaner() let inserter = FakeInserter() static let oneSecond = [Float](repeating: 0.1, count: 16_000) func makeController() -> DictationController { DictationController( recorder: recorder, transcriber: transcriber, cleaner: cleaner, inserter: inserter ) } @Test func pressAndReleaseTranscribesCleansAndInserts() async { recorder.samples = Self.oneSecond let controller = makeController() var states: [DictationState] = [] controller.onStateChange = { states.append($0) } await controller.handle(.pressed) await controller.handle(.released) #expect(states == [.recording, .transcribing, .inserting, .idle]) #expect(inserter.inserted == ["[hallo wereld]"]) } @Test func shortRecordingIsDiscarded() async { recorder.samples = [Float](repeating: 0.1, count: 1_000) let controller = makeController() var states: [DictationState] = [] controller.onStateChange = { states.append($0) } await controller.handle(.pressed) await controller.handle(.released) #expect(states == [.recording, .idle]) #expect(transcriber.callCount == 0) #expect(inserter.inserted.isEmpty) } @Test func emptyTranscriptionIsNotInserted() async { recorder.samples = Self.oneSecond transcriber.result = .success(" \n") let controller = makeController() var errors: [String] = [] controller.onError = { errors.append($0) } await controller.handle(.pressed) await controller.handle(.released) #expect(inserter.inserted.isEmpty) #expect(errors.isEmpty) #expect(controller.state == .idle) } @Test func transcriberErrorReportsAndReturnsToIdle() async { recorder.samples = Self.oneSecond transcriber.result = .failure(TestError()) let controller = makeController() var errors: [String] = [] controller.onError = { errors.append($0) } await controller.handle(.pressed) await controller.handle(.released) #expect(errors.count == 1) #expect(inserter.inserted.isEmpty) #expect(controller.state == .idle) } @Test func inserterErrorReportsAndReturnsToIdle() async { recorder.samples = Self.oneSecond inserter.error = TestError() let controller = makeController() var errors: [String] = [] controller.onError = { errors.append($0) } await controller.handle(.pressed) await controller.handle(.released) #expect(errors.count == 1) #expect(controller.state == .idle) } @Test func recorderStartFailureStaysIdle() async { recorder.startError = TestError() let controller = makeController() var errors: [String] = [] controller.onError = { errors.append($0) } await controller.handle(.pressed) #expect(errors.count == 1) #expect(controller.state == .idle) } @Test func secondPressWhileRecordingIsIgnored() async { let controller = makeController() await controller.handle(.pressed) await controller.handle(.pressed) #expect(recorder.startCount == 1) } @Test func releaseWithoutPressIsIgnored() async { let controller = makeController() var states: [DictationState] = [] controller.onStateChange = { states.append($0) } await controller.handle(.released) #expect(states.isEmpty) #expect(transcriber.callCount == 0) } } ``` - [ ] **Step 3: Test draaien, moet falen** Run: `swift test --filter DictationControllerTests` Expected: compilatiefout `cannot find 'DictationController' in scope`. - [ ] **Step 4: Minimale implementatie** `Sources/SstCore/DictationController.swift`: ```swift import Foundation @MainActor public final class DictationController { public private(set) var state: DictationState = .idle { didSet { onStateChange?(state) } } public var onStateChange: ((DictationState) -> Void)? public var onError: ((String) -> Void)? private let recorder: AudioRecording private let transcriber: Transcriber private let cleaner: Cleaner private let inserter: TextInserting private let minimumSamples: Int /// - Parameter minimumSamples: 16 kHz samples; default is 0.3 s. public init( recorder: AudioRecording, transcriber: Transcriber, cleaner: Cleaner, inserter: TextInserting, minimumSamples: Int = 4_800 ) { self.recorder = recorder self.transcriber = transcriber self.cleaner = cleaner self.inserter = inserter self.minimumSamples = minimumSamples } public func handle(_ event: HotkeyEvent) async { switch (event, state) { case (.pressed, .idle): do { try recorder.start() state = .recording } catch { onError?(error.localizedDescription) } case (.released, .recording): let samples = recorder.stop() guard samples.count >= minimumSamples else { state = .idle return } await process(samples) default: break } } private func process(_ samples: [Float]) async { defer { state = .idle } do { state = .transcribing let raw = try await transcriber.transcribe(samples) let text = raw.trimmingCharacters(in: .whitespacesAndNewlines) guard !text.isEmpty else { return } let cleaned = try await cleaner.clean(text, context: CleaningContext()) state = .inserting try await inserter.insert(cleaned) } catch { onError?(error.localizedDescription) } } } ``` - [ ] **Step 5: Test draaien, moet slagen** Run: `swift test --filter DictationControllerTests` Expected: 8 tests PASS. - [ ] **Step 6: Commit** ```bash git add Package.swift .gitignore Sources Tests git commit -m "feat: add package scaffold and DictationController state machine" -m "Co-Authored-By: Claude Sonnet 5.5 " ``` --- ### Task 2: PassthroughCleaner en TextInserter **Files:** - Create: `Sources/SstCore/PassthroughCleaner.swift`, `Sources/SstCore/TextInserter.swift`, `Sources/SstCore/SystemPasteboard.swift` - Test: `Tests/SstCoreTests/TextInserterTests.swift` **Interfaces:** - Consumes: `TextInserting`, `Cleaner`, `CleaningContext` uit Task 1. - Produces: - `typealias PasteboardItem = [String: Data]` - `protocol PasteboardProviding { func readItems() -> [PasteboardItem]; func writeString(_ text: String); func writeItems(_ items: [PasteboardItem]) }` - `protocol KeystrokePosting { func postPaste() throws }` - `struct TextInserter: TextInserting`, `init(pasteboard:paster:restoreDelay: Duration = .milliseconds(150))` - `struct PassthroughCleaner: Cleaner` - `struct SystemPasteboard: PasteboardProviding`, `struct CGEventPaster: KeystrokePosting` (init zonder argumenten) - [ ] **Step 1: Falende tests schrijven** `Tests/SstCoreTests/TextInserterTests.swift`: ```swift import Foundation import Testing @testable import SstCore final class FakePasteboard: PasteboardProviding { var items: [PasteboardItem] = [] func readItems() -> [PasteboardItem] { items } func writeString(_ text: String) { items = [["public.utf8-plain-text": Data(text.utf8)]] } func writeItems(_ items: [PasteboardItem]) { self.items = items } } final class FakePaster: KeystrokePosting { var shouldFail = false var onPaste: (() -> Void)? private(set) var pasteCount = 0 func postPaste() throws { pasteCount += 1 if shouldFail { throw TestError() } onPaste?() } } struct TextInserterTests { let pasteboard = FakePasteboard() let paster = FakePaster() func makeInserter() -> TextInserter { TextInserter(pasteboard: pasteboard, paster: paster, restoreDelay: .zero) } @Test func pastesTextThenRestoresPreviousClipboard() async throws { let original: [PasteboardItem] = [["public.utf8-plain-text": Data("oud".utf8)]] pasteboard.items = original var seenAtPaste: Data? paster.onPaste = { seenAtPaste = pasteboard.items.first?["public.utf8-plain-text"] } try await makeInserter().insert("hallo") #expect(seenAtPaste == Data("hallo".utf8)) #expect(paster.pasteCount == 1) #expect(pasteboard.items == original) } @Test func failedPasteLeavesTextOnClipboard() async { pasteboard.items = [["public.utf8-plain-text": Data("oud".utf8)]] paster.shouldFail = true await #expect(throws: TestError.self) { try await makeInserter().insert("hallo") } #expect(pasteboard.items.first?["public.utf8-plain-text"] == Data("hallo".utf8)) } @Test func passthroughCleanerReturnsTextUnchanged() async throws { let result = try await PassthroughCleaner().clean("hoi", context: CleaningContext()) #expect(result == "hoi") } } ``` - [ ] **Step 2: Test draaien, moet falen** Run: `swift test --filter TextInserterTests` Expected: compilatiefout `cannot find type 'PasteboardProviding' in scope`. - [ ] **Step 3: Implementatie** `Sources/SstCore/PassthroughCleaner.swift`: ```swift public struct PassthroughCleaner: Cleaner { public init() {} public func clean(_ text: String, context: CleaningContext) async throws -> String { text } } ``` `Sources/SstCore/TextInserter.swift`: ```swift import Foundation public typealias PasteboardItem = [String: Data] public protocol PasteboardProviding { func readItems() -> [PasteboardItem] func writeString(_ text: String) func writeItems(_ items: [PasteboardItem]) } public protocol KeystrokePosting { func postPaste() throws } public struct TextInserter: TextInserting { private let pasteboard: PasteboardProviding private let paster: KeystrokePosting private let restoreDelay: Duration public init( pasteboard: PasteboardProviding, paster: KeystrokePosting, restoreDelay: Duration = .milliseconds(150) ) { self.pasteboard = pasteboard self.paster = paster self.restoreDelay = restoreDelay } public func insert(_ text: String) async throws { let saved = pasteboard.readItems() pasteboard.writeString(text) // On failure the text deliberately stays on the clipboard. try paster.postPaste() // Give the target app time to read the clipboard before restoring it. try await Task.sleep(for: restoreDelay) pasteboard.writeItems(saved) } } ``` `Sources/SstCore/SystemPasteboard.swift`: ```swift import AppKit import ApplicationServices public struct SystemPasteboard: PasteboardProviding { public init() {} public func readItems() -> [PasteboardItem] { (NSPasteboard.general.pasteboardItems ?? []).map { item in var dict: PasteboardItem = [:] for type in item.types { if let data = item.data(forType: type) { dict[type.rawValue] = data } } return dict } } public func writeString(_ text: String) { let pasteboard = NSPasteboard.general pasteboard.clearContents() pasteboard.setString(text, forType: .string) } public func writeItems(_ items: [PasteboardItem]) { let pasteboard = NSPasteboard.general pasteboard.clearContents() let restored = items.map { dict -> NSPasteboardItem in let item = NSPasteboardItem() for (type, data) in dict { item.setData(data, forType: NSPasteboard.PasteboardType(type)) } return item } if !restored.isEmpty { pasteboard.writeObjects(restored) } } } public struct CGEventPaster: KeystrokePosting { public enum PasteError: Error, LocalizedError { case accessibilityDenied case eventCreationFailed public var errorDescription: String? { switch self { case .accessibilityDenied: "Geen Accessibility-rechten: tekst staat op het klembord, plak zelf met ⌘V." case .eventCreationFailed: "Kon het plak-event niet aanmaken: tekst staat op het klembord." } } } public init() {} public func postPaste() throws { guard AXIsProcessTrusted() else { throw PasteError.accessibilityDenied } let source = CGEventSource(stateID: .combinedSessionState) let vKey: CGKeyCode = 0x09 guard let down = CGEvent(keyboardEventSource: source, virtualKey: vKey, keyDown: true), let up = CGEvent(keyboardEventSource: source, virtualKey: vKey, keyDown: false) else { throw PasteError.eventCreationFailed } down.flags = .maskCommand up.flags = .maskCommand down.post(tap: .cgAnnotatedSessionEventTap) up.post(tap: .cgAnnotatedSessionEventTap) } } ``` - [ ] **Step 4: Alle tests draaien** Run: `swift test` Expected: alle tests PASS (8 controller + 3 inserter/cleaner). - [ ] **Step 5: Commit** ```bash git add Sources Tests git commit -m "feat: add TextInserter with clipboard restore and PassthroughCleaner" -m "Co-Authored-By: Claude Sonnet 5.5 " ``` --- ### Task 3: Fn-hotkey (FnKeyTracker, HotkeyMonitor) en probe **Files:** - Create: `Sources/SstCore/FnKeyTracker.swift`, `Sources/SstCore/HotkeyMonitor.swift` - Create: `Sources/SstProbe/main.swift`, `Sources/SstProbe/HotkeyProbe.swift` - Modify: `Package.swift` - Test: `Tests/SstCoreTests/FnKeyTrackerTests.swift` **Interfaces:** - Consumes: `HotkeyEvent` uit Task 1. - Produces: - `struct FnKeyTracker { init(); mutating func process(fnDown: Bool) -> HotkeyEvent? }` - `final class HotkeyMonitor { init(); var onEvent: ((HotkeyEvent) -> Void)?; func start() throws; func stop() }`, `HotkeyMonitor.HotkeyError: Error, LocalizedError` - Executable `SstProbe` met subcommando `hotkey`. - [ ] **Step 1: Falende test schrijven** `Tests/SstCoreTests/FnKeyTrackerTests.swift`: ```swift import Testing @testable import SstCore struct FnKeyTrackerTests { @Test func emitsPressedThenReleasedOnlyOnTransitions() { var tracker = FnKeyTracker() #expect(tracker.process(fnDown: false) == nil) #expect(tracker.process(fnDown: true) == .pressed) #expect(tracker.process(fnDown: true) == nil) #expect(tracker.process(fnDown: false) == .released) #expect(tracker.process(fnDown: false) == nil) } } ``` - [ ] **Step 2: Test draaien, moet falen** Run: `swift test --filter FnKeyTrackerTests` Expected: compilatiefout `cannot find 'FnKeyTracker' in scope`. - [ ] **Step 3: Implementatie** `Sources/SstCore/FnKeyTracker.swift`: ```swift public struct FnKeyTracker { private var isDown = false public init() {} /// Feed the current Fn flag state; returns an event only on a transition. public mutating func process(fnDown: Bool) -> HotkeyEvent? { guard fnDown != isDown else { return nil } isDown = fnDown return fnDown ? .pressed : .released } } ``` `Sources/SstCore/HotkeyMonitor.swift`: ```swift import ApplicationServices import CoreGraphics public final class HotkeyMonitor { public enum HotkeyError: Error, LocalizedError { case tapUnavailable public var errorDescription: String? { "Kan de Fn-toets niet volgen: geef Input Monitoring-rechten in Systeeminstellingen en probeer opnieuw (soms is een herstart van de app nodig)." } } public var onEvent: ((HotkeyEvent) -> Void)? private var tracker = FnKeyTracker() private var tap: CFMachPort? private var source: CFRunLoopSource? public init() {} public func start() throws { guard tap == nil else { return } let mask = CGEventMask(1 << CGEventType.flagsChanged.rawValue) let callback: CGEventTapCallBack = { _, type, event, refcon in if let refcon { let monitor = Unmanaged.fromOpaque(refcon).takeUnretainedValue() monitor.handle(type: type, event: event) } return Unmanaged.passUnretained(event) } guard let tap = CGEvent.tapCreate( tap: .cgSessionEventTap, place: .headInsertEventTap, options: .listenOnly, eventsOfInterest: mask, callback: callback, userInfo: Unmanaged.passUnretained(self).toOpaque() ) else { throw HotkeyError.tapUnavailable } let source = CFMachPortCreateRunLoopSource(kCFAllocatorDefault, tap, 0) CFRunLoopAddSource(CFRunLoopGetMain(), source, .commonModes) CGEvent.tapEnable(tap: tap, enable: true) self.tap = tap self.source = source } public func stop() { if let source { CFRunLoopRemoveSource(CFRunLoopGetMain(), source, .commonModes) } if let tap { CGEvent.tapEnable(tap: tap, enable: false) } tap = nil source = nil } private func handle(type: CGEventType, event: CGEvent) { switch type { case .tapDisabledByTimeout, .tapDisabledByUserInput: if let tap { CGEvent.tapEnable(tap: tap, enable: true) } case .flagsChanged: let fnDown = event.flags.contains(.maskSecondaryFn) if let hotkeyEvent = tracker.process(fnDown: fnDown) { onEvent?(hotkeyEvent) } default: break } } } ``` `Package.swift` (vervang volledig): ```swift // swift-tools-version: 6.0 import PackageDescription let package = Package( name: "Sst", platforms: [.macOS(.v14)], targets: [ .target(name: "SstCore"), .executableTarget(name: "SstProbe", dependencies: ["SstCore"]), .testTarget(name: "SstCoreTests", dependencies: ["SstCore"]), ], swiftLanguageModes: [.v5] ) ``` `Sources/SstProbe/main.swift`: ```swift import Foundation let command = CommandLine.arguments.dropFirst().first ?? "help" switch command { case "hotkey": runHotkeyProbe() default: print("usage: sst-probe hotkey") } ``` `Sources/SstProbe/HotkeyProbe.swift`: ```swift import Foundation import CoreGraphics import SstCore func runHotkeyProbe() -> Never { if !CGPreflightListenEventAccess() { _ = CGRequestListenEventAccess() } let monitor = HotkeyMonitor() monitor.onEvent = { print("hotkey:", $0) } do { try monitor.start() } catch { print("failed:", error.localizedDescription) exit(1) } print("Houd Fn ingedrukt en laat los. Ctrl-C om te stoppen.") RunLoop.main.run() exit(0) } ``` - [ ] **Step 4: Tests draaien** Run: `swift test` Expected: alle tests PASS (incl. `FnKeyTrackerTests`). - [ ] **Step 5: Handmatige check van de hotkey** Zet eerst in Systeeminstellingen → Toetsenbord "Druk op 🌐 voor" op **Doe niets** (anders opent Fn de emoji-kiezer/dicteren). Run: `swift run SstProbe hotkey` Geef de terminal-app (waar je dit draait) Input Monitoring-rechten als daarom gevraagd wordt en start opnieuw. Expected: bij Fn omlaag `hotkey: pressed`, bij loslaten `hotkey: released`, geen dubbele events. - [ ] **Step 6: Commit** ```bash git add Package.swift Sources Tests git commit -m "feat: add Fn hotkey monitor and probe CLI" -m "Co-Authored-By: Claude Sonnet 5.5 " ``` --- ### Task 4: AudioRecorder en record-probe **Files:** - Create: `Sources/SstCore/AudioRecorder.swift`, `Sources/SstProbe/RecordProbe.swift` - Modify: `Sources/SstProbe/main.swift` **Interfaces:** - Consumes: `AudioRecording` uit Task 1. - Produces: - `final class AudioRecorder: AudioRecording`, `init()`, `AudioRecorder.RecorderError: Error, LocalizedError` - Probe-helper `func recordSamples(seconds: Double) async -> [Float]?` (hergebruikt in Task 5) - Probe-subcommando `record` De AudioRecorder is hardware-afhankelijk en wordt daarom handmatig getest via de probe. - [ ] **Step 1: Implementatie** `Sources/SstCore/AudioRecorder.swift`: ```swift import AVFoundation public final class AudioRecorder: AudioRecording { public enum RecorderError: Error, LocalizedError { case converterUnavailable public var errorDescription: String? { "Kon de microfoon niet omzetten naar 16 kHz mono." } } private let engine = AVAudioEngine() private let lock = NSLock() private var buffer: [Float] = [] private let targetFormat = AVAudioFormat( commonFormat: .pcmFormatFloat32, sampleRate: 16_000, channels: 1, interleaved: false )! public init() {} public func start() throws { lock.withLock { buffer.removeAll(keepingCapacity: true) } let input = engine.inputNode let inputFormat = input.outputFormat(forBus: 0) guard let converter = AVAudioConverter(from: inputFormat, to: targetFormat) else { throw RecorderError.converterUnavailable } let ratio = targetFormat.sampleRate / inputFormat.sampleRate input.installTap(onBus: 0, bufferSize: 4096, format: inputFormat) { [weak self] pcm, _ in guard let self else { return } let capacity = AVAudioFrameCount(Double(pcm.frameLength) * ratio) + 1 guard let out = AVAudioPCMBuffer(pcmFormat: self.targetFormat, frameCapacity: capacity) else { return } var consumed = false var error: NSError? converter.convert(to: out, error: &error) { _, status in if consumed { status.pointee = .noDataNow return nil } consumed = true status.pointee = .haveData return pcm } guard error == nil, let channel = out.floatChannelData?[0] else { return } let samples = Array(UnsafeBufferPointer(start: channel, count: Int(out.frameLength))) self.lock.withLock { self.buffer.append(contentsOf: samples) } } engine.prepare() try engine.start() } public func stop() -> [Float] { engine.inputNode.removeTap(onBus: 0) engine.stop() return lock.withLock { let samples = buffer buffer = [] return samples } } } ``` `Sources/SstProbe/RecordProbe.swift`: ```swift import AVFoundation import SstCore /// Records `seconds` of microphone audio; returns nil if access is denied or recording fails. func recordSamples(seconds: Double) async -> [Float]? { guard await AVCaptureDevice.requestAccess(for: .audio) else { print("Microfoon-toegang geweigerd (Systeeminstellingen → Privacy → Microfoon).") return nil } let recorder = AudioRecorder() do { try recorder.start() } catch { print("start mislukt:", error.localizedDescription) return nil } print("Neem \(Int(seconds)) s op — praat nu") try? await Task.sleep(for: .seconds(seconds)) return recorder.stop() } func runRecordProbe() async { guard let samples = await recordSamples(seconds: 3) else { return } let peak = samples.map { abs($0) }.max() ?? 0 print("samples: \(samples.count) (verwacht ≈ 48000), piek: \(peak)") } ``` `Sources/SstProbe/main.swift` (vervang volledig): ```swift import Foundation let command = CommandLine.arguments.dropFirst().first ?? "help" switch command { case "hotkey": runHotkeyProbe() case "record": await runRecordProbe() default: print("usage: sst-probe hotkey|record") } ``` - [ ] **Step 2: Bouwen** Run: `swift build` Expected: build slaagt zonder errors. - [ ] **Step 3: Handmatige check** Run: `swift run SstProbe record` en praat 3 seconden. Expected: `samples:` ligt rond 48000 (±2000) en `piek` is duidelijk > 0,05. Bij stilte is de piek ≈ 0. Bij een geweigerde microfoon staat er een duidelijke melding. - [ ] **Step 4: Commit** ```bash git add Sources git commit -m "feat: add AudioRecorder (16 kHz mono) and record probe" -m "Co-Authored-By: Claude Sonnet 5.5 " ``` --- ### Task 5: WhisperKitTranscriber en transcribe-probe **Files:** - Create: `Sources/SstWhisper/WhisperKitTranscriber.swift` - Create: `Sources/SstProbe/TranscribeProbe.swift` - Modify: `Package.swift`, `Sources/SstProbe/main.swift` **Interfaces:** - Consumes: `Transcriber` uit Task 1, `recordSamples(seconds:)` uit Task 4. - Produces: - `final class WhisperKitTranscriber: Transcriber` met `init(model: String = "large-v3-v20240930_turbo")`, `func load() async throws`, `WhisperKitTranscriber.TranscriberError: Error, LocalizedError` - Probe-subcommando `transcribe` - [ ] **Step 1: Dependency en target toevoegen** `Package.swift` (vervang volledig): ```swift // swift-tools-version: 6.0 import PackageDescription let package = Package( name: "Sst", platforms: [.macOS(.v14)], dependencies: [ .package(url: "https://github.com/argmaxinc/WhisperKit.git", from: "0.9.0"), ], targets: [ .target(name: "SstCore"), .target( name: "SstWhisper", dependencies: [ "SstCore", .product(name: "WhisperKit", package: "WhisperKit"), ] ), .executableTarget(name: "SstProbe", dependencies: ["SstCore", "SstWhisper"]), .testTarget(name: "SstCoreTests", dependencies: ["SstCore"]), ], swiftLanguageModes: [.v5] ) ``` - [ ] **Step 2: Implementatie** `Sources/SstWhisper/WhisperKitTranscriber.swift`: ```swift import Foundation import SstCore import WhisperKit public final class WhisperKitTranscriber: Transcriber { public enum TranscriberError: Error, LocalizedError { case notLoaded public var errorDescription: String? { "Spraakmodel is nog niet geladen." } } private let modelName: String private var pipeline: WhisperKit? public init(model: String = "large-v3-v20240930_turbo") { self.modelName = model } /// Downloads the model on first run (≈1.5 GB) and loads it into memory. public func load() async throws { pipeline = try await WhisperKit(WhisperKitConfig(model: modelName)) } public func transcribe(_ samples: [Float]) async throws -> String { guard let pipeline else { throw TranscriberError.notLoaded } // language nil + detectLanguage: Whisper picks NL or EN per utterance. let options = DecodingOptions( task: .transcribe, language: nil, detectLanguage: true, skipSpecialTokens: true, withoutTimestamps: true ) let results = try await pipeline.transcribe(audioArray: samples, decodeOptions: options) return results .map(\.text) .joined(separator: " ") .trimmingCharacters(in: .whitespacesAndNewlines) } } ``` > Als de WhisperKit-API of het modelnaam-patroon in de opgeloste versie afwijkt (compilatiefout of "model not found"), pas dan alleen dit bestand aan. Beschikbare modelnamen: `WhisperKit.fetchAvailableModels()`. Houd `Transcriber` en `load()` gelijk, want Task 6 hangt daarvan af. `Sources/SstProbe/TranscribeProbe.swift`: ```swift import Foundation import SstWhisper func runTranscribeProbe() async { guard let samples = await recordSamples(seconds: 6) else { return } print("Model laden (eerste keer: download ≈1.5 GB)…") let transcriber = WhisperKitTranscriber() do { try await transcriber.load() let started = Date.now let text = try await transcriber.transcribe(samples) let elapsed = Date.now.timeIntervalSince(started) print(String(format: "(%.2f s) ", elapsed) + text) } catch { print("mislukt:", error.localizedDescription) } } ``` `Sources/SstProbe/main.swift` (vervang volledig): ```swift import Foundation let command = CommandLine.arguments.dropFirst().first ?? "help" switch command { case "hotkey": runHotkeyProbe() case "record": await runRecordProbe() case "transcribe": await runTranscribeProbe() default: print("usage: sst-probe hotkey|record|transcribe") } ``` - [ ] **Step 3: Bouwen** Run: `swift build` Expected: dependencies worden opgehaald, build slaagt. Los API-afwijkingen op in `WhisperKitTranscriber.swift` (zie opmerking hierboven). - [ ] **Step 4: Handmatige check (NL, EN, gemengd)** Run: `swift run SstProbe transcribe` Zeg achtereenvolgens: een Nederlandse zin, een Engelse zin, en een zin die begint in het Nederlands met een paar Engelse woorden. Expected: correcte transcriptie van de zinnen; na het laden duurt transcriberen van 6 s audio ruim onder 3 s op de M4 Pro. Noteer de gemeten tijd. Bekende beperking: taaldetectie is per opname, dus wisselen midden in een zin kan een van beide talen minder goed herkennen. Dat is acceptabel voor de skeleton. - [ ] **Step 5: Commit** ```bash git add Package.swift Package.resolved Sources git commit -m "feat: add WhisperKit transcriber and transcribe probe" -m "Co-Authored-By: Claude Sonnet 5.5 " ``` --- ### Task 6: Menubalk-app, app-bundle en rooktest **Files:** - Create: `Sources/Sst/SstApp.swift`, `Sources/Sst/AppModel.swift`, `Sources/Sst/MenuContent.swift` - Create: `scripts/make-app.sh` - Modify: `Package.swift` **Interfaces:** - Consumes: `DictationController`, `HotkeyMonitor`, `AudioRecorder`, `TextInserter`, `SystemPasteboard`, `CGEventPaster`, `PassthroughCleaner` (SstCore); `WhisperKitTranscriber` met `load()` (SstWhisper). - Produces: de app-bundle `build/Sst.app`. - [ ] **Step 1: Executable target toevoegen** In `Package.swift`: voeg de regel `.executableTarget(name: "Sst", dependencies: ["SstCore", "SstWhisper"]),` toe aan `targets`, direct boven de `SstProbe`-regel. - [ ] **Step 2: App-code** `Sources/Sst/AppModel.swift`: ```swift import AVFoundation import ApplicationServices import SwiftUI import SstCore import SstWhisper @MainActor final class AppModel: ObservableObject { @Published var state: DictationState = .idle @Published var modelStatus = "Model laden…" @Published var modelReady = false @Published var lastError: String? @Published var micGranted = false @Published var accessibilityGranted = false @Published var inputMonitoringGranted = false private let transcriber = WhisperKitTranscriber() private let hotkey = HotkeyMonitor() private let controller: DictationController init() { controller = DictationController( recorder: AudioRecorder(), transcriber: transcriber, cleaner: PassthroughCleaner(), inserter: TextInserter(pasteboard: SystemPasteboard(), paster: CGEventPaster()) ) controller.onStateChange = { [weak self] in self?.state = $0 } controller.onError = { [weak self] in self?.lastError = $0 } hotkey.onEvent = { [weak self] event in guard let self, self.modelReady else { return } Task { await self.controller.handle(event) } } Task { await start() } } var statusLine: String { guard modelReady else { return modelStatus } switch state { case .idle: return "Klaar — houd Fn ingedrukt om te dicteren" case .recording: return "Luistert…" case .transcribing: return "Transcriberen…" case .inserting: return "Invoegen…" } } var iconName: String { switch state { case .idle: return modelReady ? "mic" : "mic.slash" case .recording: return "mic.fill" case .transcribing, .inserting: return "waveform" } } func refreshPermissions() { micGranted = AVCaptureDevice.authorizationStatus(for: .audio) == .authorized accessibilityGranted = AXIsProcessTrusted() inputMonitoringGranted = CGPreflightListenEventAccess() } func retryHotkey() { refreshPermissions() do { try hotkey.start() lastError = nil } catch { lastError = error.localizedDescription } } private func start() async { _ = await AVCaptureDevice.requestAccess(for: .audio) if !CGPreflightListenEventAccess() { _ = CGRequestListenEventAccess() } if !AXIsProcessTrusted() { _ = AXIsProcessTrustedWithOptions(["AXTrustedCheckOptionPrompt": true] as CFDictionary) } retryHotkey() do { try await transcriber.load() modelReady = true modelStatus = "Klaar" } catch { modelStatus = "Model laden mislukt" lastError = error.localizedDescription } } } ``` `Sources/Sst/MenuContent.swift`: ```swift import AppKit import SwiftUI struct MenuContent: View { @ObservedObject var model: AppModel var body: some View { Text(model.statusLine) if let error = model.lastError { Text("Fout: \(error)") } Divider() permissionRow("Microfoon", granted: model.micGranted, pane: "Privacy_Microphone") permissionRow("Accessibility", granted: model.accessibilityGranted, pane: "Privacy_Accessibility") permissionRow("Input Monitoring", granted: model.inputMonitoringGranted, pane: "Privacy_ListenEvent") Button("Rechten opnieuw controleren") { model.retryHotkey() } Divider() Button("Stop") { NSApplication.shared.terminate(nil) } .keyboardShortcut("q") } @ViewBuilder private func permissionRow(_ title: String, granted: Bool, pane: String) -> some View { if granted { Text("✓ \(title)") } else { Button("⚠︎ \(title) — open Instellingen") { if let url = URL(string: "x-apple.systempreferences:com.apple.preference.security?\(pane)") { NSWorkspace.shared.open(url) } } } } } ``` `Sources/Sst/SstApp.swift`: ```swift import SwiftUI @main struct SstApp: App { @StateObject private var model = AppModel() var body: some Scene { MenuBarExtra { MenuContent(model: model) } label: { Image(systemName: model.iconName) } } } ``` > Als de model-start pas begint na het openen van het menu (StateObject wordt lui aangemaakt), verplaats dan `AppModel` naar een `NSApplicationDelegateAdaptor`-delegate die het model in `applicationDidFinishLaunching` aanmaakt. - [ ] **Step 3: Bundle-script** `scripts/make-app.sh`: ```bash #!/usr/bin/env bash set -euo pipefail cd "$(dirname "$0")/.." swift build -c release --product Sst BIN=".build/release/Sst" APP="build/Sst.app" rm -rf "$APP" mkdir -p "$APP/Contents/MacOS" "$APP/Contents/Resources" cp "$BIN" "$APP/Contents/MacOS/Sst" # SwiftPM resource bundles (if any dependency ships them) live next to the binary. for b in .build/release/*.bundle; do [ -e "$b" ] && cp -R "$b" "$APP/Contents/Resources/" done cat > "$APP/Contents/Info.plist" <<'PLIST' CFBundleIdentifierlocal.sst.app CFBundleNameSst CFBundleExecutableSst CFBundlePackageTypeAPPL CFBundleShortVersionString0.1.0 CFBundleVersion1 LSMinimumSystemVersion14.0 LSUIElement NSMicrophoneUsageDescription Sst neemt je stem op zolang je Fn ingedrukt houdt, om die lokaal naar tekst om te zetten. PLIST codesign --force --sign - --identifier local.sst.app "$APP" echo "Gebouwd: $APP" ``` - [ ] **Step 4: Bouwen** Run: `chmod +x scripts/make-app.sh && ./scripts/make-app.sh` Expected: `Gebouwd: build/Sst.app`, zonder errors. - [ ] **Step 5: Rooktest van de hele keten** Run: `open build/Sst.app` Controleer in deze volgorde: 1. Mic-icoon verschijnt in de menubalk (geen Dock-icoon). Menu toont "Model laden…" en daarna "Klaar — houd Fn ingedrukt om te dicteren" (eerste keer duurt de download enkele minuten). 2. Geef Microfoon, Accessibility en Input Monitoring toe via de prompts of de knoppen in het menu; klik "Rechten opnieuw controleren" (zo nodig de app herstarten). Alle drie staan op ✓. 3. Open TextEdit, klik in een leeg document, houd Fn ingedrukt, zeg "Dit is een test van de dictatie", laat los. Icoon: `mic.fill` → `waveform` → `mic`. Tekst verschijnt op de cursor. 4. Herhaal in Notes en in een browser-tekstveld, met een Engelse en een gemengde zin. 5. Zet een eigen tekst op het klembord, dicteer, en controleer dat het klembord daarna weer je eigen tekst bevat. 6. Tik Fn kort aan (< 0,3 s): er mag niets ingevoegd worden. 7. Weiger Accessibility tijdelijk: het menu toont een foutmelding en de tekst staat op het klembord. Bekende beperking: omdat de app ad-hoc is gesigneerd (`codesign --sign -`), kan macOS de rechten na elke herbouw opnieuw vragen. Een vast ontwikkelcertificaat lost dat later op. - [ ] **Step 6: Commit** ```bash git add Package.swift Sources scripts git commit -m "feat: add menu bar app and bundle script" -m "Co-Authored-By: Claude Sonnet 5.5 " ``` --- ## Self-review - **Spec-dekking:** HotkeyMonitor (Task 3), AudioRecorder (4), Transcriber/WhisperKit (5), Cleaner-naad (2), TextInserter met klembordherstel en klembord-bij-falen (2), DictationController met statemachine, minimale opnameduur, lege transcriptie en foutpaden (1), MenuBarExtra met rechten-check en modelstatus (6). Niet gedekt, bewust: overlay-pill, cleanup, branding (latere specs). - **Placeholders:** geen; alle codestappen bevatten volledige code. De twee opmerkingen over WhisperKit-API en StateObject zijn gerichte fallbacks, geen open eindjes. - **Typeconsistentie:** `HotkeyEvent`, `DictationState`, `CleaningContext`, de vier protocollen en `DictationController.init(recorder:transcriber:cleaner:inserter:minimumSamples:)` zijn in Task 1 gedefinieerd en komen identiek terug in Task 2 t/m 6. `WhisperKitTranscriber.load()` uit Task 5 wordt in Task 6 gebruikt. - **Risico's:** de WhisperKit-API en het exacte modelnaam-patroon zijn het enige onverifieerde punt; Task 5 stap 3 vangt dat vroeg af.