Files
sst/docs/superpowers/plans/2026-10-01-dictation-skeleton.md
T
2026-10-01 09:35:24 +02:00

1400 lines
44 KiB
Markdown

# Dictatie-skeleton Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Een werkende macOS-menubalk-app: Fn ingedrukt houden neemt op, loslaten transcribeert lokaal (NL/EN) en plakt de tekst op de cursor.
**Architecture:** Een Swift Package. `SstCore` bevat de logica achter protocollen (testbaar met fakes), `SstWhisper` bevat de WhisperKit-implementatie, `Sst` is de SwiftUI-menubalk-app en `SstProbe` is een CLI om hotkey, opname en transcriptie los te testen. Een script bouwt de package tot een `.app`-bundle.
**Tech Stack:** Swift 6.4 (language mode 5), SwiftUI `MenuBarExtra`, `AVAudioEngine`, `CGEventTap`, WhisperKit, Swift Testing.
**Spec:** `docs/superpowers/specs/2026-10-01-push-to-talk-dictation-design.md`
## Global Constraints
- Alleen Apple Silicon, macOS 14+ als deployment target (doelmachine: M4 Pro, macOS 27, Xcode 27).
- Alles lokaal: geen netwerkverkeer behalve de eenmalige modeldownload; audio wordt nooit opgeslagen.
- Audioformaat naar de transcriber: 16 kHz, mono, Float32.
- Opnames korter dan 0,3 s (4800 samples) worden genegeerd.
- Model: Whisper `large-v3-turbo` via WhisperKit, automatische taalherkenning (Nederlands en Engels).
- Niet-gesandboxed; vereiste rechten: Microfoon, Accessibility, Input Monitoring.
- Statemachine: `idle → recording → transcribing → inserting → idle`; fouten gaan naar `onError` en eindigen in `idle`.
- Falend invoegen laat de tekst op het klembord staan.
- Elke commit eindigt met de trailer `Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>`.
**Afwijkingen van de spec (bewust):**
- De "dunne Xcode-app" is een app-bundle die `scripts/make-app.sh` uit de package assembleert. Een handgeschreven `.xcodeproj` is fragiel; Xcode kan `Package.swift` rechtstreeks openen.
- Swift language mode 5, om strikte Sendable-regels buiten de skeleton-scope te houden.
- De overlay-pill (bouwvolgorde-stap 5) valt buiten dit plan; de menubalk-icoon toont de status.
## Bestandsstructuur
```
Package.swift
.gitignore
scripts/make-app.sh
Sources/SstCore/Models.swift HotkeyEvent, DictationState, CleaningContext, protocollen
Sources/SstCore/DictationController.swift statemachine
Sources/SstCore/PassthroughCleaner.swift
Sources/SstCore/TextInserter.swift TextInserter + PasteboardProviding/KeystrokePosting
Sources/SstCore/SystemPasteboard.swift NSPasteboard + CGEvent-implementaties
Sources/SstCore/FnKeyTracker.swift pure Fn-logica
Sources/SstCore/HotkeyMonitor.swift CGEventTap
Sources/SstCore/AudioRecorder.swift AVAudioEngine → 16 kHz mono
Sources/SstWhisper/WhisperKitTranscriber.swift
Sources/SstProbe/main.swift, HotkeyProbe.swift, RecordProbe.swift
Sources/Sst/SstApp.swift, AppModel.swift, MenuContent.swift
Tests/SstCoreTests/Fakes.swift, DictationControllerTests.swift,
TextInserterTests.swift, FnKeyTrackerTests.swift
```
---
### Task 1: Package-scaffold en DictationController
**Files:**
- Create: `Package.swift`, `.gitignore`
- Create: `Sources/SstCore/Models.swift`, `Sources/SstCore/DictationController.swift`
- Test: `Tests/SstCoreTests/Fakes.swift`, `Tests/SstCoreTests/DictationControllerTests.swift`
**Interfaces:**
- Produces (gebruikt door alle latere taken):
- `enum HotkeyEvent { case pressed, released }`
- `enum DictationState: Equatable { case idle, recording, transcribing, inserting }`
- `struct CleaningContext { init() }`
- `protocol AudioRecording { func start() throws; func stop() -> [Float] }`
- `protocol Transcriber { func transcribe(_ samples: [Float]) async throws -> String }`
- `protocol Cleaner { func clean(_ text: String, context: CleaningContext) async throws -> String }`
- `protocol TextInserting { func insert(_ text: String) async throws }`
- `@MainActor final class DictationController` met `init(recorder:transcriber:cleaner:inserter:minimumSamples:)`, `var onStateChange: ((DictationState) -> Void)?`, `var onError: ((String) -> Void)?`, `func handle(_ event: HotkeyEvent) async`
- [ ] **Step 1: Scaffold maken**
`Package.swift`:
```swift
// swift-tools-version: 6.0
import PackageDescription
let package = Package(
name: "Sst",
platforms: [.macOS(.v14)],
targets: [
.target(name: "SstCore"),
.testTarget(name: "SstCoreTests", dependencies: ["SstCore"]),
],
swiftLanguageModes: [.v5]
)
```
`.gitignore`:
```
.build/
build/
.DS_Store
*.xcodeproj
```
`Sources/SstCore/Models.swift`:
```swift
import Foundation
public enum HotkeyEvent: Equatable, Sendable {
case pressed
case released
}
public enum DictationState: Equatable, Sendable {
case idle
case recording
case transcribing
case inserting
}
public struct CleaningContext: Sendable {
public init() {}
}
public protocol AudioRecording {
func start() throws
/// Stops recording and returns all captured 16 kHz mono Float samples.
func stop() -> [Float]
}
public protocol Transcriber {
func transcribe(_ samples: [Float]) async throws -> String
}
public protocol Cleaner {
func clean(_ text: String, context: CleaningContext) async throws -> String
}
public protocol TextInserting {
func insert(_ text: String) async throws
}
```
- [ ] **Step 2: Fakes en falende tests schrijven**
`Tests/SstCoreTests/Fakes.swift`:
```swift
import Foundation
@testable import SstCore
struct TestError: Error {}
final class FakeRecorder: AudioRecording {
var samples: [Float] = []
var startError: Error?
private(set) var startCount = 0
func start() throws {
startCount += 1
if let startError { throw startError }
}
func stop() -> [Float] { samples }
}
final class FakeTranscriber: Transcriber {
var result: Result<String, Error> = .success("hallo wereld")
private(set) var callCount = 0
func transcribe(_ samples: [Float]) async throws -> String {
callCount += 1
return try result.get()
}
}
final class FakeCleaner: Cleaner {
func clean(_ text: String, context: CleaningContext) async throws -> String {
"[\(text)]"
}
}
final class FakeInserter: TextInserting {
var error: Error?
private(set) var inserted: [String] = []
func insert(_ text: String) async throws {
if let error { throw error }
inserted.append(text)
}
}
```
`Tests/SstCoreTests/DictationControllerTests.swift`:
```swift
import Testing
@testable import SstCore
@MainActor
struct DictationControllerTests {
let recorder = FakeRecorder()
let transcriber = FakeTranscriber()
let cleaner = FakeCleaner()
let inserter = FakeInserter()
static let oneSecond = [Float](repeating: 0.1, count: 16_000)
func makeController() -> DictationController {
DictationController(
recorder: recorder,
transcriber: transcriber,
cleaner: cleaner,
inserter: inserter
)
}
@Test func pressAndReleaseTranscribesCleansAndInserts() async {
recorder.samples = Self.oneSecond
let controller = makeController()
var states: [DictationState] = []
controller.onStateChange = { states.append($0) }
await controller.handle(.pressed)
await controller.handle(.released)
#expect(states == [.recording, .transcribing, .inserting, .idle])
#expect(inserter.inserted == ["[hallo wereld]"])
}
@Test func shortRecordingIsDiscarded() async {
recorder.samples = [Float](repeating: 0.1, count: 1_000)
let controller = makeController()
var states: [DictationState] = []
controller.onStateChange = { states.append($0) }
await controller.handle(.pressed)
await controller.handle(.released)
#expect(states == [.recording, .idle])
#expect(transcriber.callCount == 0)
#expect(inserter.inserted.isEmpty)
}
@Test func emptyTranscriptionIsNotInserted() async {
recorder.samples = Self.oneSecond
transcriber.result = .success(" \n")
let controller = makeController()
var errors: [String] = []
controller.onError = { errors.append($0) }
await controller.handle(.pressed)
await controller.handle(.released)
#expect(inserter.inserted.isEmpty)
#expect(errors.isEmpty)
#expect(controller.state == .idle)
}
@Test func transcriberErrorReportsAndReturnsToIdle() async {
recorder.samples = Self.oneSecond
transcriber.result = .failure(TestError())
let controller = makeController()
var errors: [String] = []
controller.onError = { errors.append($0) }
await controller.handle(.pressed)
await controller.handle(.released)
#expect(errors.count == 1)
#expect(inserter.inserted.isEmpty)
#expect(controller.state == .idle)
}
@Test func inserterErrorReportsAndReturnsToIdle() async {
recorder.samples = Self.oneSecond
inserter.error = TestError()
let controller = makeController()
var errors: [String] = []
controller.onError = { errors.append($0) }
await controller.handle(.pressed)
await controller.handle(.released)
#expect(errors.count == 1)
#expect(controller.state == .idle)
}
@Test func recorderStartFailureStaysIdle() async {
recorder.startError = TestError()
let controller = makeController()
var errors: [String] = []
controller.onError = { errors.append($0) }
await controller.handle(.pressed)
#expect(errors.count == 1)
#expect(controller.state == .idle)
}
@Test func secondPressWhileRecordingIsIgnored() async {
let controller = makeController()
await controller.handle(.pressed)
await controller.handle(.pressed)
#expect(recorder.startCount == 1)
}
@Test func releaseWithoutPressIsIgnored() async {
let controller = makeController()
var states: [DictationState] = []
controller.onStateChange = { states.append($0) }
await controller.handle(.released)
#expect(states.isEmpty)
#expect(transcriber.callCount == 0)
}
}
```
- [ ] **Step 3: Test draaien, moet falen**
Run: `swift test --filter DictationControllerTests`
Expected: compilatiefout `cannot find 'DictationController' in scope`.
- [ ] **Step 4: Minimale implementatie**
`Sources/SstCore/DictationController.swift`:
```swift
import Foundation
@MainActor
public final class DictationController {
public private(set) var state: DictationState = .idle {
didSet { onStateChange?(state) }
}
public var onStateChange: ((DictationState) -> Void)?
public var onError: ((String) -> Void)?
private let recorder: AudioRecording
private let transcriber: Transcriber
private let cleaner: Cleaner
private let inserter: TextInserting
private let minimumSamples: Int
/// - Parameter minimumSamples: 16 kHz samples; default is 0.3 s.
public init(
recorder: AudioRecording,
transcriber: Transcriber,
cleaner: Cleaner,
inserter: TextInserting,
minimumSamples: Int = 4_800
) {
self.recorder = recorder
self.transcriber = transcriber
self.cleaner = cleaner
self.inserter = inserter
self.minimumSamples = minimumSamples
}
public func handle(_ event: HotkeyEvent) async {
switch (event, state) {
case (.pressed, .idle):
do {
try recorder.start()
state = .recording
} catch {
onError?(error.localizedDescription)
}
case (.released, .recording):
let samples = recorder.stop()
guard samples.count >= minimumSamples else {
state = .idle
return
}
await process(samples)
default:
break
}
}
private func process(_ samples: [Float]) async {
defer { state = .idle }
do {
state = .transcribing
let raw = try await transcriber.transcribe(samples)
let text = raw.trimmingCharacters(in: .whitespacesAndNewlines)
guard !text.isEmpty else { return }
let cleaned = try await cleaner.clean(text, context: CleaningContext())
state = .inserting
try await inserter.insert(cleaned)
} catch {
onError?(error.localizedDescription)
}
}
}
```
- [ ] **Step 5: Test draaien, moet slagen**
Run: `swift test --filter DictationControllerTests`
Expected: 8 tests PASS.
- [ ] **Step 6: Commit**
```bash
git add Package.swift .gitignore Sources Tests
git commit -m "feat: add package scaffold and DictationController state machine" -m "Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"
```
---
### Task 2: PassthroughCleaner en TextInserter
**Files:**
- Create: `Sources/SstCore/PassthroughCleaner.swift`, `Sources/SstCore/TextInserter.swift`, `Sources/SstCore/SystemPasteboard.swift`
- Test: `Tests/SstCoreTests/TextInserterTests.swift`
**Interfaces:**
- Consumes: `TextInserting`, `Cleaner`, `CleaningContext` uit Task 1.
- Produces:
- `typealias PasteboardItem = [String: Data]`
- `protocol PasteboardProviding { func readItems() -> [PasteboardItem]; func writeString(_ text: String); func writeItems(_ items: [PasteboardItem]) }`
- `protocol KeystrokePosting { func postPaste() throws }`
- `struct TextInserter: TextInserting`, `init(pasteboard:paster:restoreDelay: Duration = .milliseconds(150))`
- `struct PassthroughCleaner: Cleaner`
- `struct SystemPasteboard: PasteboardProviding`, `struct CGEventPaster: KeystrokePosting` (init zonder argumenten)
- [ ] **Step 1: Falende tests schrijven**
`Tests/SstCoreTests/TextInserterTests.swift`:
```swift
import Foundation
import Testing
@testable import SstCore
final class FakePasteboard: PasteboardProviding {
var items: [PasteboardItem] = []
func readItems() -> [PasteboardItem] { items }
func writeString(_ text: String) {
items = [["public.utf8-plain-text": Data(text.utf8)]]
}
func writeItems(_ items: [PasteboardItem]) { self.items = items }
}
final class FakePaster: KeystrokePosting {
var shouldFail = false
var onPaste: (() -> Void)?
private(set) var pasteCount = 0
func postPaste() throws {
pasteCount += 1
if shouldFail { throw TestError() }
onPaste?()
}
}
struct TextInserterTests {
let pasteboard = FakePasteboard()
let paster = FakePaster()
func makeInserter() -> TextInserter {
TextInserter(pasteboard: pasteboard, paster: paster, restoreDelay: .zero)
}
@Test func pastesTextThenRestoresPreviousClipboard() async throws {
let original: [PasteboardItem] = [["public.utf8-plain-text": Data("oud".utf8)]]
pasteboard.items = original
var seenAtPaste: Data?
paster.onPaste = { seenAtPaste = pasteboard.items.first?["public.utf8-plain-text"] }
try await makeInserter().insert("hallo")
#expect(seenAtPaste == Data("hallo".utf8))
#expect(paster.pasteCount == 1)
#expect(pasteboard.items == original)
}
@Test func failedPasteLeavesTextOnClipboard() async {
pasteboard.items = [["public.utf8-plain-text": Data("oud".utf8)]]
paster.shouldFail = true
await #expect(throws: TestError.self) {
try await makeInserter().insert("hallo")
}
#expect(pasteboard.items.first?["public.utf8-plain-text"] == Data("hallo".utf8))
}
@Test func passthroughCleanerReturnsTextUnchanged() async throws {
let result = try await PassthroughCleaner().clean("hoi", context: CleaningContext())
#expect(result == "hoi")
}
}
```
- [ ] **Step 2: Test draaien, moet falen**
Run: `swift test --filter TextInserterTests`
Expected: compilatiefout `cannot find type 'PasteboardProviding' in scope`.
- [ ] **Step 3: Implementatie**
`Sources/SstCore/PassthroughCleaner.swift`:
```swift
public struct PassthroughCleaner: Cleaner {
public init() {}
public func clean(_ text: String, context: CleaningContext) async throws -> String {
text
}
}
```
`Sources/SstCore/TextInserter.swift`:
```swift
import Foundation
public typealias PasteboardItem = [String: Data]
public protocol PasteboardProviding {
func readItems() -> [PasteboardItem]
func writeString(_ text: String)
func writeItems(_ items: [PasteboardItem])
}
public protocol KeystrokePosting {
func postPaste() throws
}
public struct TextInserter: TextInserting {
private let pasteboard: PasteboardProviding
private let paster: KeystrokePosting
private let restoreDelay: Duration
public init(
pasteboard: PasteboardProviding,
paster: KeystrokePosting,
restoreDelay: Duration = .milliseconds(150)
) {
self.pasteboard = pasteboard
self.paster = paster
self.restoreDelay = restoreDelay
}
public func insert(_ text: String) async throws {
let saved = pasteboard.readItems()
pasteboard.writeString(text)
// On failure the text deliberately stays on the clipboard.
try paster.postPaste()
// Give the target app time to read the clipboard before restoring it.
try await Task.sleep(for: restoreDelay)
pasteboard.writeItems(saved)
}
}
```
`Sources/SstCore/SystemPasteboard.swift`:
```swift
import AppKit
import ApplicationServices
public struct SystemPasteboard: PasteboardProviding {
public init() {}
public func readItems() -> [PasteboardItem] {
(NSPasteboard.general.pasteboardItems ?? []).map { item in
var dict: PasteboardItem = [:]
for type in item.types {
if let data = item.data(forType: type) { dict[type.rawValue] = data }
}
return dict
}
}
public func writeString(_ text: String) {
let pasteboard = NSPasteboard.general
pasteboard.clearContents()
pasteboard.setString(text, forType: .string)
}
public func writeItems(_ items: [PasteboardItem]) {
let pasteboard = NSPasteboard.general
pasteboard.clearContents()
let restored = items.map { dict -> NSPasteboardItem in
let item = NSPasteboardItem()
for (type, data) in dict { item.setData(data, forType: NSPasteboard.PasteboardType(type)) }
return item
}
if !restored.isEmpty { pasteboard.writeObjects(restored) }
}
}
public struct CGEventPaster: KeystrokePosting {
public enum PasteError: Error, LocalizedError {
case accessibilityDenied
case eventCreationFailed
public var errorDescription: String? {
switch self {
case .accessibilityDenied:
"Geen Accessibility-rechten: tekst staat op het klembord, plak zelf met ⌘V."
case .eventCreationFailed:
"Kon het plak-event niet aanmaken: tekst staat op het klembord."
}
}
}
public init() {}
public func postPaste() throws {
guard AXIsProcessTrusted() else { throw PasteError.accessibilityDenied }
let source = CGEventSource(stateID: .combinedSessionState)
let vKey: CGKeyCode = 0x09
guard
let down = CGEvent(keyboardEventSource: source, virtualKey: vKey, keyDown: true),
let up = CGEvent(keyboardEventSource: source, virtualKey: vKey, keyDown: false)
else { throw PasteError.eventCreationFailed }
down.flags = .maskCommand
up.flags = .maskCommand
down.post(tap: .cgAnnotatedSessionEventTap)
up.post(tap: .cgAnnotatedSessionEventTap)
}
}
```
- [ ] **Step 4: Alle tests draaien**
Run: `swift test`
Expected: alle tests PASS (8 controller + 3 inserter/cleaner).
- [ ] **Step 5: Commit**
```bash
git add Sources Tests
git commit -m "feat: add TextInserter with clipboard restore and PassthroughCleaner" -m "Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"
```
---
### Task 3: Fn-hotkey (FnKeyTracker, HotkeyMonitor) en probe
**Files:**
- Create: `Sources/SstCore/FnKeyTracker.swift`, `Sources/SstCore/HotkeyMonitor.swift`
- Create: `Sources/SstProbe/main.swift`, `Sources/SstProbe/HotkeyProbe.swift`
- Modify: `Package.swift`
- Test: `Tests/SstCoreTests/FnKeyTrackerTests.swift`
**Interfaces:**
- Consumes: `HotkeyEvent` uit Task 1.
- Produces:
- `struct FnKeyTracker { init(); mutating func process(fnDown: Bool) -> HotkeyEvent? }`
- `final class HotkeyMonitor { init(); var onEvent: ((HotkeyEvent) -> Void)?; func start() throws; func stop() }`, `HotkeyMonitor.HotkeyError: Error, LocalizedError`
- Executable `SstProbe` met subcommando `hotkey`.
- [ ] **Step 1: Falende test schrijven**
`Tests/SstCoreTests/FnKeyTrackerTests.swift`:
```swift
import Testing
@testable import SstCore
struct FnKeyTrackerTests {
@Test func emitsPressedThenReleasedOnlyOnTransitions() {
var tracker = FnKeyTracker()
#expect(tracker.process(fnDown: false) == nil)
#expect(tracker.process(fnDown: true) == .pressed)
#expect(tracker.process(fnDown: true) == nil)
#expect(tracker.process(fnDown: false) == .released)
#expect(tracker.process(fnDown: false) == nil)
}
}
```
- [ ] **Step 2: Test draaien, moet falen**
Run: `swift test --filter FnKeyTrackerTests`
Expected: compilatiefout `cannot find 'FnKeyTracker' in scope`.
- [ ] **Step 3: Implementatie**
`Sources/SstCore/FnKeyTracker.swift`:
```swift
public struct FnKeyTracker {
private var isDown = false
public init() {}
/// Feed the current Fn flag state; returns an event only on a transition.
public mutating func process(fnDown: Bool) -> HotkeyEvent? {
guard fnDown != isDown else { return nil }
isDown = fnDown
return fnDown ? .pressed : .released
}
}
```
`Sources/SstCore/HotkeyMonitor.swift`:
```swift
import ApplicationServices
import CoreGraphics
public final class HotkeyMonitor {
public enum HotkeyError: Error, LocalizedError {
case tapUnavailable
public var errorDescription: String? {
"Kan de Fn-toets niet volgen: geef Input Monitoring-rechten in Systeeminstellingen en probeer opnieuw (soms is een herstart van de app nodig)."
}
}
public var onEvent: ((HotkeyEvent) -> Void)?
private var tracker = FnKeyTracker()
private var tap: CFMachPort?
private var source: CFRunLoopSource?
public init() {}
public func start() throws {
guard tap == nil else { return }
let mask = CGEventMask(1 << CGEventType.flagsChanged.rawValue)
let callback: CGEventTapCallBack = { _, type, event, refcon in
if let refcon {
let monitor = Unmanaged<HotkeyMonitor>.fromOpaque(refcon).takeUnretainedValue()
monitor.handle(type: type, event: event)
}
return Unmanaged.passUnretained(event)
}
guard let tap = CGEvent.tapCreate(
tap: .cgSessionEventTap,
place: .headInsertEventTap,
options: .listenOnly,
eventsOfInterest: mask,
callback: callback,
userInfo: Unmanaged.passUnretained(self).toOpaque()
) else { throw HotkeyError.tapUnavailable }
let source = CFMachPortCreateRunLoopSource(kCFAllocatorDefault, tap, 0)
CFRunLoopAddSource(CFRunLoopGetMain(), source, .commonModes)
CGEvent.tapEnable(tap: tap, enable: true)
self.tap = tap
self.source = source
}
public func stop() {
if let source { CFRunLoopRemoveSource(CFRunLoopGetMain(), source, .commonModes) }
if let tap { CGEvent.tapEnable(tap: tap, enable: false) }
tap = nil
source = nil
}
private func handle(type: CGEventType, event: CGEvent) {
switch type {
case .tapDisabledByTimeout, .tapDisabledByUserInput:
if let tap { CGEvent.tapEnable(tap: tap, enable: true) }
case .flagsChanged:
let fnDown = event.flags.contains(.maskSecondaryFn)
if let hotkeyEvent = tracker.process(fnDown: fnDown) {
onEvent?(hotkeyEvent)
}
default:
break
}
}
}
```
`Package.swift` (vervang volledig):
```swift
// swift-tools-version: 6.0
import PackageDescription
let package = Package(
name: "Sst",
platforms: [.macOS(.v14)],
targets: [
.target(name: "SstCore"),
.executableTarget(name: "SstProbe", dependencies: ["SstCore"]),
.testTarget(name: "SstCoreTests", dependencies: ["SstCore"]),
],
swiftLanguageModes: [.v5]
)
```
`Sources/SstProbe/main.swift`:
```swift
import Foundation
let command = CommandLine.arguments.dropFirst().first ?? "help"
switch command {
case "hotkey":
runHotkeyProbe()
default:
print("usage: sst-probe hotkey")
}
```
`Sources/SstProbe/HotkeyProbe.swift`:
```swift
import Foundation
import CoreGraphics
import SstCore
func runHotkeyProbe() -> Never {
if !CGPreflightListenEventAccess() { _ = CGRequestListenEventAccess() }
let monitor = HotkeyMonitor()
monitor.onEvent = { print("hotkey:", $0) }
do { try monitor.start() } catch {
print("failed:", error.localizedDescription)
exit(1)
}
print("Houd Fn ingedrukt en laat los. Ctrl-C om te stoppen.")
RunLoop.main.run()
exit(0)
}
```
- [ ] **Step 4: Tests draaien**
Run: `swift test`
Expected: alle tests PASS (incl. `FnKeyTrackerTests`).
- [ ] **Step 5: Handmatige check van de hotkey**
Zet eerst in Systeeminstellingen → Toetsenbord "Druk op 🌐 voor" op **Doe niets** (anders opent Fn de emoji-kiezer/dicteren).
Run: `swift run SstProbe hotkey`
Geef de terminal-app (waar je dit draait) Input Monitoring-rechten als daarom gevraagd wordt en start opnieuw.
Expected: bij Fn omlaag `hotkey: pressed`, bij loslaten `hotkey: released`, geen dubbele events.
- [ ] **Step 6: Commit**
```bash
git add Package.swift Sources Tests
git commit -m "feat: add Fn hotkey monitor and probe CLI" -m "Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"
```
---
### Task 4: AudioRecorder en record-probe
**Files:**
- Create: `Sources/SstCore/AudioRecorder.swift`, `Sources/SstProbe/RecordProbe.swift`
- Modify: `Sources/SstProbe/main.swift`
**Interfaces:**
- Consumes: `AudioRecording` uit Task 1.
- Produces:
- `final class AudioRecorder: AudioRecording`, `init()`, `AudioRecorder.RecorderError: Error, LocalizedError`
- Probe-helper `func recordSamples(seconds: Double) async -> [Float]?` (hergebruikt in Task 5)
- Probe-subcommando `record`
De AudioRecorder is hardware-afhankelijk en wordt daarom handmatig getest via de probe.
- [ ] **Step 1: Implementatie**
`Sources/SstCore/AudioRecorder.swift`:
```swift
import AVFoundation
public final class AudioRecorder: AudioRecording {
public enum RecorderError: Error, LocalizedError {
case converterUnavailable
public var errorDescription: String? {
"Kon de microfoon niet omzetten naar 16 kHz mono."
}
}
private let engine = AVAudioEngine()
private let lock = NSLock()
private var buffer: [Float] = []
private let targetFormat = AVAudioFormat(
commonFormat: .pcmFormatFloat32,
sampleRate: 16_000,
channels: 1,
interleaved: false
)!
public init() {}
public func start() throws {
lock.withLock { buffer.removeAll(keepingCapacity: true) }
let input = engine.inputNode
let inputFormat = input.outputFormat(forBus: 0)
guard let converter = AVAudioConverter(from: inputFormat, to: targetFormat) else {
throw RecorderError.converterUnavailable
}
let ratio = targetFormat.sampleRate / inputFormat.sampleRate
input.installTap(onBus: 0, bufferSize: 4096, format: inputFormat) { [weak self] pcm, _ in
guard let self else { return }
let capacity = AVAudioFrameCount(Double(pcm.frameLength) * ratio) + 1
guard let out = AVAudioPCMBuffer(pcmFormat: self.targetFormat, frameCapacity: capacity) else { return }
var consumed = false
var error: NSError?
converter.convert(to: out, error: &error) { _, status in
if consumed {
status.pointee = .noDataNow
return nil
}
consumed = true
status.pointee = .haveData
return pcm
}
guard error == nil, let channel = out.floatChannelData?[0] else { return }
let samples = Array(UnsafeBufferPointer(start: channel, count: Int(out.frameLength)))
self.lock.withLock { self.buffer.append(contentsOf: samples) }
}
engine.prepare()
try engine.start()
}
public func stop() -> [Float] {
engine.inputNode.removeTap(onBus: 0)
engine.stop()
return lock.withLock {
let samples = buffer
buffer = []
return samples
}
}
}
```
`Sources/SstProbe/RecordProbe.swift`:
```swift
import AVFoundation
import SstCore
/// Records `seconds` of microphone audio; returns nil if access is denied or recording fails.
func recordSamples(seconds: Double) async -> [Float]? {
guard await AVCaptureDevice.requestAccess(for: .audio) else {
print("Microfoon-toegang geweigerd (Systeeminstellingen → Privacy → Microfoon).")
return nil
}
let recorder = AudioRecorder()
do { try recorder.start() } catch {
print("start mislukt:", error.localizedDescription)
return nil
}
print("Neem \(Int(seconds)) s op — praat nu")
try? await Task.sleep(for: .seconds(seconds))
return recorder.stop()
}
func runRecordProbe() async {
guard let samples = await recordSamples(seconds: 3) else { return }
let peak = samples.map { abs($0) }.max() ?? 0
print("samples: \(samples.count) (verwacht ≈ 48000), piek: \(peak)")
}
```
`Sources/SstProbe/main.swift` (vervang volledig):
```swift
import Foundation
let command = CommandLine.arguments.dropFirst().first ?? "help"
switch command {
case "hotkey":
runHotkeyProbe()
case "record":
await runRecordProbe()
default:
print("usage: sst-probe hotkey|record")
}
```
- [ ] **Step 2: Bouwen**
Run: `swift build`
Expected: build slaagt zonder errors.
- [ ] **Step 3: Handmatige check**
Run: `swift run SstProbe record` en praat 3 seconden.
Expected: `samples:` ligt rond 48000 (±2000) en `piek` is duidelijk > 0,05. Bij stilte is de piek ≈ 0. Bij een geweigerde microfoon staat er een duidelijke melding.
- [ ] **Step 4: Commit**
```bash
git add Sources
git commit -m "feat: add AudioRecorder (16 kHz mono) and record probe" -m "Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"
```
---
### Task 5: WhisperKitTranscriber en transcribe-probe
**Files:**
- Create: `Sources/SstWhisper/WhisperKitTranscriber.swift`
- Create: `Sources/SstProbe/TranscribeProbe.swift`
- Modify: `Package.swift`, `Sources/SstProbe/main.swift`
**Interfaces:**
- Consumes: `Transcriber` uit Task 1, `recordSamples(seconds:)` uit Task 4.
- Produces:
- `final class WhisperKitTranscriber: Transcriber` met `init(model: String = "large-v3-v20240930_turbo")`, `func load() async throws`, `WhisperKitTranscriber.TranscriberError: Error, LocalizedError`
- Probe-subcommando `transcribe`
- [ ] **Step 1: Dependency en target toevoegen**
`Package.swift` (vervang volledig):
```swift
// swift-tools-version: 6.0
import PackageDescription
let package = Package(
name: "Sst",
platforms: [.macOS(.v14)],
dependencies: [
.package(url: "https://github.com/argmaxinc/WhisperKit.git", from: "0.9.0"),
],
targets: [
.target(name: "SstCore"),
.target(
name: "SstWhisper",
dependencies: [
"SstCore",
.product(name: "WhisperKit", package: "WhisperKit"),
]
),
.executableTarget(name: "SstProbe", dependencies: ["SstCore", "SstWhisper"]),
.testTarget(name: "SstCoreTests", dependencies: ["SstCore"]),
],
swiftLanguageModes: [.v5]
)
```
- [ ] **Step 2: Implementatie**
`Sources/SstWhisper/WhisperKitTranscriber.swift`:
```swift
import Foundation
import SstCore
import WhisperKit
public final class WhisperKitTranscriber: Transcriber {
public enum TranscriberError: Error, LocalizedError {
case notLoaded
public var errorDescription: String? { "Spraakmodel is nog niet geladen." }
}
private let modelName: String
private var pipeline: WhisperKit?
public init(model: String = "large-v3-v20240930_turbo") {
self.modelName = model
}
/// Downloads the model on first run (≈1.5 GB) and loads it into memory.
public func load() async throws {
pipeline = try await WhisperKit(WhisperKitConfig(model: modelName))
}
public func transcribe(_ samples: [Float]) async throws -> String {
guard let pipeline else { throw TranscriberError.notLoaded }
// language nil + detectLanguage: Whisper picks NL or EN per utterance.
let options = DecodingOptions(
task: .transcribe,
language: nil,
detectLanguage: true,
skipSpecialTokens: true,
withoutTimestamps: true
)
let results = try await pipeline.transcribe(audioArray: samples, decodeOptions: options)
return results
.map(\.text)
.joined(separator: " ")
.trimmingCharacters(in: .whitespacesAndNewlines)
}
}
```
> Als de WhisperKit-API of het modelnaam-patroon in de opgeloste versie afwijkt (compilatiefout of "model not found"), pas dan alleen dit bestand aan. Beschikbare modelnamen: `WhisperKit.fetchAvailableModels()`. Houd `Transcriber` en `load()` gelijk, want Task 6 hangt daarvan af.
`Sources/SstProbe/TranscribeProbe.swift`:
```swift
import Foundation
import SstWhisper
func runTranscribeProbe() async {
guard let samples = await recordSamples(seconds: 6) else { return }
print("Model laden (eerste keer: download ≈1.5 GB)…")
let transcriber = WhisperKitTranscriber()
do {
try await transcriber.load()
let started = Date.now
let text = try await transcriber.transcribe(samples)
let elapsed = Date.now.timeIntervalSince(started)
print(String(format: "(%.2f s) ", elapsed) + text)
} catch {
print("mislukt:", error.localizedDescription)
}
}
```
`Sources/SstProbe/main.swift` (vervang volledig):
```swift
import Foundation
let command = CommandLine.arguments.dropFirst().first ?? "help"
switch command {
case "hotkey":
runHotkeyProbe()
case "record":
await runRecordProbe()
case "transcribe":
await runTranscribeProbe()
default:
print("usage: sst-probe hotkey|record|transcribe")
}
```
- [ ] **Step 3: Bouwen**
Run: `swift build`
Expected: dependencies worden opgehaald, build slaagt. Los API-afwijkingen op in `WhisperKitTranscriber.swift` (zie opmerking hierboven).
- [ ] **Step 4: Handmatige check (NL, EN, gemengd)**
Run: `swift run SstProbe transcribe`
Zeg achtereenvolgens: een Nederlandse zin, een Engelse zin, en een zin die begint in het Nederlands met een paar Engelse woorden.
Expected: correcte transcriptie van de zinnen; na het laden duurt transcriberen van 6 s audio ruim onder 3 s op de M4 Pro. Noteer de gemeten tijd.
Bekende beperking: taaldetectie is per opname, dus wisselen midden in een zin kan een van beide talen minder goed herkennen. Dat is acceptabel voor de skeleton.
- [ ] **Step 5: Commit**
```bash
git add Package.swift Package.resolved Sources
git commit -m "feat: add WhisperKit transcriber and transcribe probe" -m "Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"
```
---
### Task 6: Menubalk-app, app-bundle en rooktest
**Files:**
- Create: `Sources/Sst/SstApp.swift`, `Sources/Sst/AppModel.swift`, `Sources/Sst/MenuContent.swift`
- Create: `scripts/make-app.sh`
- Modify: `Package.swift`
**Interfaces:**
- Consumes: `DictationController`, `HotkeyMonitor`, `AudioRecorder`, `TextInserter`, `SystemPasteboard`, `CGEventPaster`, `PassthroughCleaner` (SstCore); `WhisperKitTranscriber` met `load()` (SstWhisper).
- Produces: de app-bundle `build/Sst.app`.
- [ ] **Step 1: Executable target toevoegen**
In `Package.swift`: voeg de regel `.executableTarget(name: "Sst", dependencies: ["SstCore", "SstWhisper"]),` toe aan `targets`, direct boven de `SstProbe`-regel.
- [ ] **Step 2: App-code**
`Sources/Sst/AppModel.swift`:
```swift
import AVFoundation
import ApplicationServices
import SwiftUI
import SstCore
import SstWhisper
@MainActor
final class AppModel: ObservableObject {
@Published var state: DictationState = .idle
@Published var modelStatus = "Model laden…"
@Published var modelReady = false
@Published var lastError: String?
@Published var micGranted = false
@Published var accessibilityGranted = false
@Published var inputMonitoringGranted = false
private let transcriber = WhisperKitTranscriber()
private let hotkey = HotkeyMonitor()
private let controller: DictationController
init() {
controller = DictationController(
recorder: AudioRecorder(),
transcriber: transcriber,
cleaner: PassthroughCleaner(),
inserter: TextInserter(pasteboard: SystemPasteboard(), paster: CGEventPaster())
)
controller.onStateChange = { [weak self] in self?.state = $0 }
controller.onError = { [weak self] in self?.lastError = $0 }
hotkey.onEvent = { [weak self] event in
guard let self, self.modelReady else { return }
Task { await self.controller.handle(event) }
}
Task { await start() }
}
var statusLine: String {
guard modelReady else { return modelStatus }
switch state {
case .idle: return "Klaar — houd Fn ingedrukt om te dicteren"
case .recording: return "Luistert…"
case .transcribing: return "Transcriberen…"
case .inserting: return "Invoegen…"
}
}
var iconName: String {
switch state {
case .idle: return modelReady ? "mic" : "mic.slash"
case .recording: return "mic.fill"
case .transcribing, .inserting: return "waveform"
}
}
func refreshPermissions() {
micGranted = AVCaptureDevice.authorizationStatus(for: .audio) == .authorized
accessibilityGranted = AXIsProcessTrusted()
inputMonitoringGranted = CGPreflightListenEventAccess()
}
func retryHotkey() {
refreshPermissions()
do {
try hotkey.start()
lastError = nil
} catch {
lastError = error.localizedDescription
}
}
private func start() async {
_ = await AVCaptureDevice.requestAccess(for: .audio)
if !CGPreflightListenEventAccess() { _ = CGRequestListenEventAccess() }
if !AXIsProcessTrusted() {
_ = AXIsProcessTrustedWithOptions(["AXTrustedCheckOptionPrompt": true] as CFDictionary)
}
retryHotkey()
do {
try await transcriber.load()
modelReady = true
modelStatus = "Klaar"
} catch {
modelStatus = "Model laden mislukt"
lastError = error.localizedDescription
}
}
}
```
`Sources/Sst/MenuContent.swift`:
```swift
import AppKit
import SwiftUI
struct MenuContent: View {
@ObservedObject var model: AppModel
var body: some View {
Text(model.statusLine)
if let error = model.lastError {
Text("Fout: \(error)")
}
Divider()
permissionRow("Microfoon", granted: model.micGranted, pane: "Privacy_Microphone")
permissionRow("Accessibility", granted: model.accessibilityGranted, pane: "Privacy_Accessibility")
permissionRow("Input Monitoring", granted: model.inputMonitoringGranted, pane: "Privacy_ListenEvent")
Button("Rechten opnieuw controleren") { model.retryHotkey() }
Divider()
Button("Stop") { NSApplication.shared.terminate(nil) }
.keyboardShortcut("q")
}
@ViewBuilder
private func permissionRow(_ title: String, granted: Bool, pane: String) -> some View {
if granted {
Text("✓ \(title)")
} else {
Button("⚠︎ \(title) — open Instellingen") {
if let url = URL(string: "x-apple.systempreferences:com.apple.preference.security?\(pane)") {
NSWorkspace.shared.open(url)
}
}
}
}
}
```
`Sources/Sst/SstApp.swift`:
```swift
import SwiftUI
@main
struct SstApp: App {
@StateObject private var model = AppModel()
var body: some Scene {
MenuBarExtra {
MenuContent(model: model)
} label: {
Image(systemName: model.iconName)
}
}
}
```
> Als de model-start pas begint na het openen van het menu (StateObject wordt lui aangemaakt), verplaats dan `AppModel` naar een `NSApplicationDelegateAdaptor`-delegate die het model in `applicationDidFinishLaunching` aanmaakt.
- [ ] **Step 3: Bundle-script**
`scripts/make-app.sh`:
```bash
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
swift build -c release --product Sst
BIN=".build/release/Sst"
APP="build/Sst.app"
rm -rf "$APP"
mkdir -p "$APP/Contents/MacOS" "$APP/Contents/Resources"
cp "$BIN" "$APP/Contents/MacOS/Sst"
# SwiftPM resource bundles (if any dependency ships them) live next to the binary.
for b in .build/release/*.bundle; do
[ -e "$b" ] && cp -R "$b" "$APP/Contents/Resources/"
done
cat > "$APP/Contents/Info.plist" <<'PLIST'
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>CFBundleIdentifier</key><string>local.sst.app</string>
<key>CFBundleName</key><string>Sst</string>
<key>CFBundleExecutable</key><string>Sst</string>
<key>CFBundlePackageType</key><string>APPL</string>
<key>CFBundleShortVersionString</key><string>0.1.0</string>
<key>CFBundleVersion</key><string>1</string>
<key>LSMinimumSystemVersion</key><string>14.0</string>
<key>LSUIElement</key><true/>
<key>NSMicrophoneUsageDescription</key>
<string>Sst neemt je stem op zolang je Fn ingedrukt houdt, om die lokaal naar tekst om te zetten.</string>
</dict>
</plist>
PLIST
codesign --force --sign - --identifier local.sst.app "$APP"
echo "Gebouwd: $APP"
```
- [ ] **Step 4: Bouwen**
Run: `chmod +x scripts/make-app.sh && ./scripts/make-app.sh`
Expected: `Gebouwd: build/Sst.app`, zonder errors.
- [ ] **Step 5: Rooktest van de hele keten**
Run: `open build/Sst.app`
Controleer in deze volgorde:
1. Mic-icoon verschijnt in de menubalk (geen Dock-icoon). Menu toont "Model laden…" en daarna "Klaar — houd Fn ingedrukt om te dicteren" (eerste keer duurt de download enkele minuten).
2. Geef Microfoon, Accessibility en Input Monitoring toe via de prompts of de knoppen in het menu; klik "Rechten opnieuw controleren" (zo nodig de app herstarten). Alle drie staan op ✓.
3. Open TextEdit, klik in een leeg document, houd Fn ingedrukt, zeg "Dit is een test van de dictatie", laat los. Icoon: `mic.fill` → `waveform` → `mic`. Tekst verschijnt op de cursor.
4. Herhaal in Notes en in een browser-tekstveld, met een Engelse en een gemengde zin.
5. Zet een eigen tekst op het klembord, dicteer, en controleer dat het klembord daarna weer je eigen tekst bevat.
6. Tik Fn kort aan (< 0,3 s): er mag niets ingevoegd worden.
7. Weiger Accessibility tijdelijk: het menu toont een foutmelding en de tekst staat op het klembord.
Bekende beperking: omdat de app ad-hoc is gesigneerd (`codesign --sign -`), kan macOS de rechten na elke herbouw opnieuw vragen. Een vast ontwikkelcertificaat lost dat later op.
- [ ] **Step 6: Commit**
```bash
git add Package.swift Sources scripts
git commit -m "feat: add menu bar app and bundle script" -m "Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"
```
---
## Self-review
- **Spec-dekking:** HotkeyMonitor (Task 3), AudioRecorder (4), Transcriber/WhisperKit (5), Cleaner-naad (2), TextInserter met klembordherstel en klembord-bij-falen (2), DictationController met statemachine, minimale opnameduur, lege transcriptie en foutpaden (1), MenuBarExtra met rechten-check en modelstatus (6). Niet gedekt, bewust: overlay-pill, cleanup, branding (latere specs).
- **Placeholders:** geen; alle codestappen bevatten volledige code. De twee opmerkingen over WhisperKit-API en StateObject zijn gerichte fallbacks, geen open eindjes.
- **Typeconsistentie:** `HotkeyEvent`, `DictationState`, `CleaningContext`, de vier protocollen en `DictationController.init(recorder:transcriber:cleaner:inserter:minimumSamples:)` zijn in Task 1 gedefinieerd en komen identiek terug in Task 2 t/m 6. `WhisperKitTranscriber.load()` uit Task 5 wordt in Task 6 gebruikt.
- **Risico's:** de WhisperKit-API en het exacte modelnaam-patroon zijn het enige onverifieerde punt; Task 5 stap 3 vangt dat vroeg af.