1400 lines
44 KiB
Markdown
1400 lines
44 KiB
Markdown
# Dictatie-skeleton Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Een werkende macOS-menubalk-app: Fn ingedrukt houden neemt op, loslaten transcribeert lokaal (NL/EN) en plakt de tekst op de cursor.
|
|
|
|
**Architecture:** Een Swift Package. `SstCore` bevat de logica achter protocollen (testbaar met fakes), `SstWhisper` bevat de WhisperKit-implementatie, `Sst` is de SwiftUI-menubalk-app en `SstProbe` is een CLI om hotkey, opname en transcriptie los te testen. Een script bouwt de package tot een `.app`-bundle.
|
|
|
|
**Tech Stack:** Swift 6.4 (language mode 5), SwiftUI `MenuBarExtra`, `AVAudioEngine`, `CGEventTap`, WhisperKit, Swift Testing.
|
|
|
|
**Spec:** `docs/superpowers/specs/2026-10-01-push-to-talk-dictation-design.md`
|
|
|
|
## Global Constraints
|
|
|
|
- Alleen Apple Silicon, macOS 14+ als deployment target (doelmachine: M4 Pro, macOS 27, Xcode 27).
|
|
- Alles lokaal: geen netwerkverkeer behalve de eenmalige modeldownload; audio wordt nooit opgeslagen.
|
|
- Audioformaat naar de transcriber: 16 kHz, mono, Float32.
|
|
- Opnames korter dan 0,3 s (4800 samples) worden genegeerd.
|
|
- Model: Whisper `large-v3-turbo` via WhisperKit, automatische taalherkenning (Nederlands en Engels).
|
|
- Niet-gesandboxed; vereiste rechten: Microfoon, Accessibility, Input Monitoring.
|
|
- Statemachine: `idle → recording → transcribing → inserting → idle`; fouten gaan naar `onError` en eindigen in `idle`.
|
|
- Falend invoegen laat de tekst op het klembord staan.
|
|
- Elke commit eindigt met de trailer `Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>`.
|
|
|
|
**Afwijkingen van de spec (bewust):**
|
|
- De "dunne Xcode-app" is een app-bundle die `scripts/make-app.sh` uit de package assembleert. Een handgeschreven `.xcodeproj` is fragiel; Xcode kan `Package.swift` rechtstreeks openen.
|
|
- Swift language mode 5, om strikte Sendable-regels buiten de skeleton-scope te houden.
|
|
- De overlay-pill (bouwvolgorde-stap 5) valt buiten dit plan; de menubalk-icoon toont de status.
|
|
|
|
## Bestandsstructuur
|
|
|
|
```
|
|
Package.swift
|
|
.gitignore
|
|
scripts/make-app.sh
|
|
Sources/SstCore/Models.swift HotkeyEvent, DictationState, CleaningContext, protocollen
|
|
Sources/SstCore/DictationController.swift statemachine
|
|
Sources/SstCore/PassthroughCleaner.swift
|
|
Sources/SstCore/TextInserter.swift TextInserter + PasteboardProviding/KeystrokePosting
|
|
Sources/SstCore/SystemPasteboard.swift NSPasteboard + CGEvent-implementaties
|
|
Sources/SstCore/FnKeyTracker.swift pure Fn-logica
|
|
Sources/SstCore/HotkeyMonitor.swift CGEventTap
|
|
Sources/SstCore/AudioRecorder.swift AVAudioEngine → 16 kHz mono
|
|
Sources/SstWhisper/WhisperKitTranscriber.swift
|
|
Sources/SstProbe/main.swift, HotkeyProbe.swift, RecordProbe.swift
|
|
Sources/Sst/SstApp.swift, AppModel.swift, MenuContent.swift
|
|
Tests/SstCoreTests/Fakes.swift, DictationControllerTests.swift,
|
|
TextInserterTests.swift, FnKeyTrackerTests.swift
|
|
```
|
|
|
|
---
|
|
|
|
### Task 1: Package-scaffold en DictationController
|
|
|
|
**Files:**
|
|
- Create: `Package.swift`, `.gitignore`
|
|
- Create: `Sources/SstCore/Models.swift`, `Sources/SstCore/DictationController.swift`
|
|
- Test: `Tests/SstCoreTests/Fakes.swift`, `Tests/SstCoreTests/DictationControllerTests.swift`
|
|
|
|
**Interfaces:**
|
|
- Produces (gebruikt door alle latere taken):
|
|
- `enum HotkeyEvent { case pressed, released }`
|
|
- `enum DictationState: Equatable { case idle, recording, transcribing, inserting }`
|
|
- `struct CleaningContext { init() }`
|
|
- `protocol AudioRecording { func start() throws; func stop() -> [Float] }`
|
|
- `protocol Transcriber { func transcribe(_ samples: [Float]) async throws -> String }`
|
|
- `protocol Cleaner { func clean(_ text: String, context: CleaningContext) async throws -> String }`
|
|
- `protocol TextInserting { func insert(_ text: String) async throws }`
|
|
- `@MainActor final class DictationController` met `init(recorder:transcriber:cleaner:inserter:minimumSamples:)`, `var onStateChange: ((DictationState) -> Void)?`, `var onError: ((String) -> Void)?`, `func handle(_ event: HotkeyEvent) async`
|
|
|
|
- [ ] **Step 1: Scaffold maken**
|
|
|
|
`Package.swift`:
|
|
|
|
```swift
|
|
// swift-tools-version: 6.0
|
|
import PackageDescription
|
|
|
|
let package = Package(
|
|
name: "Sst",
|
|
platforms: [.macOS(.v14)],
|
|
targets: [
|
|
.target(name: "SstCore"),
|
|
.testTarget(name: "SstCoreTests", dependencies: ["SstCore"]),
|
|
],
|
|
swiftLanguageModes: [.v5]
|
|
)
|
|
```
|
|
|
|
`.gitignore`:
|
|
|
|
```
|
|
.build/
|
|
build/
|
|
.DS_Store
|
|
*.xcodeproj
|
|
```
|
|
|
|
`Sources/SstCore/Models.swift`:
|
|
|
|
```swift
|
|
import Foundation
|
|
|
|
public enum HotkeyEvent: Equatable, Sendable {
|
|
case pressed
|
|
case released
|
|
}
|
|
|
|
public enum DictationState: Equatable, Sendable {
|
|
case idle
|
|
case recording
|
|
case transcribing
|
|
case inserting
|
|
}
|
|
|
|
public struct CleaningContext: Sendable {
|
|
public init() {}
|
|
}
|
|
|
|
public protocol AudioRecording {
|
|
func start() throws
|
|
/// Stops recording and returns all captured 16 kHz mono Float samples.
|
|
func stop() -> [Float]
|
|
}
|
|
|
|
public protocol Transcriber {
|
|
func transcribe(_ samples: [Float]) async throws -> String
|
|
}
|
|
|
|
public protocol Cleaner {
|
|
func clean(_ text: String, context: CleaningContext) async throws -> String
|
|
}
|
|
|
|
public protocol TextInserting {
|
|
func insert(_ text: String) async throws
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Fakes en falende tests schrijven**
|
|
|
|
`Tests/SstCoreTests/Fakes.swift`:
|
|
|
|
```swift
|
|
import Foundation
|
|
@testable import SstCore
|
|
|
|
struct TestError: Error {}
|
|
|
|
final class FakeRecorder: AudioRecording {
|
|
var samples: [Float] = []
|
|
var startError: Error?
|
|
private(set) var startCount = 0
|
|
|
|
func start() throws {
|
|
startCount += 1
|
|
if let startError { throw startError }
|
|
}
|
|
|
|
func stop() -> [Float] { samples }
|
|
}
|
|
|
|
final class FakeTranscriber: Transcriber {
|
|
var result: Result<String, Error> = .success("hallo wereld")
|
|
private(set) var callCount = 0
|
|
|
|
func transcribe(_ samples: [Float]) async throws -> String {
|
|
callCount += 1
|
|
return try result.get()
|
|
}
|
|
}
|
|
|
|
final class FakeCleaner: Cleaner {
|
|
func clean(_ text: String, context: CleaningContext) async throws -> String {
|
|
"[\(text)]"
|
|
}
|
|
}
|
|
|
|
final class FakeInserter: TextInserting {
|
|
var error: Error?
|
|
private(set) var inserted: [String] = []
|
|
|
|
func insert(_ text: String) async throws {
|
|
if let error { throw error }
|
|
inserted.append(text)
|
|
}
|
|
}
|
|
```
|
|
|
|
`Tests/SstCoreTests/DictationControllerTests.swift`:
|
|
|
|
```swift
|
|
import Testing
|
|
@testable import SstCore
|
|
|
|
@MainActor
|
|
struct DictationControllerTests {
|
|
let recorder = FakeRecorder()
|
|
let transcriber = FakeTranscriber()
|
|
let cleaner = FakeCleaner()
|
|
let inserter = FakeInserter()
|
|
static let oneSecond = [Float](repeating: 0.1, count: 16_000)
|
|
|
|
func makeController() -> DictationController {
|
|
DictationController(
|
|
recorder: recorder,
|
|
transcriber: transcriber,
|
|
cleaner: cleaner,
|
|
inserter: inserter
|
|
)
|
|
}
|
|
|
|
@Test func pressAndReleaseTranscribesCleansAndInserts() async {
|
|
recorder.samples = Self.oneSecond
|
|
let controller = makeController()
|
|
var states: [DictationState] = []
|
|
controller.onStateChange = { states.append($0) }
|
|
|
|
await controller.handle(.pressed)
|
|
await controller.handle(.released)
|
|
|
|
#expect(states == [.recording, .transcribing, .inserting, .idle])
|
|
#expect(inserter.inserted == ["[hallo wereld]"])
|
|
}
|
|
|
|
@Test func shortRecordingIsDiscarded() async {
|
|
recorder.samples = [Float](repeating: 0.1, count: 1_000)
|
|
let controller = makeController()
|
|
var states: [DictationState] = []
|
|
controller.onStateChange = { states.append($0) }
|
|
|
|
await controller.handle(.pressed)
|
|
await controller.handle(.released)
|
|
|
|
#expect(states == [.recording, .idle])
|
|
#expect(transcriber.callCount == 0)
|
|
#expect(inserter.inserted.isEmpty)
|
|
}
|
|
|
|
@Test func emptyTranscriptionIsNotInserted() async {
|
|
recorder.samples = Self.oneSecond
|
|
transcriber.result = .success(" \n")
|
|
let controller = makeController()
|
|
var errors: [String] = []
|
|
controller.onError = { errors.append($0) }
|
|
|
|
await controller.handle(.pressed)
|
|
await controller.handle(.released)
|
|
|
|
#expect(inserter.inserted.isEmpty)
|
|
#expect(errors.isEmpty)
|
|
#expect(controller.state == .idle)
|
|
}
|
|
|
|
@Test func transcriberErrorReportsAndReturnsToIdle() async {
|
|
recorder.samples = Self.oneSecond
|
|
transcriber.result = .failure(TestError())
|
|
let controller = makeController()
|
|
var errors: [String] = []
|
|
controller.onError = { errors.append($0) }
|
|
|
|
await controller.handle(.pressed)
|
|
await controller.handle(.released)
|
|
|
|
#expect(errors.count == 1)
|
|
#expect(inserter.inserted.isEmpty)
|
|
#expect(controller.state == .idle)
|
|
}
|
|
|
|
@Test func inserterErrorReportsAndReturnsToIdle() async {
|
|
recorder.samples = Self.oneSecond
|
|
inserter.error = TestError()
|
|
let controller = makeController()
|
|
var errors: [String] = []
|
|
controller.onError = { errors.append($0) }
|
|
|
|
await controller.handle(.pressed)
|
|
await controller.handle(.released)
|
|
|
|
#expect(errors.count == 1)
|
|
#expect(controller.state == .idle)
|
|
}
|
|
|
|
@Test func recorderStartFailureStaysIdle() async {
|
|
recorder.startError = TestError()
|
|
let controller = makeController()
|
|
var errors: [String] = []
|
|
controller.onError = { errors.append($0) }
|
|
|
|
await controller.handle(.pressed)
|
|
|
|
#expect(errors.count == 1)
|
|
#expect(controller.state == .idle)
|
|
}
|
|
|
|
@Test func secondPressWhileRecordingIsIgnored() async {
|
|
let controller = makeController()
|
|
await controller.handle(.pressed)
|
|
await controller.handle(.pressed)
|
|
#expect(recorder.startCount == 1)
|
|
}
|
|
|
|
@Test func releaseWithoutPressIsIgnored() async {
|
|
let controller = makeController()
|
|
var states: [DictationState] = []
|
|
controller.onStateChange = { states.append($0) }
|
|
|
|
await controller.handle(.released)
|
|
|
|
#expect(states.isEmpty)
|
|
#expect(transcriber.callCount == 0)
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 3: Test draaien, moet falen**
|
|
|
|
Run: `swift test --filter DictationControllerTests`
|
|
Expected: compilatiefout `cannot find 'DictationController' in scope`.
|
|
|
|
- [ ] **Step 4: Minimale implementatie**
|
|
|
|
`Sources/SstCore/DictationController.swift`:
|
|
|
|
```swift
|
|
import Foundation
|
|
|
|
@MainActor
|
|
public final class DictationController {
|
|
public private(set) var state: DictationState = .idle {
|
|
didSet { onStateChange?(state) }
|
|
}
|
|
public var onStateChange: ((DictationState) -> Void)?
|
|
public var onError: ((String) -> Void)?
|
|
|
|
private let recorder: AudioRecording
|
|
private let transcriber: Transcriber
|
|
private let cleaner: Cleaner
|
|
private let inserter: TextInserting
|
|
private let minimumSamples: Int
|
|
|
|
/// - Parameter minimumSamples: 16 kHz samples; default is 0.3 s.
|
|
public init(
|
|
recorder: AudioRecording,
|
|
transcriber: Transcriber,
|
|
cleaner: Cleaner,
|
|
inserter: TextInserting,
|
|
minimumSamples: Int = 4_800
|
|
) {
|
|
self.recorder = recorder
|
|
self.transcriber = transcriber
|
|
self.cleaner = cleaner
|
|
self.inserter = inserter
|
|
self.minimumSamples = minimumSamples
|
|
}
|
|
|
|
public func handle(_ event: HotkeyEvent) async {
|
|
switch (event, state) {
|
|
case (.pressed, .idle):
|
|
do {
|
|
try recorder.start()
|
|
state = .recording
|
|
} catch {
|
|
onError?(error.localizedDescription)
|
|
}
|
|
case (.released, .recording):
|
|
let samples = recorder.stop()
|
|
guard samples.count >= minimumSamples else {
|
|
state = .idle
|
|
return
|
|
}
|
|
await process(samples)
|
|
default:
|
|
break
|
|
}
|
|
}
|
|
|
|
private func process(_ samples: [Float]) async {
|
|
defer { state = .idle }
|
|
do {
|
|
state = .transcribing
|
|
let raw = try await transcriber.transcribe(samples)
|
|
let text = raw.trimmingCharacters(in: .whitespacesAndNewlines)
|
|
guard !text.isEmpty else { return }
|
|
let cleaned = try await cleaner.clean(text, context: CleaningContext())
|
|
state = .inserting
|
|
try await inserter.insert(cleaned)
|
|
} catch {
|
|
onError?(error.localizedDescription)
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 5: Test draaien, moet slagen**
|
|
|
|
Run: `swift test --filter DictationControllerTests`
|
|
Expected: 8 tests PASS.
|
|
|
|
- [ ] **Step 6: Commit**
|
|
|
|
```bash
|
|
git add Package.swift .gitignore Sources Tests
|
|
git commit -m "feat: add package scaffold and DictationController state machine" -m "Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 2: PassthroughCleaner en TextInserter
|
|
|
|
**Files:**
|
|
- Create: `Sources/SstCore/PassthroughCleaner.swift`, `Sources/SstCore/TextInserter.swift`, `Sources/SstCore/SystemPasteboard.swift`
|
|
- Test: `Tests/SstCoreTests/TextInserterTests.swift`
|
|
|
|
**Interfaces:**
|
|
- Consumes: `TextInserting`, `Cleaner`, `CleaningContext` uit Task 1.
|
|
- Produces:
|
|
- `typealias PasteboardItem = [String: Data]`
|
|
- `protocol PasteboardProviding { func readItems() -> [PasteboardItem]; func writeString(_ text: String); func writeItems(_ items: [PasteboardItem]) }`
|
|
- `protocol KeystrokePosting { func postPaste() throws }`
|
|
- `struct TextInserter: TextInserting`, `init(pasteboard:paster:restoreDelay: Duration = .milliseconds(150))`
|
|
- `struct PassthroughCleaner: Cleaner`
|
|
- `struct SystemPasteboard: PasteboardProviding`, `struct CGEventPaster: KeystrokePosting` (init zonder argumenten)
|
|
|
|
- [ ] **Step 1: Falende tests schrijven**
|
|
|
|
`Tests/SstCoreTests/TextInserterTests.swift`:
|
|
|
|
```swift
|
|
import Foundation
|
|
import Testing
|
|
@testable import SstCore
|
|
|
|
final class FakePasteboard: PasteboardProviding {
|
|
var items: [PasteboardItem] = []
|
|
|
|
func readItems() -> [PasteboardItem] { items }
|
|
func writeString(_ text: String) {
|
|
items = [["public.utf8-plain-text": Data(text.utf8)]]
|
|
}
|
|
func writeItems(_ items: [PasteboardItem]) { self.items = items }
|
|
}
|
|
|
|
final class FakePaster: KeystrokePosting {
|
|
var shouldFail = false
|
|
var onPaste: (() -> Void)?
|
|
private(set) var pasteCount = 0
|
|
|
|
func postPaste() throws {
|
|
pasteCount += 1
|
|
if shouldFail { throw TestError() }
|
|
onPaste?()
|
|
}
|
|
}
|
|
|
|
struct TextInserterTests {
|
|
let pasteboard = FakePasteboard()
|
|
let paster = FakePaster()
|
|
|
|
func makeInserter() -> TextInserter {
|
|
TextInserter(pasteboard: pasteboard, paster: paster, restoreDelay: .zero)
|
|
}
|
|
|
|
@Test func pastesTextThenRestoresPreviousClipboard() async throws {
|
|
let original: [PasteboardItem] = [["public.utf8-plain-text": Data("oud".utf8)]]
|
|
pasteboard.items = original
|
|
var seenAtPaste: Data?
|
|
paster.onPaste = { seenAtPaste = pasteboard.items.first?["public.utf8-plain-text"] }
|
|
|
|
try await makeInserter().insert("hallo")
|
|
|
|
#expect(seenAtPaste == Data("hallo".utf8))
|
|
#expect(paster.pasteCount == 1)
|
|
#expect(pasteboard.items == original)
|
|
}
|
|
|
|
@Test func failedPasteLeavesTextOnClipboard() async {
|
|
pasteboard.items = [["public.utf8-plain-text": Data("oud".utf8)]]
|
|
paster.shouldFail = true
|
|
|
|
await #expect(throws: TestError.self) {
|
|
try await makeInserter().insert("hallo")
|
|
}
|
|
|
|
#expect(pasteboard.items.first?["public.utf8-plain-text"] == Data("hallo".utf8))
|
|
}
|
|
|
|
@Test func passthroughCleanerReturnsTextUnchanged() async throws {
|
|
let result = try await PassthroughCleaner().clean("hoi", context: CleaningContext())
|
|
#expect(result == "hoi")
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Test draaien, moet falen**
|
|
|
|
Run: `swift test --filter TextInserterTests`
|
|
Expected: compilatiefout `cannot find type 'PasteboardProviding' in scope`.
|
|
|
|
- [ ] **Step 3: Implementatie**
|
|
|
|
`Sources/SstCore/PassthroughCleaner.swift`:
|
|
|
|
```swift
|
|
public struct PassthroughCleaner: Cleaner {
|
|
public init() {}
|
|
|
|
public func clean(_ text: String, context: CleaningContext) async throws -> String {
|
|
text
|
|
}
|
|
}
|
|
```
|
|
|
|
`Sources/SstCore/TextInserter.swift`:
|
|
|
|
```swift
|
|
import Foundation
|
|
|
|
public typealias PasteboardItem = [String: Data]
|
|
|
|
public protocol PasteboardProviding {
|
|
func readItems() -> [PasteboardItem]
|
|
func writeString(_ text: String)
|
|
func writeItems(_ items: [PasteboardItem])
|
|
}
|
|
|
|
public protocol KeystrokePosting {
|
|
func postPaste() throws
|
|
}
|
|
|
|
public struct TextInserter: TextInserting {
|
|
private let pasteboard: PasteboardProviding
|
|
private let paster: KeystrokePosting
|
|
private let restoreDelay: Duration
|
|
|
|
public init(
|
|
pasteboard: PasteboardProviding,
|
|
paster: KeystrokePosting,
|
|
restoreDelay: Duration = .milliseconds(150)
|
|
) {
|
|
self.pasteboard = pasteboard
|
|
self.paster = paster
|
|
self.restoreDelay = restoreDelay
|
|
}
|
|
|
|
public func insert(_ text: String) async throws {
|
|
let saved = pasteboard.readItems()
|
|
pasteboard.writeString(text)
|
|
// On failure the text deliberately stays on the clipboard.
|
|
try paster.postPaste()
|
|
// Give the target app time to read the clipboard before restoring it.
|
|
try await Task.sleep(for: restoreDelay)
|
|
pasteboard.writeItems(saved)
|
|
}
|
|
}
|
|
```
|
|
|
|
`Sources/SstCore/SystemPasteboard.swift`:
|
|
|
|
```swift
|
|
import AppKit
|
|
import ApplicationServices
|
|
|
|
public struct SystemPasteboard: PasteboardProviding {
|
|
public init() {}
|
|
|
|
public func readItems() -> [PasteboardItem] {
|
|
(NSPasteboard.general.pasteboardItems ?? []).map { item in
|
|
var dict: PasteboardItem = [:]
|
|
for type in item.types {
|
|
if let data = item.data(forType: type) { dict[type.rawValue] = data }
|
|
}
|
|
return dict
|
|
}
|
|
}
|
|
|
|
public func writeString(_ text: String) {
|
|
let pasteboard = NSPasteboard.general
|
|
pasteboard.clearContents()
|
|
pasteboard.setString(text, forType: .string)
|
|
}
|
|
|
|
public func writeItems(_ items: [PasteboardItem]) {
|
|
let pasteboard = NSPasteboard.general
|
|
pasteboard.clearContents()
|
|
let restored = items.map { dict -> NSPasteboardItem in
|
|
let item = NSPasteboardItem()
|
|
for (type, data) in dict { item.setData(data, forType: NSPasteboard.PasteboardType(type)) }
|
|
return item
|
|
}
|
|
if !restored.isEmpty { pasteboard.writeObjects(restored) }
|
|
}
|
|
}
|
|
|
|
public struct CGEventPaster: KeystrokePosting {
|
|
public enum PasteError: Error, LocalizedError {
|
|
case accessibilityDenied
|
|
case eventCreationFailed
|
|
|
|
public var errorDescription: String? {
|
|
switch self {
|
|
case .accessibilityDenied:
|
|
"Geen Accessibility-rechten: tekst staat op het klembord, plak zelf met ⌘V."
|
|
case .eventCreationFailed:
|
|
"Kon het plak-event niet aanmaken: tekst staat op het klembord."
|
|
}
|
|
}
|
|
}
|
|
|
|
public init() {}
|
|
|
|
public func postPaste() throws {
|
|
guard AXIsProcessTrusted() else { throw PasteError.accessibilityDenied }
|
|
let source = CGEventSource(stateID: .combinedSessionState)
|
|
let vKey: CGKeyCode = 0x09
|
|
guard
|
|
let down = CGEvent(keyboardEventSource: source, virtualKey: vKey, keyDown: true),
|
|
let up = CGEvent(keyboardEventSource: source, virtualKey: vKey, keyDown: false)
|
|
else { throw PasteError.eventCreationFailed }
|
|
down.flags = .maskCommand
|
|
up.flags = .maskCommand
|
|
down.post(tap: .cgAnnotatedSessionEventTap)
|
|
up.post(tap: .cgAnnotatedSessionEventTap)
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Alle tests draaien**
|
|
|
|
Run: `swift test`
|
|
Expected: alle tests PASS (8 controller + 3 inserter/cleaner).
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add Sources Tests
|
|
git commit -m "feat: add TextInserter with clipboard restore and PassthroughCleaner" -m "Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 3: Fn-hotkey (FnKeyTracker, HotkeyMonitor) en probe
|
|
|
|
**Files:**
|
|
- Create: `Sources/SstCore/FnKeyTracker.swift`, `Sources/SstCore/HotkeyMonitor.swift`
|
|
- Create: `Sources/SstProbe/main.swift`, `Sources/SstProbe/HotkeyProbe.swift`
|
|
- Modify: `Package.swift`
|
|
- Test: `Tests/SstCoreTests/FnKeyTrackerTests.swift`
|
|
|
|
**Interfaces:**
|
|
- Consumes: `HotkeyEvent` uit Task 1.
|
|
- Produces:
|
|
- `struct FnKeyTracker { init(); mutating func process(fnDown: Bool) -> HotkeyEvent? }`
|
|
- `final class HotkeyMonitor { init(); var onEvent: ((HotkeyEvent) -> Void)?; func start() throws; func stop() }`, `HotkeyMonitor.HotkeyError: Error, LocalizedError`
|
|
- Executable `SstProbe` met subcommando `hotkey`.
|
|
|
|
- [ ] **Step 1: Falende test schrijven**
|
|
|
|
`Tests/SstCoreTests/FnKeyTrackerTests.swift`:
|
|
|
|
```swift
|
|
import Testing
|
|
@testable import SstCore
|
|
|
|
struct FnKeyTrackerTests {
|
|
@Test func emitsPressedThenReleasedOnlyOnTransitions() {
|
|
var tracker = FnKeyTracker()
|
|
#expect(tracker.process(fnDown: false) == nil)
|
|
#expect(tracker.process(fnDown: true) == .pressed)
|
|
#expect(tracker.process(fnDown: true) == nil)
|
|
#expect(tracker.process(fnDown: false) == .released)
|
|
#expect(tracker.process(fnDown: false) == nil)
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Test draaien, moet falen**
|
|
|
|
Run: `swift test --filter FnKeyTrackerTests`
|
|
Expected: compilatiefout `cannot find 'FnKeyTracker' in scope`.
|
|
|
|
- [ ] **Step 3: Implementatie**
|
|
|
|
`Sources/SstCore/FnKeyTracker.swift`:
|
|
|
|
```swift
|
|
public struct FnKeyTracker {
|
|
private var isDown = false
|
|
|
|
public init() {}
|
|
|
|
/// Feed the current Fn flag state; returns an event only on a transition.
|
|
public mutating func process(fnDown: Bool) -> HotkeyEvent? {
|
|
guard fnDown != isDown else { return nil }
|
|
isDown = fnDown
|
|
return fnDown ? .pressed : .released
|
|
}
|
|
}
|
|
```
|
|
|
|
`Sources/SstCore/HotkeyMonitor.swift`:
|
|
|
|
```swift
|
|
import ApplicationServices
|
|
import CoreGraphics
|
|
|
|
public final class HotkeyMonitor {
|
|
public enum HotkeyError: Error, LocalizedError {
|
|
case tapUnavailable
|
|
|
|
public var errorDescription: String? {
|
|
"Kan de Fn-toets niet volgen: geef Input Monitoring-rechten in Systeeminstellingen en probeer opnieuw (soms is een herstart van de app nodig)."
|
|
}
|
|
}
|
|
|
|
public var onEvent: ((HotkeyEvent) -> Void)?
|
|
|
|
private var tracker = FnKeyTracker()
|
|
private var tap: CFMachPort?
|
|
private var source: CFRunLoopSource?
|
|
|
|
public init() {}
|
|
|
|
public func start() throws {
|
|
guard tap == nil else { return }
|
|
let mask = CGEventMask(1 << CGEventType.flagsChanged.rawValue)
|
|
let callback: CGEventTapCallBack = { _, type, event, refcon in
|
|
if let refcon {
|
|
let monitor = Unmanaged<HotkeyMonitor>.fromOpaque(refcon).takeUnretainedValue()
|
|
monitor.handle(type: type, event: event)
|
|
}
|
|
return Unmanaged.passUnretained(event)
|
|
}
|
|
guard let tap = CGEvent.tapCreate(
|
|
tap: .cgSessionEventTap,
|
|
place: .headInsertEventTap,
|
|
options: .listenOnly,
|
|
eventsOfInterest: mask,
|
|
callback: callback,
|
|
userInfo: Unmanaged.passUnretained(self).toOpaque()
|
|
) else { throw HotkeyError.tapUnavailable }
|
|
|
|
let source = CFMachPortCreateRunLoopSource(kCFAllocatorDefault, tap, 0)
|
|
CFRunLoopAddSource(CFRunLoopGetMain(), source, .commonModes)
|
|
CGEvent.tapEnable(tap: tap, enable: true)
|
|
self.tap = tap
|
|
self.source = source
|
|
}
|
|
|
|
public func stop() {
|
|
if let source { CFRunLoopRemoveSource(CFRunLoopGetMain(), source, .commonModes) }
|
|
if let tap { CGEvent.tapEnable(tap: tap, enable: false) }
|
|
tap = nil
|
|
source = nil
|
|
}
|
|
|
|
private func handle(type: CGEventType, event: CGEvent) {
|
|
switch type {
|
|
case .tapDisabledByTimeout, .tapDisabledByUserInput:
|
|
if let tap { CGEvent.tapEnable(tap: tap, enable: true) }
|
|
case .flagsChanged:
|
|
let fnDown = event.flags.contains(.maskSecondaryFn)
|
|
if let hotkeyEvent = tracker.process(fnDown: fnDown) {
|
|
onEvent?(hotkeyEvent)
|
|
}
|
|
default:
|
|
break
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
`Package.swift` (vervang volledig):
|
|
|
|
```swift
|
|
// swift-tools-version: 6.0
|
|
import PackageDescription
|
|
|
|
let package = Package(
|
|
name: "Sst",
|
|
platforms: [.macOS(.v14)],
|
|
targets: [
|
|
.target(name: "SstCore"),
|
|
.executableTarget(name: "SstProbe", dependencies: ["SstCore"]),
|
|
.testTarget(name: "SstCoreTests", dependencies: ["SstCore"]),
|
|
],
|
|
swiftLanguageModes: [.v5]
|
|
)
|
|
```
|
|
|
|
`Sources/SstProbe/main.swift`:
|
|
|
|
```swift
|
|
import Foundation
|
|
|
|
let command = CommandLine.arguments.dropFirst().first ?? "help"
|
|
switch command {
|
|
case "hotkey":
|
|
runHotkeyProbe()
|
|
default:
|
|
print("usage: sst-probe hotkey")
|
|
}
|
|
```
|
|
|
|
`Sources/SstProbe/HotkeyProbe.swift`:
|
|
|
|
```swift
|
|
import Foundation
|
|
import CoreGraphics
|
|
import SstCore
|
|
|
|
func runHotkeyProbe() -> Never {
|
|
if !CGPreflightListenEventAccess() { _ = CGRequestListenEventAccess() }
|
|
let monitor = HotkeyMonitor()
|
|
monitor.onEvent = { print("hotkey:", $0) }
|
|
do { try monitor.start() } catch {
|
|
print("failed:", error.localizedDescription)
|
|
exit(1)
|
|
}
|
|
print("Houd Fn ingedrukt en laat los. Ctrl-C om te stoppen.")
|
|
RunLoop.main.run()
|
|
exit(0)
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Tests draaien**
|
|
|
|
Run: `swift test`
|
|
Expected: alle tests PASS (incl. `FnKeyTrackerTests`).
|
|
|
|
- [ ] **Step 5: Handmatige check van de hotkey**
|
|
|
|
Zet eerst in Systeeminstellingen → Toetsenbord "Druk op 🌐 voor" op **Doe niets** (anders opent Fn de emoji-kiezer/dicteren).
|
|
|
|
Run: `swift run SstProbe hotkey`
|
|
Geef de terminal-app (waar je dit draait) Input Monitoring-rechten als daarom gevraagd wordt en start opnieuw.
|
|
Expected: bij Fn omlaag `hotkey: pressed`, bij loslaten `hotkey: released`, geen dubbele events.
|
|
|
|
- [ ] **Step 6: Commit**
|
|
|
|
```bash
|
|
git add Package.swift Sources Tests
|
|
git commit -m "feat: add Fn hotkey monitor and probe CLI" -m "Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 4: AudioRecorder en record-probe
|
|
|
|
**Files:**
|
|
- Create: `Sources/SstCore/AudioRecorder.swift`, `Sources/SstProbe/RecordProbe.swift`
|
|
- Modify: `Sources/SstProbe/main.swift`
|
|
|
|
**Interfaces:**
|
|
- Consumes: `AudioRecording` uit Task 1.
|
|
- Produces:
|
|
- `final class AudioRecorder: AudioRecording`, `init()`, `AudioRecorder.RecorderError: Error, LocalizedError`
|
|
- Probe-helper `func recordSamples(seconds: Double) async -> [Float]?` (hergebruikt in Task 5)
|
|
- Probe-subcommando `record`
|
|
|
|
De AudioRecorder is hardware-afhankelijk en wordt daarom handmatig getest via de probe.
|
|
|
|
- [ ] **Step 1: Implementatie**
|
|
|
|
`Sources/SstCore/AudioRecorder.swift`:
|
|
|
|
```swift
|
|
import AVFoundation
|
|
|
|
public final class AudioRecorder: AudioRecording {
|
|
public enum RecorderError: Error, LocalizedError {
|
|
case converterUnavailable
|
|
|
|
public var errorDescription: String? {
|
|
"Kon de microfoon niet omzetten naar 16 kHz mono."
|
|
}
|
|
}
|
|
|
|
private let engine = AVAudioEngine()
|
|
private let lock = NSLock()
|
|
private var buffer: [Float] = []
|
|
private let targetFormat = AVAudioFormat(
|
|
commonFormat: .pcmFormatFloat32,
|
|
sampleRate: 16_000,
|
|
channels: 1,
|
|
interleaved: false
|
|
)!
|
|
|
|
public init() {}
|
|
|
|
public func start() throws {
|
|
lock.withLock { buffer.removeAll(keepingCapacity: true) }
|
|
|
|
let input = engine.inputNode
|
|
let inputFormat = input.outputFormat(forBus: 0)
|
|
guard let converter = AVAudioConverter(from: inputFormat, to: targetFormat) else {
|
|
throw RecorderError.converterUnavailable
|
|
}
|
|
let ratio = targetFormat.sampleRate / inputFormat.sampleRate
|
|
|
|
input.installTap(onBus: 0, bufferSize: 4096, format: inputFormat) { [weak self] pcm, _ in
|
|
guard let self else { return }
|
|
let capacity = AVAudioFrameCount(Double(pcm.frameLength) * ratio) + 1
|
|
guard let out = AVAudioPCMBuffer(pcmFormat: self.targetFormat, frameCapacity: capacity) else { return }
|
|
|
|
var consumed = false
|
|
var error: NSError?
|
|
converter.convert(to: out, error: &error) { _, status in
|
|
if consumed {
|
|
status.pointee = .noDataNow
|
|
return nil
|
|
}
|
|
consumed = true
|
|
status.pointee = .haveData
|
|
return pcm
|
|
}
|
|
guard error == nil, let channel = out.floatChannelData?[0] else { return }
|
|
let samples = Array(UnsafeBufferPointer(start: channel, count: Int(out.frameLength)))
|
|
self.lock.withLock { self.buffer.append(contentsOf: samples) }
|
|
}
|
|
engine.prepare()
|
|
try engine.start()
|
|
}
|
|
|
|
public func stop() -> [Float] {
|
|
engine.inputNode.removeTap(onBus: 0)
|
|
engine.stop()
|
|
return lock.withLock {
|
|
let samples = buffer
|
|
buffer = []
|
|
return samples
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
`Sources/SstProbe/RecordProbe.swift`:
|
|
|
|
```swift
|
|
import AVFoundation
|
|
import SstCore
|
|
|
|
/// Records `seconds` of microphone audio; returns nil if access is denied or recording fails.
|
|
func recordSamples(seconds: Double) async -> [Float]? {
|
|
guard await AVCaptureDevice.requestAccess(for: .audio) else {
|
|
print("Microfoon-toegang geweigerd (Systeeminstellingen → Privacy → Microfoon).")
|
|
return nil
|
|
}
|
|
let recorder = AudioRecorder()
|
|
do { try recorder.start() } catch {
|
|
print("start mislukt:", error.localizedDescription)
|
|
return nil
|
|
}
|
|
print("Neem \(Int(seconds)) s op — praat nu")
|
|
try? await Task.sleep(for: .seconds(seconds))
|
|
return recorder.stop()
|
|
}
|
|
|
|
func runRecordProbe() async {
|
|
guard let samples = await recordSamples(seconds: 3) else { return }
|
|
let peak = samples.map { abs($0) }.max() ?? 0
|
|
print("samples: \(samples.count) (verwacht ≈ 48000), piek: \(peak)")
|
|
}
|
|
```
|
|
|
|
`Sources/SstProbe/main.swift` (vervang volledig):
|
|
|
|
```swift
|
|
import Foundation
|
|
|
|
let command = CommandLine.arguments.dropFirst().first ?? "help"
|
|
switch command {
|
|
case "hotkey":
|
|
runHotkeyProbe()
|
|
case "record":
|
|
await runRecordProbe()
|
|
default:
|
|
print("usage: sst-probe hotkey|record")
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Bouwen**
|
|
|
|
Run: `swift build`
|
|
Expected: build slaagt zonder errors.
|
|
|
|
- [ ] **Step 3: Handmatige check**
|
|
|
|
Run: `swift run SstProbe record` en praat 3 seconden.
|
|
Expected: `samples:` ligt rond 48000 (±2000) en `piek` is duidelijk > 0,05. Bij stilte is de piek ≈ 0. Bij een geweigerde microfoon staat er een duidelijke melding.
|
|
|
|
- [ ] **Step 4: Commit**
|
|
|
|
```bash
|
|
git add Sources
|
|
git commit -m "feat: add AudioRecorder (16 kHz mono) and record probe" -m "Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 5: WhisperKitTranscriber en transcribe-probe
|
|
|
|
**Files:**
|
|
- Create: `Sources/SstWhisper/WhisperKitTranscriber.swift`
|
|
- Create: `Sources/SstProbe/TranscribeProbe.swift`
|
|
- Modify: `Package.swift`, `Sources/SstProbe/main.swift`
|
|
|
|
**Interfaces:**
|
|
- Consumes: `Transcriber` uit Task 1, `recordSamples(seconds:)` uit Task 4.
|
|
- Produces:
|
|
- `final class WhisperKitTranscriber: Transcriber` met `init(model: String = "large-v3-v20240930_turbo")`, `func load() async throws`, `WhisperKitTranscriber.TranscriberError: Error, LocalizedError`
|
|
- Probe-subcommando `transcribe`
|
|
|
|
- [ ] **Step 1: Dependency en target toevoegen**
|
|
|
|
`Package.swift` (vervang volledig):
|
|
|
|
```swift
|
|
// swift-tools-version: 6.0
|
|
import PackageDescription
|
|
|
|
let package = Package(
|
|
name: "Sst",
|
|
platforms: [.macOS(.v14)],
|
|
dependencies: [
|
|
.package(url: "https://github.com/argmaxinc/WhisperKit.git", from: "0.9.0"),
|
|
],
|
|
targets: [
|
|
.target(name: "SstCore"),
|
|
.target(
|
|
name: "SstWhisper",
|
|
dependencies: [
|
|
"SstCore",
|
|
.product(name: "WhisperKit", package: "WhisperKit"),
|
|
]
|
|
),
|
|
.executableTarget(name: "SstProbe", dependencies: ["SstCore", "SstWhisper"]),
|
|
.testTarget(name: "SstCoreTests", dependencies: ["SstCore"]),
|
|
],
|
|
swiftLanguageModes: [.v5]
|
|
)
|
|
```
|
|
|
|
- [ ] **Step 2: Implementatie**
|
|
|
|
`Sources/SstWhisper/WhisperKitTranscriber.swift`:
|
|
|
|
```swift
|
|
import Foundation
|
|
import SstCore
|
|
import WhisperKit
|
|
|
|
public final class WhisperKitTranscriber: Transcriber {
|
|
public enum TranscriberError: Error, LocalizedError {
|
|
case notLoaded
|
|
|
|
public var errorDescription: String? { "Spraakmodel is nog niet geladen." }
|
|
}
|
|
|
|
private let modelName: String
|
|
private var pipeline: WhisperKit?
|
|
|
|
public init(model: String = "large-v3-v20240930_turbo") {
|
|
self.modelName = model
|
|
}
|
|
|
|
/// Downloads the model on first run (≈1.5 GB) and loads it into memory.
|
|
public func load() async throws {
|
|
pipeline = try await WhisperKit(WhisperKitConfig(model: modelName))
|
|
}
|
|
|
|
public func transcribe(_ samples: [Float]) async throws -> String {
|
|
guard let pipeline else { throw TranscriberError.notLoaded }
|
|
// language nil + detectLanguage: Whisper picks NL or EN per utterance.
|
|
let options = DecodingOptions(
|
|
task: .transcribe,
|
|
language: nil,
|
|
detectLanguage: true,
|
|
skipSpecialTokens: true,
|
|
withoutTimestamps: true
|
|
)
|
|
let results = try await pipeline.transcribe(audioArray: samples, decodeOptions: options)
|
|
return results
|
|
.map(\.text)
|
|
.joined(separator: " ")
|
|
.trimmingCharacters(in: .whitespacesAndNewlines)
|
|
}
|
|
}
|
|
```
|
|
|
|
> Als de WhisperKit-API of het modelnaam-patroon in de opgeloste versie afwijkt (compilatiefout of "model not found"), pas dan alleen dit bestand aan. Beschikbare modelnamen: `WhisperKit.fetchAvailableModels()`. Houd `Transcriber` en `load()` gelijk, want Task 6 hangt daarvan af.
|
|
|
|
`Sources/SstProbe/TranscribeProbe.swift`:
|
|
|
|
```swift
|
|
import Foundation
|
|
import SstWhisper
|
|
|
|
func runTranscribeProbe() async {
|
|
guard let samples = await recordSamples(seconds: 6) else { return }
|
|
print("Model laden (eerste keer: download ≈1.5 GB)…")
|
|
let transcriber = WhisperKitTranscriber()
|
|
do {
|
|
try await transcriber.load()
|
|
let started = Date.now
|
|
let text = try await transcriber.transcribe(samples)
|
|
let elapsed = Date.now.timeIntervalSince(started)
|
|
print(String(format: "(%.2f s) ", elapsed) + text)
|
|
} catch {
|
|
print("mislukt:", error.localizedDescription)
|
|
}
|
|
}
|
|
```
|
|
|
|
`Sources/SstProbe/main.swift` (vervang volledig):
|
|
|
|
```swift
|
|
import Foundation
|
|
|
|
let command = CommandLine.arguments.dropFirst().first ?? "help"
|
|
switch command {
|
|
case "hotkey":
|
|
runHotkeyProbe()
|
|
case "record":
|
|
await runRecordProbe()
|
|
case "transcribe":
|
|
await runTranscribeProbe()
|
|
default:
|
|
print("usage: sst-probe hotkey|record|transcribe")
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 3: Bouwen**
|
|
|
|
Run: `swift build`
|
|
Expected: dependencies worden opgehaald, build slaagt. Los API-afwijkingen op in `WhisperKitTranscriber.swift` (zie opmerking hierboven).
|
|
|
|
- [ ] **Step 4: Handmatige check (NL, EN, gemengd)**
|
|
|
|
Run: `swift run SstProbe transcribe`
|
|
Zeg achtereenvolgens: een Nederlandse zin, een Engelse zin, en een zin die begint in het Nederlands met een paar Engelse woorden.
|
|
Expected: correcte transcriptie van de zinnen; na het laden duurt transcriberen van 6 s audio ruim onder 3 s op de M4 Pro. Noteer de gemeten tijd.
|
|
Bekende beperking: taaldetectie is per opname, dus wisselen midden in een zin kan een van beide talen minder goed herkennen. Dat is acceptabel voor de skeleton.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add Package.swift Package.resolved Sources
|
|
git commit -m "feat: add WhisperKit transcriber and transcribe probe" -m "Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 6: Menubalk-app, app-bundle en rooktest
|
|
|
|
**Files:**
|
|
- Create: `Sources/Sst/SstApp.swift`, `Sources/Sst/AppModel.swift`, `Sources/Sst/MenuContent.swift`
|
|
- Create: `scripts/make-app.sh`
|
|
- Modify: `Package.swift`
|
|
|
|
**Interfaces:**
|
|
- Consumes: `DictationController`, `HotkeyMonitor`, `AudioRecorder`, `TextInserter`, `SystemPasteboard`, `CGEventPaster`, `PassthroughCleaner` (SstCore); `WhisperKitTranscriber` met `load()` (SstWhisper).
|
|
- Produces: de app-bundle `build/Sst.app`.
|
|
|
|
- [ ] **Step 1: Executable target toevoegen**
|
|
|
|
In `Package.swift`: voeg de regel `.executableTarget(name: "Sst", dependencies: ["SstCore", "SstWhisper"]),` toe aan `targets`, direct boven de `SstProbe`-regel.
|
|
|
|
- [ ] **Step 2: App-code**
|
|
|
|
`Sources/Sst/AppModel.swift`:
|
|
|
|
```swift
|
|
import AVFoundation
|
|
import ApplicationServices
|
|
import SwiftUI
|
|
import SstCore
|
|
import SstWhisper
|
|
|
|
@MainActor
|
|
final class AppModel: ObservableObject {
|
|
@Published var state: DictationState = .idle
|
|
@Published var modelStatus = "Model laden…"
|
|
@Published var modelReady = false
|
|
@Published var lastError: String?
|
|
@Published var micGranted = false
|
|
@Published var accessibilityGranted = false
|
|
@Published var inputMonitoringGranted = false
|
|
|
|
private let transcriber = WhisperKitTranscriber()
|
|
private let hotkey = HotkeyMonitor()
|
|
private let controller: DictationController
|
|
|
|
init() {
|
|
controller = DictationController(
|
|
recorder: AudioRecorder(),
|
|
transcriber: transcriber,
|
|
cleaner: PassthroughCleaner(),
|
|
inserter: TextInserter(pasteboard: SystemPasteboard(), paster: CGEventPaster())
|
|
)
|
|
controller.onStateChange = { [weak self] in self?.state = $0 }
|
|
controller.onError = { [weak self] in self?.lastError = $0 }
|
|
hotkey.onEvent = { [weak self] event in
|
|
guard let self, self.modelReady else { return }
|
|
Task { await self.controller.handle(event) }
|
|
}
|
|
Task { await start() }
|
|
}
|
|
|
|
var statusLine: String {
|
|
guard modelReady else { return modelStatus }
|
|
switch state {
|
|
case .idle: return "Klaar — houd Fn ingedrukt om te dicteren"
|
|
case .recording: return "Luistert…"
|
|
case .transcribing: return "Transcriberen…"
|
|
case .inserting: return "Invoegen…"
|
|
}
|
|
}
|
|
|
|
var iconName: String {
|
|
switch state {
|
|
case .idle: return modelReady ? "mic" : "mic.slash"
|
|
case .recording: return "mic.fill"
|
|
case .transcribing, .inserting: return "waveform"
|
|
}
|
|
}
|
|
|
|
func refreshPermissions() {
|
|
micGranted = AVCaptureDevice.authorizationStatus(for: .audio) == .authorized
|
|
accessibilityGranted = AXIsProcessTrusted()
|
|
inputMonitoringGranted = CGPreflightListenEventAccess()
|
|
}
|
|
|
|
func retryHotkey() {
|
|
refreshPermissions()
|
|
do {
|
|
try hotkey.start()
|
|
lastError = nil
|
|
} catch {
|
|
lastError = error.localizedDescription
|
|
}
|
|
}
|
|
|
|
private func start() async {
|
|
_ = await AVCaptureDevice.requestAccess(for: .audio)
|
|
if !CGPreflightListenEventAccess() { _ = CGRequestListenEventAccess() }
|
|
if !AXIsProcessTrusted() {
|
|
_ = AXIsProcessTrustedWithOptions(["AXTrustedCheckOptionPrompt": true] as CFDictionary)
|
|
}
|
|
retryHotkey()
|
|
do {
|
|
try await transcriber.load()
|
|
modelReady = true
|
|
modelStatus = "Klaar"
|
|
} catch {
|
|
modelStatus = "Model laden mislukt"
|
|
lastError = error.localizedDescription
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
`Sources/Sst/MenuContent.swift`:
|
|
|
|
```swift
|
|
import AppKit
|
|
import SwiftUI
|
|
|
|
struct MenuContent: View {
|
|
@ObservedObject var model: AppModel
|
|
|
|
var body: some View {
|
|
Text(model.statusLine)
|
|
if let error = model.lastError {
|
|
Text("Fout: \(error)")
|
|
}
|
|
Divider()
|
|
permissionRow("Microfoon", granted: model.micGranted, pane: "Privacy_Microphone")
|
|
permissionRow("Accessibility", granted: model.accessibilityGranted, pane: "Privacy_Accessibility")
|
|
permissionRow("Input Monitoring", granted: model.inputMonitoringGranted, pane: "Privacy_ListenEvent")
|
|
Button("Rechten opnieuw controleren") { model.retryHotkey() }
|
|
Divider()
|
|
Button("Stop") { NSApplication.shared.terminate(nil) }
|
|
.keyboardShortcut("q")
|
|
}
|
|
|
|
@ViewBuilder
|
|
private func permissionRow(_ title: String, granted: Bool, pane: String) -> some View {
|
|
if granted {
|
|
Text("✓ \(title)")
|
|
} else {
|
|
Button("⚠︎ \(title) — open Instellingen") {
|
|
if let url = URL(string: "x-apple.systempreferences:com.apple.preference.security?\(pane)") {
|
|
NSWorkspace.shared.open(url)
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
`Sources/Sst/SstApp.swift`:
|
|
|
|
```swift
|
|
import SwiftUI
|
|
|
|
@main
|
|
struct SstApp: App {
|
|
@StateObject private var model = AppModel()
|
|
|
|
var body: some Scene {
|
|
MenuBarExtra {
|
|
MenuContent(model: model)
|
|
} label: {
|
|
Image(systemName: model.iconName)
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
> Als de model-start pas begint na het openen van het menu (StateObject wordt lui aangemaakt), verplaats dan `AppModel` naar een `NSApplicationDelegateAdaptor`-delegate die het model in `applicationDidFinishLaunching` aanmaakt.
|
|
|
|
- [ ] **Step 3: Bundle-script**
|
|
|
|
`scripts/make-app.sh`:
|
|
|
|
```bash
|
|
#!/usr/bin/env bash
|
|
set -euo pipefail
|
|
cd "$(dirname "$0")/.."
|
|
|
|
swift build -c release --product Sst
|
|
BIN=".build/release/Sst"
|
|
APP="build/Sst.app"
|
|
|
|
rm -rf "$APP"
|
|
mkdir -p "$APP/Contents/MacOS" "$APP/Contents/Resources"
|
|
cp "$BIN" "$APP/Contents/MacOS/Sst"
|
|
# SwiftPM resource bundles (if any dependency ships them) live next to the binary.
|
|
for b in .build/release/*.bundle; do
|
|
[ -e "$b" ] && cp -R "$b" "$APP/Contents/Resources/"
|
|
done
|
|
|
|
cat > "$APP/Contents/Info.plist" <<'PLIST'
|
|
<?xml version="1.0" encoding="UTF-8"?>
|
|
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
|
<plist version="1.0">
|
|
<dict>
|
|
<key>CFBundleIdentifier</key><string>local.sst.app</string>
|
|
<key>CFBundleName</key><string>Sst</string>
|
|
<key>CFBundleExecutable</key><string>Sst</string>
|
|
<key>CFBundlePackageType</key><string>APPL</string>
|
|
<key>CFBundleShortVersionString</key><string>0.1.0</string>
|
|
<key>CFBundleVersion</key><string>1</string>
|
|
<key>LSMinimumSystemVersion</key><string>14.0</string>
|
|
<key>LSUIElement</key><true/>
|
|
<key>NSMicrophoneUsageDescription</key>
|
|
<string>Sst neemt je stem op zolang je Fn ingedrukt houdt, om die lokaal naar tekst om te zetten.</string>
|
|
</dict>
|
|
</plist>
|
|
PLIST
|
|
|
|
codesign --force --sign - --identifier local.sst.app "$APP"
|
|
echo "Gebouwd: $APP"
|
|
```
|
|
|
|
- [ ] **Step 4: Bouwen**
|
|
|
|
Run: `chmod +x scripts/make-app.sh && ./scripts/make-app.sh`
|
|
Expected: `Gebouwd: build/Sst.app`, zonder errors.
|
|
|
|
- [ ] **Step 5: Rooktest van de hele keten**
|
|
|
|
Run: `open build/Sst.app`
|
|
Controleer in deze volgorde:
|
|
1. Mic-icoon verschijnt in de menubalk (geen Dock-icoon). Menu toont "Model laden…" en daarna "Klaar — houd Fn ingedrukt om te dicteren" (eerste keer duurt de download enkele minuten).
|
|
2. Geef Microfoon, Accessibility en Input Monitoring toe via de prompts of de knoppen in het menu; klik "Rechten opnieuw controleren" (zo nodig de app herstarten). Alle drie staan op ✓.
|
|
3. Open TextEdit, klik in een leeg document, houd Fn ingedrukt, zeg "Dit is een test van de dictatie", laat los. Icoon: `mic.fill` → `waveform` → `mic`. Tekst verschijnt op de cursor.
|
|
4. Herhaal in Notes en in een browser-tekstveld, met een Engelse en een gemengde zin.
|
|
5. Zet een eigen tekst op het klembord, dicteer, en controleer dat het klembord daarna weer je eigen tekst bevat.
|
|
6. Tik Fn kort aan (< 0,3 s): er mag niets ingevoegd worden.
|
|
7. Weiger Accessibility tijdelijk: het menu toont een foutmelding en de tekst staat op het klembord.
|
|
|
|
Bekende beperking: omdat de app ad-hoc is gesigneerd (`codesign --sign -`), kan macOS de rechten na elke herbouw opnieuw vragen. Een vast ontwikkelcertificaat lost dat later op.
|
|
|
|
- [ ] **Step 6: Commit**
|
|
|
|
```bash
|
|
git add Package.swift Sources scripts
|
|
git commit -m "feat: add menu bar app and bundle script" -m "Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"
|
|
```
|
|
|
|
---
|
|
|
|
## Self-review
|
|
|
|
- **Spec-dekking:** HotkeyMonitor (Task 3), AudioRecorder (4), Transcriber/WhisperKit (5), Cleaner-naad (2), TextInserter met klembordherstel en klembord-bij-falen (2), DictationController met statemachine, minimale opnameduur, lege transcriptie en foutpaden (1), MenuBarExtra met rechten-check en modelstatus (6). Niet gedekt, bewust: overlay-pill, cleanup, branding (latere specs).
|
|
- **Placeholders:** geen; alle codestappen bevatten volledige code. De twee opmerkingen over WhisperKit-API en StateObject zijn gerichte fallbacks, geen open eindjes.
|
|
- **Typeconsistentie:** `HotkeyEvent`, `DictationState`, `CleaningContext`, de vier protocollen en `DictationController.init(recorder:transcriber:cleaner:inserter:minimumSamples:)` zijn in Task 1 gedefinieerd en komen identiek terug in Task 2 t/m 6. `WhisperKitTranscriber.load()` uit Task 5 wordt in Task 6 gebruikt.
|
|
- **Risico's:** de WhisperKit-API en het exacte modelnaam-patroon zijn het enige onverifieerde punt; Task 5 stap 3 vangt dat vroeg af.
|