Clean Speech Daemon

Standalone Linux microphone-cleanup service for voice agents, GUI testbeds, and other local clients.

It captures webcam microphone audio, applies high-pass filtering, adaptive and neural echo/reference reduction when a reference input is configured, spectral noise suppression, and speech-activity gating, then publishes the cleaned stream through:

1. A PulseAudio/PipeWire virtual source named Clean Speech Microphone
2. A Unix socket at /tmp/clean-speech-daemon.sock
3. A multi-stream Unix socket at /tmp/clean-speech-daemon-streams.sock
4. A raw PCM FIFO at /tmp/clean-speech-daemon.pcm
5. An optional debug WAV file

Status: working prototype. The I/O, daemon, socket, FIFO, config, and service interfaces are stable; the DSP/model stages are still being tuned. See Quality Notes.

Features

• Acoustic echo cancellation against the system playback monitor, so it removes all speaker output (not just one app's audio) leaking into the mic. Adaptive (NLMS) and neural (DTLN / NKF / hybrid) backends are selectable.
• Spectral noise suppression and speech-activity gating with configurable pre/post-roll.
• Clock-drift compensation between the mic and the playback monitor.
• Multiple outputs: a virtual microphone, a raw PCM socket, a multi-stream diagnostic socket, a FIFO, and optional debug WAVs.
• Runtime diagnostics and ready-to-copy tuning profiles.
• Live tuning over a control socket (delay, echo canceller, blend ratios) with no restart, and a GUI testbed for A/B-ing settings by ear while watching the numbers move.

Requirements

• Linux x86-64 with PipeWire (PipeWire-Pulse) or PulseAudio
• Python 3.11+
• pactl for the virtual microphone (usually preinstalled)
• The default hybrid_localvqe echo canceller needs the neural extras (pip install -e '.[neural]': onnxruntime, soxr, torch). The bundled LocalVQE GGML libs/model (lib/, models/) are x86-64 only. Without the extras the daemon falls back to the dependency-free nlms canceller.
• pip install -e '.[test]' (pytest, scipy) if you want to run the test suite; see Running Tests.

Quick Start

From this directory:

python3 -m venv .venv
. .venv/bin/activate
pip install -e .
clean-speech-daemon write-config
clean-speech-daemon devices
clean-speech-daemon run

This gets you the dependency-free nlms echo canceller (the automatic fallback when neural extras or models are missing). For the shipped default, hybrid_localvqe, install the neural extras first: pip install -e '.[neural]'. Spectral noise suppression is off by default; re-enable it in config.toml only when you are comparing that stage.

Select Clean Speech Microphone in your GUI testbed. If the virtual source cannot be created, use the socket stream instead.

Socket Stream

The socket sends one JSON metadata line, then raw little-endian signed 16-bit mono PCM at 48 kHz by default.

Record ten seconds from the cleaned stream:

clean-speech-daemon record-socket /tmp/cleaned.wav --seconds 10

The multi-stream socket is for diagnostics and GUI testbeds. It sends mic_raw, system_reference, reference_aligned, reference_matched, after_echo, and cleaned_output frames without writing them to disk. Use after_echo to judge AEC efficacy; cleaned_output is the final published stream and may include VAD pre-roll delay:

/tmp/clean-speech-daemon-streams.sock

Runtime Control Socket

Most tuning parameters can be changed on a running daemon without a restart, over a Unix socket at /tmp/clean-speech-daemon-control.sock. Send one JSON object per line; the daemon replies with {"ok": true, "queued": true} or {"ok": false, "error": "..."} (unrecognized echo_canceller/ reference_delay_mode values are rejected here rather than silently applied).

from clean_speech_daemon.control import send_control_command

send_control_command("/tmp/clean-speech-daemon-control.sock", {"mic_delay_ms": 45})
send_control_command("/tmp/clean-speech-daemon-control.sock", {"echo_canceller": "hybrid"})

Tunable keys: reference_delay_ms, reference_delay_mode, mic_delay_ms, reference_sync_latency_frames, echo_canceller, dtln_mask_smoothing, localvqe_blend, and reset_echo_filter (boolean, clears the adaptive filter's state). The GUI testbed drives this socket for its live A/B panels, so it's a convenient way to explore these interactively before hardcoding values into config.toml.

Configure Devices

List inputs:

clean-speech-daemon devices

Edit:

~/.config/clean-speech-daemon/config.toml

Set input.microphone to a device index or a substring of the webcam microphone name. microphone = "auto" skips the daemon's own Clean Speech Microphone virtual source so the process cannot record its own output if that source has become the PortAudio default. On Linux, system audio is exposed through Pulse/PipeWire monitor sources, and the correct source is often the currently RUNNING monitor rather than the default sink. input.system_audio_reference = "auto" prefers the active monitor source and falls back to the default sink monitor.

List Pulse/PipeWire sources and monitor states:

clean-speech-daemon pulse-sources

If auto-selection chooses the wrong output device, set input.system_audio_reference to the exact .monitor source name from that list.

Example:

[input]
microphone = "HD Webcam"
system_audio_reference = "auto"

systemd User Service

The daemon should normally be run manually while testing. Do not leave it always listening unless you explicitly need that behavior.

Manual run:

clean-speech-daemon run

Install the optional user service without starting it:

scripts/install-user-service.sh

Start it manually for a test session:

systemctl --user start clean-speech-daemon.service

Stop it when done:

systemctl --user stop clean-speech-daemon.service

If it was previously enabled at login, disable it:

systemctl --user disable --now clean-speech-daemon.service

View logs:

journalctl --user -u clean-speech-daemon.service -f

Virtual Microphone

The daemon can automatically run pactl load-module module-pipe-source so GUI applications see Clean Speech Microphone. This works on typical Ubuntu PipeWire-Pulse and PulseAudio setups.

If auto-loading fails, create it manually:

scripts/create-virtual-source.sh

GUI Testbed

testbed/ is a separate Tkinter GUI for watching and tuning the daemon while it runs, without touching the mic yourself:

cd testbed
python3 clean_speech_testbed.py

It connects to the multi-stream socket (read) and the control socket (write) to show live per-stage waveforms and levels, switch the echo canceller model live and see an "echo removed" readout update in real time, tune reference delay/gain alignment with plain-language guidance, and save buffered streams plus an alignment report for offline analysis (consumed by tools/aec_compare.py, below). See testbed/README.md for the full panel reference.

Offline A/B Testing

tools/aec_compare.py runs the neural echo cancellers (dtln/nkf/hybrid) over a saved mic/reference pair -- either a session saved from the GUI testbed or your own WAV files -- and prints a metrics table (echo reduction, residual correlation, gain-jitter) so you can pick a backend by ear and by number instead of editing config.toml and restarting repeatedly:

python tools/aec_compare.py --session 20260622-092844
python tools/aec_compare.py --mic mic.wav --ref reference.wav --methods hybrid dtln

Quality Notes

This implementation is a local, testable daemon; the GUI testbed is optional tooling, not a dependency of the daemon itself. The written default canceller is hybrid_localvqe, but the daemon falls back to dependency-free nlms when neural extras or bundled models/libs are missing. Spectral suppression is off unless you turn it on. The test suite (see Running Tests) covers the daemon runtime, control socket, config loading, echo cancellation, delay alignment, and clock-drift sync, and runs in CI on every push and PR. For higher quality still, replace or extend ProcessingPipeline with WebRTC Audio Processing, DeepFilterNet, RNNoise, or Silero VAD -- the I/O, daemon, socket, FIFO, config, and service interfaces are already separated for that upgrade path.

Echo Cancellation (echo_canceller)

Real desktop-speaker echo reaches the mic delayed and filtered by the room, so a single per-frame gain cannot remove it. Several backends are available via processing.echo_canceller:

Neural (neural_aec.py, 16 kHz models wrapped behind the same per-frame interface; ~100 ms priming latency). These need the neural extras — pip install -e '.[neural]' (onnxruntime, soxr, torch) — plus the bundled GGML libs/model in lib/ and models/:

• "hybrid_localvqe" (default): the hybrid chain (NKF → DTLN) blended with LocalVQE (a small GGML model run via the bundled liblocalvqe.so) to scrub residual reference leakage. Best quality; localvqe_blend sets LocalVQE-vs-hybrid mix.
• "hybrid": NKF → DTLN. NKF removes the linear echo first so DTLN only suppresses the weak non-linear residual.
• "dtln": DTLN-aec (ONNX). Deep non-linear cancellation; can warble on speech.
• "nkf": NKF-AEC (neural Kalman, PyTorch). Linear and artifact-free but shallower.

Adaptive / classical (no extra deps):

• "nlms": a frequency-domain (overlap-save) multi-tap adaptive filter (aec.py) that learns the room impulse response. Clears ~90–95% of broadband echo. The automatic fallback if the neural extras/models are unavailable. Tunables: echo_filter_taps, echo_step_size (raise toward 0.6 to track clock drift faster), echo_filter_leak.
• "scalar": the legacy single-gain reducer; kept for back-compat.

If the configured neural backend can't load (missing extras, models, or libs), the daemon logs a warning and falls back to "nlms" rather than failing to start. Echo cancellation disabled in config uses a passthrough canceller so torch/ONNX/GGML are not loaded. make_echo_canceller is the seam for dropping in a native WebRTC/Speex AEC behind the same interface.

High-pass filtering (when enabled) runs on both the microphone and the system reference before echo cancellation, so DC and rumble do not consume adaptive-filter taps. Judge AEC on the after_echo stream, not VAD-gated cleaned_output.

Clock-Drift Synchronisation

The mic ADC and the Pulse/PipeWire monitor run on independent clocks. The daemon feeds the monitor reference through DriftCompensatingReference (sync.py), a continuous sample buffer with a proportional rate controller that resamples the reference onto the mic clock and holds latency constant. This replaces the old "reuse the last frame for 350 ms" pairing, which let latency drift unbounded and broke alignment. The controller's resample_ratio, latency_frames, and underruns are reported under config.reference_reader.sync in the diagnostics status.

Noise Suppression

The spectral suppressor uses weighted overlap-add (sqrt-Hann analysis/synthesis at 50% overlap, a COLA pair), so signal it passes is reconstructed without the per-frame amplitude warble of naive framing. It adds one frame (frame_ms) of latency.

Diagnostics

The daemon writes diagnostics that can be inspected without listening to the audio:

1. Latest status: /tmp/clean-speech-daemon-status.json
2. JSONL history: /tmp/clean-speech-daemon-diagnostics.jsonl
3. Optional per-stage WAV files: /tmp/clean-speech-daemon-stages

Print the current status:

clean-speech-daemon status

Watch status updates:

clean-speech-daemon status --watch

Useful fields:

1. mic.rms_dbfs shows microphone input level.
2. reference.rms_dbfs shows captured system playback reference level.
3. reference_correlation shows whether system playback resembles the mic signal.
4. output.rms_dbfs and output.clipped_pct show output level and clipping.
5. pipeline.vad_score shows speech activity confidence.
6. pipeline.echo_gain shows adaptive reference subtraction strength.
7. config shows which processing stages are enabled.
8. pipeline.reference_gain shows the adaptive gain applied to the system reference before echo cancellation.
9. pipeline.reference_delay_ms shows the current delay applied to the system reference before echo cancellation.
10. pipeline.reference_delay_correlation shows the correlation score used to select the delay.

Reference timing alignment is controlled by:

[processing]
enable_reference_delay_align = true
reference_delay_mode = "manual"
reference_delay_ms = 0
reference_max_delay_ms = 500
reference_delay_smoothing = 0.85

The raw system monitor remains available as system_reference; the delay-adjusted version is reference_aligned; the delay-adjusted and gain-adjusted version used by echo cancellation is reference_matched. For NLMS profiles, reference delay alignment and level matching are normally disabled, so these reference streams should match the raw reference.

Recommended timing workflow:

1. Open the GUI testbed while system audio is playing through speakers.
2. Press Save Streams.
3. Open the generated *-alignment_report.json in ~/clean-speech-recordings.
4. Copy offsets_vs_mic_raw.system_reference.suggested_reference_delay_ms into reference_delay_ms.
5. Restart with systemctl --user restart clean-speech-daemon.service.

Use reference_delay_mode = "manual" for repeatable testing. Use "auto" only when the reference correlation is strong and stable.

Reference auto-leveling is controlled by:

[processing]
enable_reference_level_match = true
reference_gain_min = 0.05
reference_gain_max = 20.0
reference_gain_smoothing = 0.92
reference_target_ratio = 1.0

To capture stage WAVs for offline inspection, edit ~/.config/clean-speech-daemon/config.toml:

[diagnostics]
enable_stage_wavs = true
stage_wav_dir = "/tmp/clean-speech-daemon-stages"

Then restart:

systemctl --user restart clean-speech-daemon.service

Stage files include mic_raw.wav, reference.wav, after_highpass.wav (pre-AEC), after_echo.wav, after_noise.wav, and output.wav when available.

The default config leaves the prototype spectral suppressor off (enable_noise_suppression and enable_speech_enhancement are false) because it can produce robotic artifacts. The known-good path is high-pass + echo cancellation + VAD. Re-enable the suppressor only when comparing stages:

[processing]
enable_noise_suppression = true
enable_speech_enhancement = true
noise_reduction = 1.15
spectral_floor = 0.08

Diagnostic Profiles

Ready-to-copy profiles are in profiles/:

1. profiles/passthrough.toml: raw microphone path, no cleanup. Use this to confirm I/O quality.
2. profiles/echo-vad.toml: high-pass, adaptive reference reduction, and VAD only. This is the current safest cleanup baseline.
3. profiles/nlms-aec.toml: adaptive (NLMS) echo cancellation tuned for desktop speakers. input.microphone is set to an example device name -- change it to yours (clean-speech-daemon devices) before using this profile.
4. profiles/spectral-experiment.toml: enables the prototype spectral suppressor with milder settings and stage WAVs.
5. profiles/stage-wav-debug.toml: writes per-stage WAV files without spectral suppression.

Apply a profile:

cp profiles/passthrough.toml ~/.config/clean-speech-daemon/config.toml
systemctl --user restart clean-speech-daemon.service
clean-speech-daemon status

Important Paths

1. Config: ~/.config/clean-speech-daemon/config.toml
2. Socket: /tmp/clean-speech-daemon.sock
3. Multi-stream socket: /tmp/clean-speech-daemon-streams.sock
4. Control socket: /tmp/clean-speech-daemon-control.sock
5. FIFO: /tmp/clean-speech-daemon.pcm
6. systemd unit: systemd/clean-speech-daemon.service

Running Tests

pip install -e '.[test]'
pytest -q

Neural-backend tests (DTLN/NKF/hybrid) self-skip if the neural extra isn't installed; install it too (pip install -e '.[neural]') for full coverage. The same suite runs in CI (see the badge above) on Python 3.11 and 3.12 for every push and pull request.

License

Released under the MIT License.