Repository navigation
Tags: androidand/llama-swap
Tags
proxy: fix zero duration for non streaming responses (mostlygeek#723) Updates mostlygeek#654
Refactor Activity Page (mostlygeek#710) - inference handles to store an activity record for all inference endpoints - add path, status code, and content type to Activities page - toggle on/off columns no Activities page - add configurable capture level for inference endpoints so large binary blobs are not stored in memory - store captures in compressed binary format
ui-svelte: support reasoning and reasoning_content (mostlygeek#708) Support `reasoning` v1/chat/completion delta that vLLM uses.
ui-svelte: fix histogram calculation (mostlygeek#695) - Fix the histogram calculation to use server provided generation tokens/second. - Move histogram to Activities page where it can exist with the rest of the token metrics Fixes mostlygeek#681
ui-svelte: fix histogram calculation (mostlygeek#695) - Fix the histogram calculation to use server provided generation tokens/second. - Move histogram to Activities page where it can exist with the rest of the token metrics Fixes mostlygeek#681
proxy: replace fsnotify with stat-poll watcher and add SIGHUP reload (m… …ostlygeek#685) The fsnotify-based config watcher does not work reliably when the config file is bind-mounted into a Docker container as an individual file, and mishandles k8s ConfigMap projections (atomically swapped symlinks). Replace it with a small os.Stat-polling watcher and add SIGHUP as an explicit reload signal. - new proxy/configwatcher package: 2s os.Stat poller, follows symlinks, fires on mtime/size change and on missing -> present transitions - SIGHUP triggers reload unconditionally (works without --watch-config) via the same ConfigFileChangedEvent pipeline so the UI sees identical state transitions - watcher goroutine now exits cleanly on shutdown via a context - drop github.com/fsnotify/fsnotify dependency fixes mostlygeek#682
proxy: fix matrix race and process stop bug (mostlygeek#677) - matrix.go change logic to consider any proxy.Process not in StateStopped or StateShutdown - process.StopImmediately, and Stop() which called it had a subtle bug where it only handled state transitions from StateReady to StateStopping. StateStarting -> StateStopping was ignored completely. fix: mostlygeek#670
proxy: compress captures with zstd (mostlygeek#668) The previous captures were saved uncompressed in memory. In agentic workflows there can be many turns with each request containing the previous context in the body with a lot of redundant data. Use zstd to compress the request and response data before keeping a copy of memory. Results: - Average Percentage Saved: 73.19% - Average Compression Factor: ~6.77:1
PreviousNext