Skip to content

Tags: androidand/llama-swap

Tags

v211

Toggle v211's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
proxy: fix logger not checking matrix for processes

Fix matrix not being used to search for a logger causing /logs/stream/model_name to return an error

v210

Toggle v210's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
proxy: fix zero duration for non streaming responses (mostlygeek#723)

Updates mostlygeek#654

v209

Toggle v209's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Refactor Activity Page (mostlygeek#710)

- inference handles to store an activity record for all inference endpoints
- add path, status code, and content type to Activities page
- toggle on/off columns no Activities page 
- add configurable capture level for inference endpoints so large binary blobs are not stored in memory
- store captures in compressed binary format

v208

Toggle v208's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
ui-svelte: support reasoning and reasoning_content (mostlygeek#708)

Support `reasoning` v1/chat/completion delta that vLLM uses.

v207

Toggle v207's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
ui-svelte: fix histogram calculation (mostlygeek#695)

- Fix the histogram calculation to use server provided generation
tokens/second.
- Move histogram to Activities page where it can exist with the rest of
the token metrics

Fixes mostlygeek#681

v206

Toggle v206's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
ui-svelte: fix histogram calculation (mostlygeek#695)

- Fix the histogram calculation to use server provided generation
tokens/second.
- Move histogram to Activities page where it can exist with the rest of
the token metrics

Fixes mostlygeek#681

v205

Toggle v205's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
proxy: replace fsnotify with stat-poll watcher and add SIGHUP reload (m…

…ostlygeek#685)

The fsnotify-based config watcher does not work reliably when the config
file is bind-mounted into a Docker container as an individual file, and
mishandles k8s ConfigMap projections (atomically swapped symlinks).
Replace it with a small os.Stat-polling watcher and add SIGHUP as an
explicit reload signal.

- new proxy/configwatcher package: 2s os.Stat poller, follows symlinks,
  fires on mtime/size change and on missing -> present transitions
- SIGHUP triggers reload unconditionally (works without --watch-config)
  via the same ConfigFileChangedEvent pipeline so the UI sees identical
  state transitions
- watcher goroutine now exits cleanly on shutdown via a context
- drop github.com/fsnotify/fsnotify dependency

fixes mostlygeek#682

v204

Toggle v204's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
proxy: fix matrix race and process stop bug (mostlygeek#677)

- matrix.go change logic to consider any proxy.Process not in
StateStopped or StateShutdown
- process.StopImmediately, and Stop() which called it had a subtle bug
where it only handled state transitions from StateReady to
StateStopping. StateStarting -> StateStopping was ignored completely.

fix: mostlygeek#670

v203

Toggle v203's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
proxy: compress captures with zstd (mostlygeek#668)

The previous captures were saved uncompressed in memory. In agentic
workflows there can be many turns with each request containing the
previous context in the body with a lot of redundant data. Use zstd to
compress the request and response data before keeping a copy of memory.

Results: 

- Average Percentage Saved: 73.19%
- Average Compression Factor: ~6.77:1

v202

Toggle v202's commit message

Verified

This commit was signed with the committer’s verified signature.
mostlygeek Benson Wong
docs: update configuration.md for matrix