Skip to content

fix: Scope offline server permission checks to the requested project - #6803

Open
hiyufan wants to merge 2 commits into
feast-dev:masterfrom
hiyufan:fix/offline-server-project-scoped-permissions
Open

hiyufan wants to merge 2 commits into
feast-dev:masterfrom
hiyufan:fix/offline-server-project-scoped-permissions

Conversation

@hiyufan

@hiyufan hiyufan commented Sep 1, 2026

Copy link
Copy Markdown

What this PR does / why we need it:

SecurityManager.permissions is loaded per project, and every request-facing path is
supposed to bind it to the project the request names before any permission check runs.
The registry server does this — api/registry/rest/rest_utils.py,
permissions/server/rest.py, permissions/server/grpc.py and registry_server.py all
call set_current_project(...) and reset it afterwards.

The Arrow Flight offline server does not. Neither of its two dispatchers —
OfflineServer.do_get and OfflineServer._call_api — touches the current project, so
every assert_permissions call in the handlers below them is evaluated against the
Permission list of whatever project the SecurityManager was constructed from, which
is the project in the server's own feature_store.yaml.

An offline server that serves more than one project therefore enforces its home
project's policy on every request. A role granted only in that project reaches the other
projects' data sources, feature views and saved datasets, and the policies those projects
defined for themselves are never loaded. get_historical_features already asserts that
project is mandatory in its command and uses it to resolve feature views, so the
per-request project was available at the dispatcher the whole time — it just never
reached the permission layer.

The fix

Bind the SecurityManager to command["project"] around both dispatchers and reset it
in a finally, following the interceptor in permissions/server/grpc.py:

project_token = self.store.set_current_project(command.get("project"))
try:
    ...
finally:
    self.store.reset_current_project(project_token)

A command that carries no project passes None. SecurityManager.permissions already
falls back from None to the server's own project, so those paths — offline_write_batch
and write_logged_features, which read self.store.config.project — behave exactly as
before.

Which issue(s) this PR fixes:

Relates to #6784 (advisory GHSA-5px7-7gwg-6g93).

That issue is written against the registry server, and the registry paths already scope
the lookup on master. This is the same defect on the Arrow Flight offline server, which
was not covered. I have left it as "relates to" rather than "fixes" so you can decide
whether it closes the issue or belongs as a follow-up.

Checks

  • I've made sure the tests are passing.
  • My commits are signed off (git commit -s)
  • My PR title follows conventional commits format

Testing Strategy

  • Unit tests
  • Integration tests
  • Manual tests

Four tests in sdk/python/tests/unit/test_offline_server.py:

  • test_do_get_scopes_permissions_to_the_requested_project
  • test_call_api_scopes_permissions_to_the_requested_project
  • test_call_api_resets_the_project_when_the_handler_raises — a failed request must not
    leave its project bound for the next one
  • test_call_api_without_a_project_leaves_the_lookup_unchanged — pins the None
    fallback, so the paths that do not send a project keep their current behaviour

All four fail on the base revision and pass with the change:

# base revision, new tests only
4 failed

# this branch
$ pytest sdk/python/tests/unit/test_offline_server.py
14 passed

$ pytest sdk/python/tests/unit/test_offline_server.py sdk/python/tests/unit/permissions/
343 passed

pre-commit passes on both changed files.

Misc

The scope here is deliberately narrow: it moves the offline server onto the same
per-request project binding the registry server already uses, and changes nothing about
how permissions themselves are defined or evaluated. If you would rather see the binding
lifted into a shared Flight middleware — the way permissions/server/grpc.py does it for
gRPC, instead of at each dispatcher — I am happy to rework it that way.


This change was developed with AI assistance. The test runs and the base-revision
comparison reported above were executed locally against this branch as submitted.

@hiyufan
hiyufan requested a review from a team as a code owner September 1, 2026 05:06
The registry server's REST and gRPC paths bind the SecurityManager to the
project each request names, so its permission list is loaded for that project.
The Arrow Flight offline server never did: both of its dispatchers ran the
handlers, and therefore `assert_permissions`, with whatever project the
SecurityManager was constructed from — the project in the server's own
`feature_store.yaml`.

An offline server that serves more than one project consequently enforced its
home project's `Permission` list against every request, so a role granted only
in that project reached the other projects' data sources and feature views,
while the policies those projects defined for themselves were never consulted.
`get_historical_features` already requires a `project` in its command, so the
per-request project was available all along.

Bind the SecurityManager to `command["project"]` around both dispatchers and
reset it afterwards, following the interceptor in permissions/server/grpc.py. A
command that carries no project passes `None`, which the SecurityManager falls
back from to the server's own project, leaving those paths unchanged.

Fixes feast-dev#6784

Signed-off-by: Chen Yufan <[email protected]>
@hiyufan
hiyufan force-pushed the fix/offline-server-project-scoped-permissions branch from 292fa1c to bb18433 Compare September 14, 2026 09:05
@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 47.08%. Comparing base (81e1546) to head (bb18433).
⚠️ Report is 3 commits behind head on master.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #6803   +/-   ##
=======================================
  Coverage   47.08%   47.08%           
=======================================
  Files         419      419           
  Lines       51877    51877           
  Branches     7525     7525           
=======================================
  Hits        24428    24428           
  Misses      25700    25700           
  Partials     1749     1749           
Flag Coverage Δ *Carryforward flag
go-feature-server 30.58% <ø> (ø)
python-unit 48.39% <ø> (ø) Carriedforward from 81e1546

*This pull request uses carry forward flags. Click here to find out more.

Files with missing lines Coverage Δ
sdk/python/feast/offline_server.py 29.39% <ø> (ø)

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b7a8928...bb18433. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ntkathole

ntkathole commented Sep 16, 2026

Copy link
Copy Markdown
Member

@hiyufan The Arrow Flight offline server never scopes SecurityManager to the requested project, so permission checks always load the server's home project's policy. This is not a critical issue in practice (offline servers are typically deployed one-per-project, and only get_historical_features even accepts a client-supplied project), but I would still consider this a valid defense-in-depth fix.

However, the current implementation has a bug that needs to be fixed before merging:

self.store.set_current_project() sets the wrong ContextVar. FeatureStore._current_project and SecurityManager._current_project are two independent ContextVar objects — setting the former has no effect on SecurityManager.permissions, which is where the permission list is loaded from. The fix needs to call get_security_manager().set_current_project() instead, which is the same pattern used in the three working registry paths:

  • permissions/server/grpc.py:39token = sm.set_current_project(...)
  • permissions/server/rest.py:60project_token = sm.set_current_project(project)
  • api/registry/rest/rest_utils.py:35project_token = sm.set_current_project(...)

Something like this in both _call_api and do_get:

from feast.permissions.security_manager import get_security_manager

sm = get_security_manager()
sm_token = None
if sm is not None:
    sm_token = sm.set_current_project(command.get("project"))
try:
    # ... existing handler logic ...
finally:
    if sm is not None and sm_token is not None:
        sm.reset_current_project(sm_token)

You can keep self.store.set_current_project() alongside for data-access correctness (it affects FeatureStore.project used by methods like get_data_source), but it alone does not fix the permission scoping.

The tests should also be updated to assert that the SecurityManager's project was set, not just the FeatureStore's — currently they only verify store.set_current_project was called, which doesn't prove the fix works.

…roject

Review caught that the previous commit did not do what it claimed. `FeatureStore`
and `SecurityManager` each own a separate `ContextVar`, and `SecurityManager.
permissions` reads its own:

    project = self._current_project.get() or self._project
    return self._registry.list_permissions(project=project)

so `self.store.set_current_project(...)` never reached the permission lookup, and
every check still ran against the project the server was started from. The store
binding is kept -- it is what `FeatureStore.project` resolves objects through, which
`get_data_source` and friends need -- but the security manager has to be bound too,
the way `permissions/server/grpc.py`, `permissions/server/rest.py` and
`api/registry/rest/rest_utils.py` already do it.

The tests were the reason this got through: they asserted that
`store.set_current_project` had been called on a `MagicMock`, which is true either
way and proves nothing about the check. They now install a real `SecurityManager`
whose registry records the project each permission lookup resolves to, and assert on
that -- inside the handler it must be the requested project, and after the dispatcher
returns it must be the server's own again, so the reset is covered by the same
assertion. Both fail against the previous commit:

    assert ['the_project_the_server_was_started_from'] == ['project_b', ...]

Also covers a deployment with no SecurityManager at all, which the dispatchers have
to keep serving.

Fixes feast-dev#6784

Signed-off-by: Chen Yufan <[email protected]>
Co-Authored-By: Claude Opus 5 <[email protected]>
@hiyufan

hiyufan commented Sep 16, 2026

Copy link
Copy Markdown
Author

You are right, and thank you for reading the code rather than the description — the description said the right thing and the code did not do it. Fixed in 152c0936.

I confirmed it before changing anything:

feature_store.py:232   self._current_project: ContextVar[...] = ContextVar("current_project", ...)
security_manager.py:34 self._current_project: ContextVar[...] = ContextVar("current_project", ...)

security_manager.py:60  @property
                        def permissions(self) -> list[Permission]:
security_manager.py:65      project = self._current_project.get() or self._project
                            return self._registry.list_permissions(project=project)

Two independent ContextVars, and permissions reads the security manager's. self.store.set_current_project(...) never reached the lookup, so every check still ran against the server's home project — the PR fixed nothing it claimed to fix.

Both dispatchers now bind the security manager, following the three paths you pointed at, and keep the store binding for the reason you gave:

project = command.get("project")
sm = get_security_manager()
sm_token = sm.set_current_project(project) if sm is not None else None
project_token = self.store.set_current_project(project)
try:
    ...
finally:
    self.store.reset_current_project(project_token)
    if sm is not None and sm_token is not None:
        sm.reset_current_project(sm_token)

The tests were the real problem

You are right that asserting store.set_current_project was called proves nothing — it is a MagicMock, so it passes whichever variable the code sets. That is exactly how this got past me.

They now install a real SecurityManager whose registry records the project every permission lookup resolves to, and assert on that list:

registry.list_permissions.side_effect = lambda project=None, **kw: (seen.append(project) or [])
sm = SecurityManager(project=_SERVER_HOME_PROJECT, registry=registry)
...
    OfflineServer._call_api(server, command["api"], command, key)   # handler reads sm.permissions
    sm.permissions                                                  # after the dispatcher returns

assert seen == ["project_b", _SERVER_HOME_PROJECT]

One assertion covers both halves: inside the handler the check has to resolve to the requested project, and afterwards it has to be back to the server's own, so the reset is proved rather than assumed. Against the previous commit:

FAILED test_do_get_scopes_the_permission_lookup_to_the_requested_project
FAILED test_call_api_scopes_the_permission_lookup_to_the_requested_project
E   assert ['the_project_the_server_was_started_from'] == ['project_b', 'the_project_the_server_was_started_from']

Added test_dispatchers_work_without_a_security_manager as well, since an unauthenticated deployment has none and the dispatchers still have to serve.

15 passed on sdk/python/tests/unit/test_offline_server.py, ruff check and ruff format --check clean.

Unrelated: the red CI

All five failing jobs on the previous head died in dependency installation, before running anything:

error: Failed to download `torchvision==0.28.0+cpu`
make: *** [Makefile:110: install-python-dependencies-ci] Error 1

lint-python, the three unit-test jobs and smoke-test-python all have that same line. Nothing to do with this branch, but worth knowing before you read the red.

And on your framing — agreed that this is defense in depth rather than critical, given one-server-per-project deployments. I would rather it be described that way in the changelog than oversold.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants