Add notebook-driven Brev launchable for Cosmos3 VANTAGE evaluation - #382

Summary

Notebook-driven Brev launchable for Cosmos3-Nano and VANTAGE evaluation, plus a repeatable setup for an existing SSH-accessible LaunchPad host.

• Pinned Nano checkpoint and vLLM image; model API has no published host port. setup.sh --brev binds authenticated Jupyter to Brev's private mesh interface, preserving provider login and Jupyter-token authentication. Non-Brev setup defaults to loopback.
• The notebook prepares public data, checks the endpoint, runs five real VQA questions, verifies predictions/canonical IDs, and keeps the full eight-task run opt-in.
• The public Nano/VQA smoke needs no Hugging Face token. Gated assets, including full SOT/SmartSpaces preparation, still require the appropriate token and accepted access terms.
• Public and internal harnesses, Python environments, and dataset roots remain separate. Public labels are withheld; public predictions are not locally scored as ground truth. Cross-harness/reference-score parity is not claimed.
• LaunchPad setup reuses the original controller and native dataset/scorer. Preparation is separate from credential-free, network-isolated inference. A resumable, data-only workstation-staging route supports hosts that cannot reach PDX.
• Reject empty/API/video-load failures and incorrect smoke row counts. Public subprocesses use a headless plotting backend rather than inheriting Jupyter's inline backend.

Verification exercised

• Brev, commit 23cfa62968f0a395481b52c4fd91bae9ce1106dd: actual authenticated Jupyter UI Run All on one 96 GB RTX PRO 6000 Blackwell Server Edition; prepared 1,195 public VQA questions/282 videos; produced exactly five valid native predictions and five unique canonical IDs. Full-eight execution remained disabled. Screenshots, executed notebook, and provenance retained privately. Test VM confirmed stopped.
• LaunchPad, commit 0297efb4ee50584055f5ba4f95293d433f71c0d0: actual single-H200 lifecycle and endpoint checks; staged 1,195 native questions/282 videos using the original converter/preparer; verified all referenced video paths. Cached preparation, five-question inference, and native local scoring passed without any remote AWS profile. Final source digest and acceptance/lifecycle evidence retained privately.
• Exercised repeated up, status/check, stop/resume, and refusal to overwrite existing results. Final project endpoint stopped; all eight GPUs free; original Jupyter/Traefik containers unchanged. The existing LaunchPad instance remains allocated.
• Seven artifact-safety regression tests passed on native Linux. Actual invalid public prediction artifacts were accepted before the verifier fix and rejected afterward. Actual CLI source isolation rejected .envrc and escaping symlinks before state creation.
• Real notebook execution reproduced and then verified fixes for private-proxy connection refusal and inherited inline-backend failure; these were not CLI-only simulations.

Run / review entry points

bash evaluation/cosmos3/reasoner/vlmevalkit/launchable/setup.sh --brev

• Brev setup and notebook workflow
• LaunchPad lifecycle, staging, smoke, and scoring

Tokens, private dataset media/labels, predictions, local state, and operational evidence are not included in this PR. Temporary copied AWS credentials were removed; the user's existing local profile was not modified by cleanup.

Draft / remaining release gates

• Brev validation used the same stopped/resumed VM and retained caches. An uninterrupted fresh-VM cold deployment has not been rerun.
• Full eight-task inference, official submission acceptance, and published-reference/leaderboard parity remain unverified. A five-question native score is only an integration smoke, not a benchmark result.
• Repository placement still needs Dan/maintainer confirmation: this draft targets NVIDIA/cosmos, while new-code guidance points to Cosmos Framework. Do not merge before that decision and applicable data/licensing/scoring approvals.
• No public launchable release or PR merge performed. Observed hardware is a tested smoke configuration, not a minimum-sizing or full-workload compatibility guarantee.