Repository navigation
How many videos need to be tested in i2v mode? #28
Description
First of all, thank you for your work. We noticed that there are only 198 images under the switch directory, while there are 396 entries in the CSV file. Could you tell us how many videos need to be tested in total? Is it acceptable to use only the prompts of the first 198 entries?
Activity
rgeirhos commentedon Dec 15, 2025
Hi @That-boy, that's a great question! If you'd like to test a model, you'll only need to use the first 198 entries. In the csv, they're marked as
take-1.For context, the remaining videos are called
take-2- i.e., they correspond to the same real scenarios which we recorded a second time. This is done to to estimate the physical variability from real videos, which becomes an upper bound against which scores are normalized - but there's no need to sample model generations for those. I'll update the README to make this more clear.Update README.md with take-1 vs take-2 description (#28)
rgeirhos commentedon Dec 15, 2025
(deleted off-topic comment, please open standalone issue if there's a question that's not related to this issue)
Update README.md with take-1 vs take-2 description (google-deepmind#28)