Clear and concise description of the problem
At MUI we're in the midst of migrating our mocha setup to vitest. We're noticing big performance penalty compared to mocha. To start with a few numbers:
baseline mocha:
8244 passing (29s)
1225 pending
NX Successfully ran target nx_test_unit for project @mui/monorepo and 1 task it depends on (40s)
vitest run:
Test Files 523 passed | 5 skipped (528)
Tests 8251 passed | 1227 skipped (9478)
Start at 17:43:38
Duration 87.82s (transform 24.78s, setup 86.00s, collect 199.48s, tests 97.51s, environment 112.82s, prepare 32.60s)
vitest run --pool threads:
Test Files 523 passed | 5 skipped (528)
Tests 8251 passed | 1227 skipped (9478)
Start at 17:47:43
Duration 75.84s (transform 22.04s, setup 80.35s, collect 151.44s, tests 101.84s, environment 106.17s, prepare 31.04s)
vitest run --pool vmThreads:
Test Files 4 failed | 519 passed | 5 skipped (528)
Tests 7 failed | 8244 passed | 1227 skipped (9478)
Start at 17:49:18
Duration 66.08s (transform 26.45s, setup 129.90s, collect 174.66s, tests 104.33s, environment 6.03s, prepare 41.65s)
vitest run --no-isolate --no-file-parallelism
Test Files 523 passed | 5 skipped (528)
Tests 8251 passed | 1227 skipped (9478)
Start at 17:45:35
Duration 47.16s (transform 6.87s, setup 2.01s, collect 12.47s, tests 24.25s, environment 3.44s, prepare 667ms)
note: vitest picks up 9 more tests, this is caused by a bug we found in our setup
As you can see from the numbers, we get a huge benefit from running the tests in a single process, sequentially. It's the only mode which comes close to our mocha baseline in terms of performance. It speaks for itself that we want to use this mode. The main problem we see is that it looks like the jsdom environment is recreated for each test file. This causes issues with @testing-library/dom when using their global screen and @testing-library/user-event. These libraries seem to not work well when document and window are reassigned.
Suggested solution
Either avoid recreating jsdom for each suite when vitest runs in --no-isolate --no-file-parallelism or a new environment that acts that way. Or make the necessary hooks available so we can creat such an environment ourselves
Alternative
- Don't migrate away from mocha
- Take the performance hit on our CI
Additional context
No response
Validations
Clear and concise description of the problem
At MUI we're in the midst of migrating our mocha setup to vitest. We're noticing big performance penalty compared to mocha. To start with a few numbers:
baseline
mocha:vitest run:vitest run --pool threads:vitest run --pool vmThreads:vitest run --no-isolate --no-file-parallelismnote: vitest picks up 9 more tests, this is caused by a bug we found in our setup
As you can see from the numbers, we get a huge benefit from running the tests in a single process, sequentially. It's the only mode which comes close to our mocha baseline in terms of performance. It speaks for itself that we want to use this mode. The main problem we see is that it looks like the jsdom environment is recreated for each test file. This causes issues with
@testing-library/domwhen using their globalscreenand@testing-library/user-event. These libraries seem to not work well whendocumentandwindoware reassigned.Suggested solution
Either avoid recreating jsdom for each suite when vitest runs in
--no-isolate --no-file-parallelismor a new environment that acts that way. Or make the necessary hooks available so we can creat such an environment ourselvesAlternative
Additional context
No response
Validations