Alright, I finished testing 1 vs 10 seeds
@L.M.Sherlock @vaibhav @sorata
For each collection, RMSE(10 seeds) is calculated simply as min(RMSE(seed 1), RMSE(seed 2)…RMSE(seed 10)).
I deliberately chose 100 collections with a small numer of reviews and 100 collections with a large number of reviews.
Difference in RMSE and the number of reviews.
Distribution of differences
Improvement and the number of seeds used
As you can see, it is absolutely not worth it even for people with <5k reviews (for other people the average is even lower).
Note that this is different from what I proposed here. It’s not the same idea.