1. A concise explanation of the problem you’re experiencing.
I have forked the cesium repo, pulled it and started coding on a new branch. Ran all the tests, fail. Removed all changes and switched back to master branch. Tests fails again. Always a random number of tests fail.
TOTAL: 87 FAILED, 10329 SUCCESS
[15:15:58] Finished ‘test’ after 4.5 min
This is from the karma tests. Similar results with Jasmine. The quicker the tests finishes less tests seems to fail.
Is there minimum system requirements to run the tests? Using a MacBook Pro Retina 13" i5 8GB Ram.
2. A minimal code example. If you’ve found a bug, this helps us reproduce and repair it.
It definitely shouldn’t be failing on master. I have been doing some work recently to speed up some tests, some of which would just fail on lower end systems due to time out (see https://github.com/AnalyticalGraphicsInc/cesium/pull/7269).
How many tests are failing and which ones are they? Are they timeouts or something else?
Something else I discovered is in some files, if one test fails due to timeout, it doesn’t clean up resources and subsequent tests fail.
Yes, most if not all fails are timeouts.
It has failed from 8 to 106 tests. I’ve run the jasmine tests 4 times and karma 3. It varies.
I would try to isolate it, pick a Spec file that has failing tests, run it individually, then isolate it to a particular test. How long does it take in total to run all these tests for you?
If you’re working on a PR one thing you could do is open the PR to get feedback even if you’re not sure if all tests are passing, and I believe the CI should run the tests in the cloud and report back the results. And the reviewer(s) can also run the tests locally as well.
Thanks, I made a pull request and it seems like it passed the tests there.
By increasing the timeout I managed to get down the failed tests to seven, and all those were timeouts again. But no matter how much I increased the timeout after that they still kept failing. The tests took around four minutes most times.
Are these final 7 all in one file or are they across different files? Can you list them here?