The cache can be a problem for testing because, if a test's inputs are unchanging, subsequent runs of the test may simply mean that the previous result is retrieved from the cache and so the system is not tested in the expected way. In order to defeat the cache, each test will by default have an extra key inserted into it called "no_cache", which will be set to a random string. This should mean that the system-level cache views each test request as unique and will not serve cached files.
It may be that you do want to test the cache, in which case using the "-c" option with bin/runtests will mean the no_cache key is not inserted. The no_cache will also not be inserted if it's already present in the request.