This is due to the browser still referencing the file by url. Browser may have differents behaviour, but for example in the case of Chrome at least, the browser does not try to request the file (more precisely, do not even look in its disk cache).

More info & insights: https://developers.google.com/speed/docs/insights/LeverageBrowserCaching

Several solutions are possible, I guess it needs to be discussed more thoroughly.