Thanks to visit codestin.com
Credit goes to github.com

Skip to content

webpack does not generate always unique chunk hashes #1155

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
XVincentX opened this issue Jun 10, 2015 · 42 comments
Closed

webpack does not generate always unique chunk hashes #1155

XVincentX opened this issue Jun 10, 2015 · 42 comments

Comments

@XVincentX
Copy link

Hello,
we have noted that sometimes webpack generates the same hash for a chunk, even if the file content has been changed.

Due to this, changes sometimes are not propagated, we experience an error similar to #959 and we're forced to clear the cache/CDN to make the things work.

We are using the following plugins in the given order:

ExtractTextPlugin
CommonsChunkPlugin
OccurenceOrderPlugin
WebpackAssetsMapPlugin //custom plugin, writes the json stats file to the disk
DeduplePlugin
UglifyJsPlugin

Let me know if you need further details,
V.

@Baggz
Copy link

Baggz commented Jun 10, 2015

👍

@sokra
Copy link
Member

sokra commented Jun 15, 2015

Let me know if you need further details,

Yes, it would be nice if you could send me two different files which identical hash so I could reproduce it.

@sokra
Copy link
Member

sokra commented Jul 1, 2015

Should be fixed.

@XVincentX
Copy link
Author

Good news. Could you please share with us what was wrong?
Furthermore, I cannot see any new version for webpack. Is there already a release with the fix?

@sokra

@sokra
Copy link
Member

sokra commented Jul 7, 2015

The hashes of referenced chunks were not included in the chunkhash of the entry chunks.

A entry chunk include a list of urls of all additional chunks that can be loaded. Depending on the configuration there is a flag for each chunk whether it need to use to global hash or the chunk hash for caching. webpack now throws an error when using [chunkhash] for a chunk which requires a global hash (i. e. with HMR enabled). In addition to that [chunkhash] and [name] in output.chunkFilename no longer require a global hash for the entry chunk, but influence the chunkhash instead.

The first change ensures that there is an error when [chunkhash] won't be unique.

The second change ensures that [chunkhash] and [name] can be used for chunk filenames.

@XVincentX
Copy link
Author

@sokra That's great, but still I cannot find a new release or something like that to install/update.
Am I missing something to use this feature??

@XVincentX
Copy link
Author

@sokra Ping. I am not able to find the fix you're talking about.
I have also tested the thing on my environment and the issue persists.

@dmitry
Copy link

dmitry commented Sep 25, 2015

I have the same problem. Files are identical (looking through md5sum ./dist/*) but hash is different on different machines, but on the same machine hash stays the same.

v1.12.2

@bebraw bebraw added the bug label Nov 14, 2015
@viz
Copy link

viz commented Nov 18, 2015

I'm having the same problem as @dmitry both hash and chunkhash are different on different machines for the same git commit.

@dmitry
Copy link

dmitry commented Nov 18, 2015

@viz looks like the absolute path is the root of the issue.

@viz
Copy link

viz commented Nov 18, 2015

@dmitry - have you been able to fix it or found a work-around? My problem is that I need builds on CircleCI and Heroku to have the same hash...

@dmitry
Copy link

dmitry commented Nov 19, 2015

@viz at the moment I fixed that with webpack-md5-hash but I don't like this approach. For more look to #1479

@viz
Copy link

viz commented Nov 19, 2015

hmm, I tried that but I still get different hashes on Circle versus Heroku. Perhaps I'm doing something wrong - I am new to webpack, so I'm not completely clear on the whole config. Like you, I started with react-redux-universal-hot-example but adding webpack-md5-hash as per the readme doesn't fix it for me...

@XVincentX
Copy link
Author

I am still getting the issue with latest version. I am afraid this one wasn't fixed.

@dmitry
Copy link

dmitry commented Nov 19, 2015

I found the issue is appears when imported library imports at least one file in itself. For example, I've tried remove all the react imports (requires) - hash was generated identically, then I've tried to import one library in a react library - it started to generate hash differently. It depends on absolute path, because I was able to reproduce that issue on the same machine but with different root paths. If you want to see all the issues with related problem, check out this issue: #1479

@viz
Copy link

viz commented Nov 19, 2015

yes, I found that one as well - unfortunately seems it is still an outstanding bug. If I had the skill to develop a patch I would, but getting into the guts of webpack is a bit beyond me at the moment. I'll have to look for another solution..

@XVincentX
Copy link
Author

@dmitry Are you using a particular webpack version? I am getting an error using latest version (1.12.6) and webpack-dev-server as well (1.12.1)

@dmitry
Copy link

dmitry commented Nov 19, 2015

@XVincentX here we are using webpack 1.12.2, webpack-dev-server version doesn't matter, because it only starts in development, but we have an issue with the production bundle too.

@viz
Copy link

viz commented Nov 19, 2015

1.12.2 here as well

Sent from my iPad

On 19 Nov 2015, at 10:40 PM, Dmitry Polushkin [email protected] wrote:

@XVincentX here we are using webpack 1.12.2


Reply to this email directly or view it on GitHub.

@XVincentX
Copy link
Author

@dmitry
I do not get if your workaround is working or not...sorry :(

@dmitry
Copy link

dmitry commented Nov 19, 2015

For me adding the new WebpackMd5Hash() (webpack-md5-hash) inside the webpack's plugins is working.

@bradrobertson
Copy link

@dmitry are you using common chunks as well and it works? We're finding that different builds can generate different chunk references within the common chunk file itself as mentioned here

@dmitry
Copy link

dmitry commented Nov 20, 2015

@bradrobertson no common chunks plugin used in our webpack config.

@XVincentX
Copy link
Author

I am using them. Perhaps it's the reason for the error I'm getting.

@dtothefp
Copy link

Looks like this isn't fixed as of [email protected] and is causing us a lot of problems with subresource integrity because cached contents are getting served that don't have a matching integrity sha. @bebraw is a plan for a fix in the works for webpack 2?

@bebraw
Copy link
Contributor

bebraw commented Oct 19, 2016

@dtothefp I tagged the issue under webpack-2 label so we won't forget. Thanks for bringing this up.

@dtothefp
Copy link

@bebraw thanks...this is a very bad bug because if you are using subresource integrity and filenames overlap sometimes the CDN will serve previously cached content that does not match the subresource integrity hash for the current bundle, and therefore the JS/CSS generated through Webpack will not load

@jhnns
Copy link
Member

jhnns commented Oct 19, 2016

Yep, that sounds pretty bad. Do you have a way/repo to reproduce it reliably? That would be a big help.

@dtothefp
Copy link

dtothefp commented Oct 19, 2016

@jhnns essentially it took months of build up in S3 for this to start happening. I'm using [chunkhash]-[name] for CSS and JS compilation. We also switched from Webpack 1.x to Webpack 2.1.0-beta.22, not sure if that somehow could have allowed an overlapping of file name hashes? I'm using the SRI stats plugin https://github.com/mikechau/sri-stats-webpack-plugin to generate the subresource integrity stats.

Essentially here is the flow that results in failure that took us a while to figure out what was going on:

  • webpack generates CSS/JS files in CI with the hashed filename
  • SRI plugin generates stats for sha of the file that can be included on the integrity attribute for a script or link tag
  • CSS/JS assets get uploaded to S3, if a previous filename exists on S3 it is overwritten by uploaded content
  • template is generated on server, static template generator, or lambda using the generated hashed filenames and SRI to make the script and link
  • if there was a filename collision on S3 due to improper hashing by webpack then cached content will potentially be served and this content will NOT match the SRI integrity attribute sha
  • everything breaks and people get 😢 / 😠

I wish I could reproduce when / how this happens but it seems to start showing up lately across many of our applications through many Travis CI builds that upload static assets to S3. I resorted to using the md5 webpack plugin and seems to be working for now.

@jhnns
Copy link
Member

jhnns commented Oct 21, 2016

As far as I'm understanding it, we have two issues here:

  • Some people are complaining about different hashes although the file content is the same @dmitry
  • Some people are complaining about the same hashes although the file content has changed @dtothefp

Please note that webpack hashes the final result (chunks and assets), which means that loaders and plugins have been executed before. Thus it is possible that you don't get the same hashes for the same file content, because you might have installed different versions of loaders and plugins (at least when using npm).

Same goes the other way round. For instance, if you're using tree-shaking, you might get the same hashes for different file contents if unused code is removed.

It would be very helpful if you could reproduce the error without any plugin. @dtothefp maybe it's an error with the sri-stats-webpack-plugin? Try to remove as much as the possible so that we can locate the error.

@dmitry
Copy link

dmitry commented Oct 22, 2016

@jhnns I've reproduced such issue with simple code: just webpack settings without any plugins and react library (could be used any other module, that loads other modules in it). I believe I've explained this year ago somewhere here: #1479

@Ciantic
Copy link

Ciantic commented Nov 2, 2016

I'm using webpack 2, and it's so bad I always need to go fiddle with the urls. I've thrown in all the things: [name].js?[chunkhash][id][hash] but it still isn't unique always. I think I'll just start to write the darn date in there instead.

@dtothefp
Copy link

@jhnns @bebraw any word on this? Looks like Webpack 2 was released without a fix

@bebraw
Copy link
Contributor

bebraw commented Jan 25, 2017

@dtothefp Nope. It's a huge and complicated issue and I don't think there's a simple fix in sight. Reading back, a minimal case to showcase the problem would likely help.

@dtothefp
Copy link

@bebraw I just quickly put together this repo https://github.com/dtothefp/webpack-broken-sha. Made a branch where I changed JS slightly and got a different sha so ¯\_(ツ)_/¯.

What we saw in the past is that this happened overtime, some deploys with differing content had the same hash and some had different. The only way we knew is when subresource integrity broke because it got cached content (i.e. a deploy with changed content had the same hash as a previous deploy and the CDN served the previous deploys content).

If anyone wants to play around with the repo above to try and reproduce feel free. It's just a basic setup with React and CSS modules.

@le12484
Copy link

le12484 commented Jul 24, 2017

Anyone using https://github.com/scinos/webpack-plugin-hash-output plugin to work around this problem?

It generate the hash from the final content files.

@jakepeyser
Copy link

@le12484 I am using https://github.com/erm0l0v/webpack-md5-hash and it has been working fine for me these past few months.

@wzup
Copy link

wzup commented Oct 13, 2017

The same hash in filename for different entry files with different contents. Why? Is it normal behavior?
Doesn't matter if CommonsChunkPlugin is used or not, the result hash in filenames is the same.

child.js

'use strict';
let child = (params) => {
    return params;
}

export default child;

fileA.js

'use strict';
const React = require('react');
import child from './child';

let a = 10;
console.log(a);

child({a: 10})

fileB.js

'use strict';
import child from './child';

webpack config.js

'use strict';

const NODE_ENV = process.env.NODE_ENV || 'development';
const path = require('path');
const webpack = require('webpack');

module.exports = {
    context: path.resolve(__dirname, '..'),
    entry: {
        'public/fileA': './src/fileA',
        'public/fileB': './src/fileB',
    },
    output: {
        filename: '[name]-[hash].js',
        // path: path.resolve(__dirname, '..', 'dist'),
    },
    plugins: [
        // new webpack.optimize.CommonsChunkPlugin({
        //     name: 'commons',
        //     filename: "public/commons-[hash].js",
        //     minChunks: 0,
        //     // chunks: ["pageA", "pageB"],
        // })
    ]
}

"webpack": "^3.7.1"

@mpotomin
Copy link

mpotomin commented Oct 25, 2017

Switching from [chunkhash] to [contenthash] solved the problem with the same name of the css file with the different content.

new ExtractTextPlugin({ filename: "css/[name].[chunkhash].css" })
vs
new ExtractTextPlugin({ filename: "css/[name].[contenthash].css" })

@scinos
Copy link

scinos commented Nov 11, 2017

As @le12484 recommended, I wrote https://www.npmjs.com/package/webpack-plugin-hash-output to precisely fix this problem. It guarantees that you'll get a [chunkhash] that is always a hash of the content. It solves the two errors in this thread (as summarized by @jhnns).

It even has the option to fail the compilation if, for whatever reason, that postcondition is not met (see option validateOutput).

@webpack-bot
Copy link
Contributor

This issue had no activity for at least half a year.

It's subject to automatic issue closing if there is no activity in the next 15 days.

@webpack-bot
Copy link
Contributor

Issue was closed because of inactivity.

If you think this is still a valid issue, please file a new issue with additional information.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests