Skip to content

improve mangle #2948

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 23, 2018
Merged

improve mangle #2948

merged 1 commit into from
Feb 23, 2018

Conversation

alexlamsl
Copy link
Collaborator

node test/benchmark.js

master #2948
https://code.jquery.com/jquery-3.2.1.js
- parse: 0.234s
- rename: 0.203s
- compress: 1.531s
- scope: 0.078s
- mangle: 0.157s
- properties: 0.000s
- output: 0.078s
- total: 2.281s

Original: 268039 bytes
Uglified: 86047 bytes
GZipped:  30058 bytes
SHA1 sum: e3759e2b87ceb0391b0e50c8c3421a1c9ac5abab

https://cdnjs.cloudflare.com/ajax/libs/angular.js/1.6.4/angular.js
- parse: 0.484s
- rename: 0.312s
- compress: 2.563s
- scope: 0.141s
- mangle: 0.296s
- properties: 0.000s
- output: 0.157s
- total: 3.953s

Original: 1249863 bytes
Uglified: 173596 bytes
GZipped:  59986 bytes
SHA1 sum: b68526f9c8d8dfa56ada736f7f65afbab31c0b71

https://cdnjs.cloudflare.com/ajax/libs/mathjs/3.9.0/math.js
- parse: 0.906s
- rename: 0.609s
- compress: 5.594s
- scope: 0.172s
- mangle: 0.328s
- properties: 0.000s
- output: 0.234s
- total: 7.843s

Original: 1590107 bytes
Uglified: 466497 bytes
GZipped:  118809 bytes
SHA1 sum: 9b97bace1201003ddbe9d83f4891c5ba39f425fc

https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.js
- parse: 0.109s
- rename: 0.047s
- compress: 0.531s
- scope: 0.031s
- mangle: 0.063s
- properties: 0.000s
- output: 0.047s
- total: 0.828s

Original: 69707 bytes
Uglified: 36833 bytes
GZipped:  9579 bytes
SHA1 sum: df38148a95914060e20f4495d5767e06ba59ef3a

https://unpkg.com/[email protected]/dist/react.js
- parse: 0.531s
- rename: 0.250s
- compress: 2.437s
- scope: 0.125s
- mangle: 0.266s
- properties: 0.000s
- output: 0.109s
- total: 3.718s

Original: 701412 bytes
Uglified: 205113 bytes
GZipped:  62268 bytes
SHA1 sum: e49d727b1f64ba3ceb32135f1708a66718f117cf

http://builds.emberjs.com/tags/v2.11.0/ember.prod.js
- parse: 0.828s
- rename: 0.609s
- compress: 4.000s
- scope: 0.172s
- mangle: 0.406s
- properties: 0.000s
- output: 0.266s
- total: 6.281s

Original: 1852178 bytes
Uglified: 525929 bytes
GZipped:  129044 bytes
SHA1 sum: 6258c8317f1b121e6f1d08d8ed2e7d391a06735f

https://cdn.jsdelivr.net/lodash/4.17.4/lodash.js
- parse: 0.265s
- rename: 0.203s
- compress: 3.110s
- scope: 0.062s
- mangle: 0.172s
- properties: 0.000s
- output: 0.047s
- total: 3.859s

Original: 539590 bytes
Uglified: 69574 bytes
GZipped:  24884 bytes
SHA1 sum: 0f828505bbb3a1b86b2207f35aa092b951b37ec2

https://cdnjs.cloudflare.com/ajax/libs/d3/4.5.0/d3.js
- parse: 0.578s
- rename: 0.359s
- compress: 3.516s
- scope: 0.125s
- mangle: 0.234s
- properties: 0.000s
- output: 0.125s
- total: 4.937s

Original: 451131 bytes
Uglified: 211566 bytes
GZipped:  71027 bytes
SHA1 sum: 8d5326b2769c2d66f6b47439d482b09f74d35a48

https://raw.githubusercontent.com/kangax/html-minifier/v3.5.7/dist/htmlminifier.js
- parse: 0.812s
- rename: 0.547s
- compress: 4.703s
- scope: 0.156s
- mangle: 0.422s
- properties: 0.000s
- output: 0.266s
- total: 6.906s

Original: 1063075 bytes
Uglified: 507989 bytes
GZipped:  157987 bytes
SHA1 sum: f86ae107da3ac2453d5f87c7fad4d84c580e4071
https://code.jquery.com/jquery-3.2.1.js
- parse: 0.265s
- rename: 0.219s
- compress: 1.484s
- scope: 0.094s
- mangle: 0.141s
- properties: 0.000s
- output: 0.093s
- total: 2.296s

Original: 268039 bytes
Uglified: 86047 bytes
GZipped:  29988 bytes
SHA1 sum: d6594a573507818c0125f96282c7e90a6bd40f70

https://cdnjs.cloudflare.com/ajax/libs/angular.js/1.6.4/angular.js
- parse: 0.484s
- rename: 0.328s
- compress: 2.656s
- scope: 0.125s
- mangle: 0.297s
- properties: 0.000s
- output: 0.110s
- total: 4.000s

Original: 1249863 bytes
Uglified: 174128 bytes
GZipped:  59723 bytes
SHA1 sum: 3b875a8ffe96bd18d6a054c6a6664a0a894ac215

https://cdnjs.cloudflare.com/ajax/libs/mathjs/3.9.0/math.js
- parse: 0.875s
- rename: 0.468s
- compress: 5.516s
- scope: 0.250s
- mangle: 0.344s
- properties: 0.000s
- output: 0.234s
- total: 7.687s

Original: 1590107 bytes
Uglified: 466501 bytes
GZipped:  119347 bytes
SHA1 sum: 408dbba22f4667057fd678deb7929d085ae75ca1

https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.js
- parse: 0.093s
- rename: 0.063s
- compress: 0.547s
- scope: 0.031s
- mangle: 0.062s
- properties: 0.000s
- output: 0.032s
- total: 0.828s

Original: 69707 bytes
Uglified: 36833 bytes
GZipped:  9774 bytes
SHA1 sum: 86fbd01faad9c9a6525309c9ee6a3de3fe8dcd9e

https://unpkg.com/[email protected]/dist/react.js
- parse: 0.531s
- rename: 0.281s
- compress: 2.469s
- scope: 0.140s
- mangle: 0.266s
- properties: 0.000s
- output: 0.125s
- total: 3.812s

Original: 701412 bytes
Uglified: 205113 bytes
GZipped:  62174 bytes
SHA1 sum: d9c4204e88b2ac61fb5b666bc161ed32f704abee

http://builds.emberjs.com/tags/v2.11.0/ember.prod.js
- parse: 0.828s
- rename: 0.640s
- compress: 4.032s
- scope: 0.172s
- mangle: 0.437s
- properties: 0.000s
- output: 0.250s
- total: 6.359s

Original: 1852178 bytes
Uglified: 525929 bytes
GZipped:  128721 bytes
SHA1 sum: bf12371338e72eea9d0c775402eaafeb19dc6fef

https://cdn.jsdelivr.net/lodash/4.17.4/lodash.js
- parse: 0.250s
- rename: 0.187s
- compress: 3.047s
- scope: 0.078s
- mangle: 0.203s
- properties: 0.000s
- output: 0.094s
- total: 3.859s

Original: 539590 bytes
Uglified: 69643 bytes
GZipped:  24482 bytes
SHA1 sum: 1f8958f48aee217e21304d54417f7af9c6a77d53

https://cdnjs.cloudflare.com/ajax/libs/d3/4.5.0/d3.js
- parse: 0.593s
- rename: 0.344s
- compress: 3.422s
- scope: 0.109s
- mangle: 0.219s
- properties: 0.000s
- output: 0.188s
- total: 4.875s

Original: 451131 bytes
Uglified: 211590 bytes
GZipped:  70536 bytes
SHA1 sum: 8456cd08832b7f5800155d57f030875b7688dfce

https://raw.githubusercontent.com/kangax/html-minifier/v3.5.7/dist/htmlminifier.js
- parse: 0.875s
- rename: 0.562s
- compress: 4.938s
- scope: 0.172s
- mangle: 0.437s
- properties: 0.000s
- output: 0.266s
- total: 7.250s

Original: 1063075 bytes
Uglified: 508078 bytes
GZipped:  157609 bytes
SHA1 sum: df5be75226b15c7453cb58e7c66af6751f6db4d4

@alexlamsl alexlamsl changed the title improve mangle [WIP] improve mangle Feb 23, 2018
@alexlamsl
Copy link
Collaborator Author

@kzc I think I'll do 1MFuzz on this locally, and if everything's green this is good to go.

@alexlamsl alexlamsl changed the title [WIP] improve mangle improve mangle Feb 23, 2018
@alexlamsl alexlamsl merged commit f40f5eb into mishoo:master Feb 23, 2018
@alexlamsl alexlamsl deleted the mangle branch February 23, 2018 15:51
@kzc
Copy link
Contributor

kzc commented Feb 23, 2018

Seems to provide some general mangle improvement.

sum of gzip sizes before PR: 663642
sum of gzip sizes after PR:  662354

Edit: gzip improvement: 1288, or 0.194%

If you don't mind, could you briefly describe how the algorithm was changed?

@alexlamsl
Copy link
Collaborator Author

could you briefly describe how the algorithm was changed?

Take the following input:

var a;
function f(b) {
   var c;
}

Before this PR, mangle assigns symbol names in a breadth-first manner:

var n1;
function n2(n3) {
    var n4;
}

This PR assigns them from the innermost AST_Scope upwards:

var n3;
function n4(n1) {
    var n2;
}

The previous approach is easier to implement efficiently, especially when it comes to namespace utilisation, i.e. ensuring no "holes" between shortest and longest base54() name assignment.

However, this would suffer from avalanche effect, where addition of a variable would often cause many unrelated scopes to lose contiguous name assignments, which in turn harms gzip compression ratio.

This is basically another attempt at #2219, with all the bells and whistles untangled and (seemingly) functionally correct this time round.

@kzc
Copy link
Contributor

kzc commented Feb 23, 2018

This PR assigns them from the innermost AST_Scope upwards

That makes sense. Thanks for the explanation.

By the way, I fixed a typo in my post above. This PR provides close to a 0.2% improvement.

@alexlamsl
Copy link
Collaborator Author

This PR provides close to a 0.2% improvement.

Good to know 👍

FWIW, I tried to improve bootstrap.js and math.js with localised .compute_char_frequency() in the hopes of reducing local entropy, which backfired badly.

@alexlamsl
Copy link
Collaborator Author

Taking care of all those weird corner cases, most of them involving AST_SymbolCatch, would have been impossible without the current suite of tests and test/ufuzz.js

@kzc
Copy link
Contributor

kzc commented Feb 23, 2018

I noticed a 0.45% regression with this PR in math.js gzip size. I guess you can't win 'em all.

@alexlamsl
Copy link
Collaborator Author

Yeah - I compared the output size-by-size and all I can say is there is room for improvement for our current scheme of compute_char_frequency(), perhaps with digram frequency.

@Skalman
Copy link
Contributor

Skalman commented Feb 25, 2018

This is an impressive overall Gzip win!
FWIW, according to the numbers posted above, the uglified numbers are worsened by 0.03%.

cyjake added a commit to cyjake/UglifyJS2 that referenced this pull request Aug 7, 2018
Normally any collision on these names won't be an issue but the Safari on iOS 7/8/9 will yield with a SyntaxError. The problem was discuessed and (partially) fixed in mishoo#179 but function definition mangling still breaks:

```js
$ echo 'function aaa(){function foo(bar){};foo.name=2;return foo}' | ./node_modules/.bin/uglifyjs --compress --mangle
function aaa(){function n(n){}return n.name=2,n}
```

Since symbol names are now [mangled from within](mishoo#2948 (comment)), the mangled argument names can now be appended to `names` cache to make sure the later mangled function name won't collide with them.
cyjake added a commit to cyjake/UglifyJS2 that referenced this pull request Aug 8, 2018
Normally any collision on these names won't be an issue but the Safari on iOS 7/8/9 will yield with a SyntaxError. The problem was discuessed and (partially) fixed in mishoo#179 but function definition mangling still breaks:

```js
$ echo 'function aaa(){function foo(bar){};foo.name=2;return foo}' | ./node_modules/.bin/uglifyjs --compress --mangle
function aaa(){function n(n){}return n.name=2,n}
```

Since symbol names are now [mangled from within](mishoo#2948 (comment)), the mangled argument names can now be appended to `names` cache to make sure the later mangled function name won't collide with them.
cyjake added a commit to cyjake/UglifyJS2 that referenced this pull request Aug 8, 2018
Usually the function name is dropped if the function is a function expression. But when `ie8` option is enabled, the function name is kept and mangled which may break legacy Safari for said reason. Since symbol names are now [mangled from within](mishoo#2948 (comment)), this fix adds argument names to existing names when mangling the name of a function expression.
cyjake added a commit to cyjake/UglifyJS2 that referenced this pull request Aug 8, 2018
Usually the function name is dropped if the function is a function expression. But when `ie8` option is enabled, the function name is kept and mangled which may break legacy Safari for said reason. Since symbol names are now [mangled from within](mishoo#2948 (comment)), this fix adds argument names to existing names when mangling the name of a function expression.
cyjake added a commit to cyjake/UglifyJS2 that referenced this pull request Aug 8, 2018
Usually the function name is dropped if the function is a function expression. But when `ie8` option is enabled, the function name is kept and mangled which may break legacy Safari for said reason. Since symbol names are now [mangled from within](mishoo#2948 (comment)), this fix adds argument names to existing names when mangling the name of a function expression.
cyjake added a commit to cyjake/UglifyJS2 that referenced this pull request Aug 8, 2018
Usually the function name is dropped if the function is a function expression. But when `ie8` option is enabled, the function name is kept and mangled which may break legacy Safari for said reason. Since symbol names are now [mangled from within](mishoo#2948 (comment)), this fix adds argument names to existing names when mangling the name of a function expression.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants