📬 Feature: post processors #253

abovedave · 2017-10-06T16:13:10Z

Removes the previously undocumented page.settings.beautify option & closes #154 (related to #162).

Adds the ability to manipulate the raw output of the template engine, before the page is cached and served. Similar to the Wordpress feature, it is useful for linting (thinking RSS feeds in particular), minifying and formatting. They are loaded in order of definition, global first, and the output is passed from one to the next so you can chain functions.

Example usage: minifying HTML.

config.json

"globalPostProcessors": [
    "minify-html"
]

The default location in paths.processors is workspace/processors

`/workspace/processors/minify-html.js` (using html-minifier)

const minify = require('html-minifier').minify

module.exports = (data, output) => {
  return minify(output, {
  	collapseWhitespace: true,
  	minifyCSS: true,
  	minifyJS: true,
  	removeRedundantAttributes: true,
  	useShortDoctype: true,
  	removeAttributeQuotes: true
  })
}

Note:

the page json data object is also passed
in addition it can be defined or disabled at page level in the page.settings

Add a processor, in addition to the global

"settings": {
  "postProcessors": ["remove-swear-words"]
}

Disable for given page

"settings": {
  "postProcessors": false
}

Upgrading from `page.settings.beautify`

Install package

npm install js-beautify --save

New file `workspace/processors/beautify-html.js`

const beautifyHtml = require('js-beautify').html

module.exports = (data, output) => {
  return beautifyHtml(output)
}

Old `page.json`

{
  "page": {
    "name": "Sitemap page",
    "description": "Sitemap",
    "language": "en"
  },
  "settings": {
    "cache": true,
    "beautify": true
  },
  ...
}

New `page.json`

{
  "page": {
    "name": "Sitemap page",
    "description": "Sitemap",
    "language": "en"
  },
  "settings": {
    "cache": true,
    "postProcessors": [
      "beautify-html"
     ]
  },
  ...
}

jimlambie

❤️

jimlambie · 2017-10-09T08:11:33Z

This is a breaking change because it removes beautifying from core?

abovedave · 2017-10-09T08:14:17Z

Yes exactly, that setting will no longer have any effect...not sure how to handle it since it was undocumented anyway? You could only turn it on page-by-page in the settings block.

AFAIK we're not using it in any client work either.

jimlambie · 2017-10-09T08:16:47Z

Perhaps we should make some enquiries then - if it's not used we can simply make it a new feature and release as a minor version

abovedave · 2017-10-09T08:33:26Z

Added upgrade notes from settings.beautify

eduardoboucas

This is super cool! Just added some minor comments.

eduardoboucas · 2017-10-09T08:35:59Z

dadi/lib/view/index.js

-      .then(output => {
-        let err = null
+  // Send the templated page
+  let templateData = Object.assign(this.data, {


Could this be a const?

Also, to avoid mutating this.data, you could do Object.assign({}, this.data, (...)).

eduardoboucas · 2017-10-09T08:36:38Z

dadi/lib/view/index.js

+
+      // Post-process the output
+      if (config.get('globalPostProcessors') || this.page.postProcessors) {
+        let postProcessors = config


Could this be a const?

eduardoboucas · 2017-10-09T08:43:10Z

dadi/lib/view/index.js

+      keepWhitespace: this.page.keepWhitespace
+    })
+    .then(output => {
+      let err = null


This variable doesn't seem to ever be changed. You're returning an error on L60 but it comes from L59. Could we remove this initialisation and just do return done(null, output) on L65?

abovedave · 2017-10-09T09:53:15Z

Great suggestions, thanks @eduardoboucas!

jimlambie · 2017-10-10T12:03:47Z

This still has to be a major release, right? If we're not bundling in a beautify processor and honouring the existing page setting for beautify, then it's a breaking change. This would make it Web 5.0.

abovedave · 2017-10-10T12:51:45Z

Yep makes sense 👍

eduardoboucas · 2017-10-10T12:52:47Z

Agree 💯 %.

jimlambie · 2017-10-10T12:54:19Z

I'm a bit wary of another major release so soon after 4.0, is all. Feels like we're following the FB Messenger app approach - they're up to version 133 or something. Mind you, their release notes are awful. "Performance enhancements".

eduardoboucas · 2017-10-10T12:59:56Z

That's understandable. If this is not an urgent requirement, perhaps we can hit pause and bundle it with other breaking changes for the next major release? ⏸

abovedave · 2017-10-10T13:17:15Z

Could we prepare a 5.0 branch perhaps? Just conscience that this branch will get stale and drift from head.

eduardoboucas · 2017-10-10T14:34:39Z

Good idea. We could start a release/5.0 branch – I think this is the convention we agreed on, right @jimlambie?

David Longworth added 5 commits October 6, 2017 16:21

feat: post-process output

dfd2798

test: postProcessors

dd9b729

feat: pass the page data to the postProcessors

a3c56af

fix: test settings param

aa98521

test: add missing processor

ed9eebe

abovedave added type: breaking type: enhancement labels Oct 6, 2017

abovedave changed the title ~~Feature: post processors~~ 📬 Feature: post processors Oct 6, 2017

abovedave requested review from eduardoboucas and jimlambie October 6, 2017 16:22

jimlambie approved these changes Oct 9, 2017

View reviewed changes

eduardoboucas reviewed Oct 9, 2017

View reviewed changes

refactor: EB suggestions

1fba1d2

abovedave changed the base branch from develop to release/5.0 October 12, 2017 12:42

abovedave changed the base branch from release/5.0 to develop October 13, 2017 09:02

abovedave merged commit 60e7a6a into develop Oct 13, 2017

abovedave deleted the feature/postProcessors branch December 6, 2017 15:29

abovedave mentioned this pull request Apr 12, 2018

postProcessors docs #380

Closed

📬 Feature: post processors #253

📬 Feature: post processors #253

Uh oh!

Conversation

abovedave commented Oct 6, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

config.json

/workspace/processors/minify-html.js (using html-minifier)

Add a processor, in addition to the global

Disable for given page

Upgrading from page.settings.beautify

Install package

New file workspace/processors/beautify-html.js

Old page.json

New page.json

Uh oh!

jimlambie left a comment

Choose a reason for hiding this comment

Uh oh!

jimlambie commented Oct 9, 2017

Uh oh!

abovedave commented Oct 9, 2017

Uh oh!

jimlambie commented Oct 9, 2017

Uh oh!

abovedave commented Oct 9, 2017

Uh oh!

eduardoboucas left a comment

Choose a reason for hiding this comment

Uh oh!

eduardoboucas Oct 9, 2017

Choose a reason for hiding this comment

Uh oh!

eduardoboucas Oct 9, 2017

Choose a reason for hiding this comment

Uh oh!

eduardoboucas Oct 9, 2017

Choose a reason for hiding this comment

Uh oh!

eduardoboucas Oct 9, 2017

Choose a reason for hiding this comment

Uh oh!

abovedave commented Oct 9, 2017

Uh oh!

jimlambie commented Oct 10, 2017

Uh oh!

abovedave commented Oct 10, 2017

Uh oh!

eduardoboucas commented Oct 10, 2017

Uh oh!

jimlambie commented Oct 10, 2017

Uh oh!

eduardoboucas commented Oct 10, 2017

Uh oh!

abovedave commented Oct 10, 2017

Uh oh!

eduardoboucas commented Oct 10, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

abovedave commented Oct 6, 2017 •

edited

Loading

`/workspace/processors/minify-html.js` (using html-minifier)

Upgrading from `page.settings.beautify`

New file `workspace/processors/beautify-html.js`

Old `page.json`

New `page.json`