fix(sanitizePath): ensure unicode chars are encoded as expected #70

luqven · 2021-04-01T15:10:26Z

Description

This PR adds test and updates the sanitizePath method to ensure Unicode characters get encoded when needed and as expected. It builds off of the imgix-go implementation for encoded URLs and creates a new isASCIIEncoded method that validates ASCII encoding and a few more custom imgix encoding rules.

commit-lint · 2021-04-01T15:10:30Z

Tests

buildURL: ensure unicode charas get encoded (a94f952)
BuildUrl: add more unicdoe tests (2901112)
buildUrl: disable encoded unicode test for now (e17c687)
unicode: reword todo comment (39758e4)

Bug Fixes

sanatizePath: ensure unicode chars encoded (36da74c)

Chore

checkProxy: use hashmap for jdk 8 compat (df9d46b)
sanitizePath: fix method name spelling (e57dfc0)
decode: remove unused fn again (dc0cebb)
checkProxyStatus: use bool instead of str vars (bb16c43)
checkProxyStatus: remove encodedHTTP support for now (6a60952)
santiizePath: accept encoded proxy paths (66822f0)
sanitizePath: disable ascii encoded path (ab1c87b)
URLHelper: remove unused isAscii method (14cf5de)

Contributors

luqven

Commit-Lint commands

You can trigger Commit-Lint actions by commenting on this PR:

@Commit-Lint merge patch will merge dependabot PR on "patch" versions (X.X.Y - Y change)
@Commit-Lint merge minor will merge dependabot PR on "minor" versions (X.Y.Y - Y change)
@Commit-Lint merge major will merge dependabot PR on "major" versions (Y.Y.Y - Y change)
@Commit-Lint merge disable will desactivate merge dependabot PR
@Commit-Lint review will approve dependabot PR
@Commit-Lint stop review will stop approve dependabot PR

luqven · 2021-04-01T15:33:48Z

src/main/java/com/imgix/URLHelper.java

+  public static Map<String, Boolean> checkProxyStatus(String p) {
+    Map<String, Boolean> status = new HashMap<String, Boolean>();
+    String path = p;
+    path.replaceAll("^/", "");


Had to change implementation to HashMap since JDK 8 was not very happy with Map. Makes syntax read a little differently here than in does in go.

ericdeansanchez · 2021-04-01T22:35:12Z

src/main/java/com/imgix/URLHelper.java

+  public static Map<String, Boolean> checkProxyStatus(String p) {
+    Map<String, Boolean> status = new HashMap<String, Boolean>();
+    String path = p;
+    path.replaceAll("^/", "");


I'm assuming this replaceAll takes a regex, is that correct?

ericdeansanchez · 2021-04-01T22:39:31Z

src/main/java/com/imgix/URLHelper.java


 public class URLHelper {

+  private static final String IS_ENCODED = "isEncoded";


Do these values need to be String? The string value "isEncoded" is the same as its identifier IS_ENCODED, what if ... Boolean IS_ENCODED = false?

Linter was yelling at me for not storing these as a constant 🤷🏼‍♂️ , but I can change this around if you like.

My 2¢ here, I agree with @ericdeansanchez

sherwinski · 2021-04-02T19:45:56Z

src/main/java/com/imgix/URLHelper.java

+                .replaceAll("\\%21", "!")
+                .replaceAll("\\%27", "'")
+                .replaceAll("\\%28", "(")
+                .replaceAll("\\%29", ")")
+                .replaceAll("\\%7E", "~");


What's the rationale for decoding these? I imagine it's to ensure consistency but I just wanted to be sure.

sherwinski · 2021-04-02T19:46:49Z

src/main/java/com/imgix/URLHelper.java


 public class URLHelper {

+  private static final String IS_ENCODED = "isEncoded";


My 2¢ here, I agree with @ericdeansanchez

src/main/java/com/imgix/URLHelper.java

luqven · 2021-04-05T13:56:00Z

Latest build fails because of test testHelperBuildSignedURLWithWebProxyWithEncoding()

imgix-java/src/test/java/com/imgix/test/TestAll.java

Lines 173 to 183 in c8e9655

    
           public void testHelperBuildSignedURLWithWebProxyWithEncoding() { 
        
             URLHelper uh = 
        
                 new URLHelper( 
        
                     "jackttl2.imgix.net", 
        
                     "http%3A%2F%2Fa.abcnews.com%2Fassets%2Fimages%2Fnavigation%2Fabc-logo.png%3Fr%3D20", 
        
                     "http", 
        
                     "JHrM2ezd"); 
        
             assertEquals( 
        
                 "http://jackttl2.imgix.net/http%3A%2F%2Fa.abcnews.com%2Fassets%2Fimages%2Fnavigation%2Fabc-logo.png%3Fr%3D20?s=cf82defe3436a957262d0e64c21e72f9", 
        
                 uh.getURL()); 
        
           }

Maybe we need to allow encoded proxy paths through given this was established behavior? Might need to undo 0c4f0d0

sherwinski · 2021-04-06T16:19:24Z

Latest build fails because of test testHelperBuildSignedURLWithWebProxyWithEncoding()

imgix-java/src/test/java/com/imgix/test/TestAll.java

Lines 173 to 183 in c8e9655

public void testHelperBuildSignedURLWithWebProxyWithEncoding() {

URLHelper uh =

new URLHelper(

"jackttl2.imgix.net",

"http%3A%2F%2Fa.abcnews.com%2Fassets%2Fimages%2Fnavigation%2Fabc-logo.png%3Fr%3D20",

"http",

"JHrM2ezd");

assertEquals(

"http://jackttl2.imgix.net/http%3A%2F%2Fa.abcnews.com%2Fassets%2Fimages%2Fnavigation%2Fabc-logo.png%3Fr%3D20?s=cf82defe3436a957262d0e64c21e72f9",

uh.getURL());

}

Maybe we need to allow encoded proxy paths through given this was established behavior? Might need to undo 0c4f0d0

Yeah I think you're right, we should keep this in order to prevent a breaking change.

src/main/java/com/imgix/URLHelper.java

luqven · 2021-04-06T18:20:17Z

src/main/java/com/imgix/URLHelper.java

+    } else if (pathIsProxy.equals(true) && proxyIsEncoded.equals(true)) {
+      return "/" + path;


This conditional is needed to avoid encoded path support breaking change.

luqven · 2021-04-06T18:21:11Z

src/test/java/com/imgix/test/TestAll.java

+            // TODO(luis): re-enable encoded path suppport at a later date
+            // {
+            //   "Absolute Path With Encoded Unicode Characters",
+            //
+            // "/example/I%20cann%C3%B8t%20bel%C3%AE%C3%A9v%E2%88%91%20it%20wor%EF%A3%BFs!%20%F0%9F%98%B1",
+            //
+            // "http://securejackangers.imgix.net/example/I%20cann%C3%B8t%20bel%C3%AE%C3%A9v%E2%88%91%20it%20wor%EF%A3%BFs!%20%F0%9F%98%B1?w=500"
+            // },


Leaving this test commented for clarity's sake. Want to be explicit that encoded unicode chars are not supported in paths unless they are proxy paths

I'm a bit confused on this part, was this already supported behavior?

Sorry my comment isn't clear. Should say enable not re-enable. Only wanted to let next contributor know explicitly encoded UNICODE paths are not supported atm.

luqven · 2021-04-09T02:59:43Z

@sherwinski @ericdeansanchez double checking we're good to merge this one in

sherwinski · 2021-04-09T15:24:13Z

@sherwinski @ericdeansanchez double checking we're good to merge this one in

All good from my end 👍

luqven · 2021-04-12T15:31:52Z

Had to force push since I forgot to rebase on luis/pathEncodingFix before it got merged in to main. Apols.

luqven self-assigned this Apr 1, 2021

luqven changed the base branch from main to luis/pathEncodingFix April 1, 2021 15:10

luqven commented Apr 1, 2021

View reviewed changes

ericdeansanchez reviewed Apr 1, 2021

View reviewed changes

luqven force-pushed the luis/utf8test branch from 0d8450c to 29cb9f1 Compare April 2, 2021 19:04

luqven marked this pull request as ready for review April 2, 2021 19:05

luqven requested a review from a team as a code owner April 2, 2021 19:05

sherwinski suggested changes Apr 2, 2021

View reviewed changes

luqven force-pushed the luis/utf8test branch from 12b5efd to 0c4f0d0 Compare April 5, 2021 13:29

luqven requested a review from sherwinski April 5, 2021 13:31

luqven commented Apr 6, 2021

View reviewed changes

src/main/java/com/imgix/URLHelper.java Outdated Show resolved Hide resolved

luqven commented Apr 6, 2021

View reviewed changes

luqven requested a review from ericdeansanchez April 9, 2021 02:50

luqven mentioned this pull request Apr 12, 2021

fix(URLHelper): correctly encode more special chars #68

Merged

Base automatically changed from luis/pathEncodingFix to main April 12, 2021 15:20

luqven added 8 commits April 12, 2021 11:26

test(buildURL): ensure unicode charas get encoded

a94f952

fix(sanatizePath): ensure unicode chars encoded

36da74c

chore(checkProxy): use hashmap for jdk 8 compat

df9d46b

chore(sanitizePath): fix method name spelling

e57dfc0

chore(decode): remove unused fn again

dc0cebb

test(BuildUrl): add more unicdoe tests

2901112

chore(checkProxyStatus): use bool instead of str vars

bb16c43

chore(checkProxyStatus): remove encodedHTTP support for now

6a60952

luqven added 5 commits April 12, 2021 11:26

chore(santiizePath): accept encoded proxy paths

66822f0

chore(sanitizePath): disable ascii encoded path

ab1c87b

test(buildUrl): disable encoded unicode test for now

e17c687

chore(URLHelper): remove unused isAscii method

14cf5de

test(unicode): reword todo comment

39758e4

luqven force-pushed the luis/utf8test branch from 463b62c to 39758e4 Compare April 12, 2021 15:30

luqven merged commit fc5bf8f into main Apr 12, 2021

luqven deleted the luis/utf8test branch April 12, 2021 15:33

This was referenced Apr 12, 2021

Undocumented behavior: path provided to createURL must be doubly url-encoded #14

Closed

Fix for issue #14 #23

Closed

chore(release): 2.3.2 #71

Merged


		public class URLHelper {

		private static final String IS_ENCODED = "isEncoded";

		} else if (pathIsProxy.equals(true) && proxyIsEncoded.equals(true)) {
		return "/" + path;

fix(sanitizePath): ensure unicode chars are encoded as expected #70

fix(sanitizePath): ensure unicode chars are encoded as expected #70

Uh oh!

Conversation

luqven commented Apr 1, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Uh oh!

commit-lint bot commented Apr 1, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Tests

Bug Fixes

Chore

Contributors

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

luqven commented Apr 5, 2021

Uh oh!

sherwinski commented Apr 6, 2021

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

luqven commented Apr 9, 2021

Uh oh!

sherwinski commented Apr 9, 2021

Uh oh!

luqven commented Apr 12, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

luqven commented Apr 1, 2021 •

edited

Loading

commit-lint bot commented Apr 1, 2021 •

edited

Loading