@caisq Reworking this PR with changes discussed in #108

@caisq This can be reviewed now

@manrajgrover Thanks for this PR! One high-level thought I have is: since this example shares a lot of modules and code in common with the multivariate-linear-regerssion model, we may want to merge this example with that one. We can have something like two buttons in the main HTML page, one called "train linear regressor" and another called "train NN regressor". After one of the button is clicked, the corresponding model will be trained and the loss curves plotted. That way, we not only get rid of some duplicate code, but also makes it easier for the user to compare and contrast the accuracy of the two models.

What do you think? If you agree, I can rename the multivariate-linear-regression folder to boston-housing for generality in #117

@caisq Agreed. It makes sense to combine the codebase. I can probably make the changes here itself once the other PR merges if it's okay

@manrajgrover OK. I will get the comments addressed for that PR and merge it soon.

Have you done any hyperparameter tuning? Is two hidden layers of size 10 kind of an optimum? Is it possible to increase the validation accuracy by increasing the size of the neural network to a reasonable degree?

@caisq No, I haven't. Not sure if this is correct but it is tough to tune params using validation split after dataset shuffling since validation dataset keeps changing. I was thinking of setting aside validation data instead of using splitting after shuffle to make things easy. (I do agree tuning parameters on different splits will help prevent overfitting and generalize better but it is making tuning params tough)

Please correct me if I'm wrong

Actually, if you use validationSplit while doing Model.fit(), the validation will always be the subset at the end of the xs data and ys labels. For example, if validationSplit is 0.15, Model.fit() will always use the last 15% as the validation set. The shuffling is done after the validation split.

Yes, I had read the code to understand it better. We do have shuffling happening even before validation split in data.js which I was refering to.

I suggest you do some informal hyperparameter tuning, to get a best validation error as low as reasonable. I think the current layer sizes (10) is not quite there yet. In my casual testing, I find increasing the layer sizes to about 30 - 40 helps. The reason why I'd like to see some hyperparameter tuning is so that it'll show a greater comparative advantage relative to linear regression, which we will use to justify the more complex network topology and higher computation cost in the teaching material.

@caisq Agreed.

@caisq I've tuned the params. We can also change the optimizer and activation function but for now, I've kept it as sgd and sigmoid to keep things simple and training smooth. Here is a screenshot of the output.

Just curious: does sigmoid in the hidden layer give better results than relu?

@caisq Actually, training using relu with sgd was not smooth hence I changed it to sigmoid. Using adam optimizer fixed it and gave better result. I thought of keeping it simple and went with basic.