Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@Zarquan
Copy link
Collaborator

@Zarquan Zarquan commented Apr 24, 2020

Notes and playbooks to deploy Hadoop and Spark using Ansible.
Addresses issues #43, #44 and #45.

Copy link
Collaborator

@stvoutsin stvoutsin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very useful examples. It might be good to have a final set of how-to's, perhaps in a separate directory to notes, or a wiki page.

| Maximum limit of memory to allocate to each container request at the Resource Manager.
<property>
<name>yarn.scheduler.maximum-allocation-mb</name>
<value>15000</value>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps a future issue, to make the max/min allocation values configurable

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went with the defaults to start with (most of these settings are commented out in the initial deploy).
We need to have tests in place to demonstrate why we need to change these values, and be able to re-run the tests to check what happens if we change the settings.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added new issues to document the settings we need for Hadoop(#95) and Spark(#96).

@stvoutsin stvoutsin merged commit 25d46eb into wfau:master Apr 28, 2020
@Zarquan Zarquan deleted the 20200121-zrq-ansible branch May 28, 2020 11:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants