Skip to content

img0

Node Addition

Warning

Do not start this lab until the upgrade has fully completed from the previous lab.

Growing the Cluster

Info

Expanding the cluster requires preparation tasks be completed on the new node(s) prior to adding. Within the lab environment, we have completed the following tasks for you:

  1. Configuration of network interfaces for the node
  2. Set the timezone for the node
  3. Install and configure HyperStore prerequisites on the node
  4. Set up data disks on the node with ext4 file systems
  5. Mount all available data disks on the node

Instructions

  1. Click Cluster
  2. Click Data Centers
  3. Click the + next to the green hexagons

Info

Adding a node - CMC We will be adding node 5 to the cluster. For the following steps, refer to Sudent Assignments to obtain the hostname and IP address. The root password for node5 is the same as the hostname so studentXn5 where X is your student number

Instructions

  1. Enter Hostname
  2. Enter IP address
  3. Enter nodes root password
  4. Click Execute

Warning

Adding a node in a live environment.

The default token simulation may generate a warning suggesting slight imbalance after adding the new node. In the lab environment this can be safely ignored.
In most situations in a live cluster, you should liaise with your Cloudian SE so they can confirm the token sim and generate a more appropriate token file for your new node if required.

Note

CMC will issue a Precheck for adding a node where various ports and settings are checked. If you come across an Error at this stage, it is likely you have used the wrong hostname, IP, and or password for the node. We should also confirm the token simulation is correct and no other errors exist.

Instructions

  1. Once the pre-install check completes without any errors, click 'Install'

Instructions

  1. Select Cluster > Operation Status > View
  2. Monitor the add node process until completion (~20 minutes)
    Note that the progress stays at '1%' for some time
  3. Once the addNode process shows '100%', proceed to the next step

Rebalance Data

Instructions

  1. Select Cluster Tab
  2. Select Nodes Sub- Tab
  3. Select Advanced
  4. Select Maintenance
  5. Select rebalance
  6. Select Target Node
  7. Select cleanupfile so that when a replica or fragment is successfully copied to the new node, it is deleted from the older node on which it no longer belongs, this also cleans up the corresponding metadata in Cassandra
  8. Click Execute

Info

During in-progress rebalance operations the affected data remains readable by S3 client applications. Meanwhile the new nodes are immediately available to support writes of new data, even before any rebalancing occurs You can run rebalance on multiple new nodes concurrently. However, you should not run rebalance concurrently on a number of new nodes that exceeds one-third the number of your already existing nodes in that same data center

Warning

If an hsstool rebalance operation finishes with status FAILED do not run hsstool -h rebalance -resume. A FAILED operation status indicates that the operation errored out and exited prematurely. In this scenario, run hsstool -h rebalance -newrun. Do not use the -resume option.

Note

When you have rebalance running on a new node or multiple new nodes, you cannot add any additional nodes to the service region (using the CMC's Add Node feature) until rebalance has competed successfully on all of the current new nodes

Check the status of the new node indicated as a green hexagon

Adding Multiple Nodes

Info

Read the help files for the version of HyperStore you are using to ensure you are following best practices for adding multiple nodes. If in doubt, contact support or your Cloudian SE to further discuss