Now, learn what really happens in AWS when you submit a job and reveal a bit of the magic behind your automatic-scaling computational resources.
At the heart of AWS ParallelCluster exists an Auto Scaling Group. It is a logical group of instances that can scale up and scale down based on a series of criteria. In the case of AWS ParallelCluster, there are three processes controlling the scaling of the cluster. These processes:
For more details on these processes, see AWS ParallelCluster Processes.
To learn more about how this works, take a detailed look at Auto Scaling Groups:
Go back to your AWS Cloud9 environment and launch a 5 minutes sleep job with the following commands.
cat > sleep_script.sbatch << EOF #!/bin/bash #SBATCH --job-name=hello-world-job #SBATCH --ntasks=2 #SBATCH --output=%x_%j.out sleep 300 EOF sbatch sleep_script.sbatch
Go back to the EC2 Dashboard and Auto Scaling Groups.
If necessary, click the refresh button (circling arrows). You should see that an instance just appeared on the desired field instead of 0. It corresponds to the 2 physical cores or c4.xlarge equivalent that you just requested.
On the EC2 Dashboard, in the left pane, choose Instances. You should see your compute instances labeled as Compute.
Now you have a better understanding on how AWS ParallelCluster operates. For more information, see the Configuration section of the AWS ParallelCluster User Guide.