b. Configure ParallelCluster

Typically, to configure AWS ParallelCluster, you use the command pcluster configure and then provide the requested information, such as the AWS Region, Scheduler, and EC2 Instance Type. However, for this workshop you can take a shortcut by creating a basic configuration file, then customizing this file to include HPC specific options.

The following commands generate a new keypair, query the EC2 metadata to get the Subnet ID, VPC ID, and finally write a config to ~/.parallelcluster/config. You can always edit this config file to add and change configuration options.

Don’t skip this step, creating key-pair step is very important for the later steps, please follow instruction bellow.

Generate a new key-pair

aws ec2 create-key-pair --key-name lab-3-your-key --query KeyMaterial --output text > ~/.ssh/lab-3-key
chmod 600 ~/.ssh/lab-3-key

Getting your AWS networking information

IFACE=$(curl --silent
SUBNET_ID=$(curl --silent${IFACE}/subnet-id)
VPC_ID=$(curl --silent${IFACE}/vpc-id)
REGION=$(curl --silent | sed 's/[a-z]$//')
mkdir -p ~/.parallelcluster

Creating an initial AWS ParallelCluster config file

cat > ~/.parallelcluster/config << EOF
aws_region_name = ${REGION}

[cluster default]
key_name = lab-3-your-key
vpc_settings = public
base_os = alinux2
scheduler = slurm

[vpc public]
vpc_id = ${VPC_ID}
master_subnet_id = ${SUBNET_ID}

cluster_template = default
update_check = false
sanity_check = true

ssh = ssh {CFN_USER}@{MASTER_IP} {ARGS}

Now, check the content of this file using the following command:

cat ~/.parallelcluster/config

You now have a configuration file that allows you to create a simple cluster with the minimum required information. A default configuration file is good to have for testing purposes.

Next, you build a configuration to generate an optimized cluster to run typical “tightly coupled” HPC applications.