b. Configure ParallelCluster

Typically, to configure AWS ParallelCluster, you use the command pcluster configure and then provide the requested information, such as the AWS Region, Scheduler, and EC2 Instance Type. However, for this workshop you can take a shortcut by creating a basic configuration file, then customizing this file to include HPC specific options.

The following commands generate a new keypair, query the EC2 metadata to get the Subnet ID, VPC ID, and finally write a config to ~/.parallelcluster/config. You can always edit this config file to add and change configuration options.

# generate a new key-pair
aws ec2 create-key-pair --key-name lab-3-your-key --query KeyMaterial --output text > ~/.ssh/lab-3-key
chmod 600 ~/.ssh/lab-3-key

IFACE=$(curl --silent http://169.254.169.254/latest/meta-data/network/interfaces/macs/)
SUBNET_ID=$(curl --silent http://169.254.169.254/latest/meta-data/network/interfaces/macs/${IFACE}/subnet-id)
VPC_ID=$(curl --silent http://169.254.169.254/latest/meta-data/network/interfaces/macs/${IFACE}/vpc-id)
REGION=$(curl --silent http://169.254.169.254/latest/meta-data/placement/availability-zone | sed 's/[a-z]$//')
mkdir -p ~/.parallelcluster
cat > ~/.parallelcluster/config << EOF
[aws]
aws_region_name = ${REGION}

[cluster default]
key_name = lab-3-your-key
vpc_settings = public
base_os = alinux2
scheduler = slurm

[vpc public]
vpc_id = ${VPC_ID}
master_subnet_id = ${SUBNET_ID}

[global]
cluster_template = default
update_check = false
sanity_check = true

[aliases]
ssh = ssh {CFN_USER}@{MASTER_IP} {ARGS}
EOF

Now, check the content of this file using the following command:

cat ~/.parallelcluster/config

You now have a configuration file that allows you to create a simple cluster with the minimum required information. A default configuration file is good to have for testing purposes.

Next, you build a configuration to generate an optimized cluster to run typical “tightly coupled” HPC applications.