3,809 compute instances, each with 8 cores & 7GB of RAM. A total of 30,472 cores, 26.7TB of RAM and 2PB disk. Cool.
Amazon have recently added some useful features such as HTTPS health checking, choice of ciphers & protocols. Also you can now verify the authenticity of the EC2 server before sending the request…which is great!
If you’re bored of managing changes for route53 from the command line, or thinking of writing some software to help you manage route53, you might want to check this small Python app out. It’s working well for me!
Amazon ElastiCache is a web service that makes it easy to deploy, operate, and scale an in-memory cache in the cloud. Amazon ElastiCache is protocol-compliant with Memcached and offloads the management, monitoring, and operation of in-memory cache environments.
We are excited to announce the immediate availability of Cluster GPU Instances for Amazon EC2, a new instance type designed to deliver the power of GPU processing in the cloud. GPUs are increasingly being used to accelerate the performance of many general purpose computing problems. However, for many organizations, GPU processing has been out of reach due to the unique infrastructural challenges and high cost of the technology. Amazon Cluster GPU Instances remove this barrier by providing developers and businesses immediate access to the highly tuned compute performance of GPUs with no upfront investment or long-term commitment.
Amazon Cluster GPU Instances provide 22 GB of memory, 33.5 EC2 Compute Units, and utilize the Amazon EC2 Cluster network, which provides high throughput and low latency for High Performance Computing (HPC) and data intensive applications. Each GPU instance features two NVIDIA Tesla® M2050 GPUs, delivering peak performance of more than one trillion double-precision FLOPS. Many workloads can be greatly accelerated by taking advantage of the parallel processing power of hundreds of cores in the new GPU instances. Many industries including oil and gas exploration, graphics rendering and engineering design are using GPU processors to improve the performance of their critical applications.
Amazon Cluster GPU Instances extend the options for running HPC workloads in the AWS cloud. Cluster Compute Instances, launched earlier this year, provide the ability to create clusters of instances connected by a low latency, high throughput network. Cluster GPU Instances give customers with HPC workloads an additional option to further customize their high performance clusters in the cloud. For those customers who have applications that can benefit from the parallel computing power of GPUs, Amazon Cluster GPU Instances can often lead to even further efficiency gains over what can be achieved with traditional processors. By leveraging both instance types, HPC customers can tailor their compute cluster to best meet the performance needs of their workloads. For more information on HPC capabilities provided by Amazon EC2, visit aws.amazon.com/ec2/hpc-applications.
Now that Amazon EBS, or Elastic Block Store is publicly available, a lot of people are probably interested in some benchmarks!
The setup;
All that was installed was mdadm & bonnie++.
Mounting a single 1gb EBS volume to /dev/sdd,
dd if=/dev/zero of=/dev/sdd bs=128k
dd: writing `/dev/sdd': No space left on device 8193+0 records in 8192+0 records out 1073741824 bytes (1.1 GB) copied, 16.9026 s, 63.5 MB/s
dd if=/dev/zero of=/dev/sdd bs=256k
dd: writing `/dev/sdd': No space left on device 4097+0 records in 4096+0 records out 1073741824 bytes (1.1 GB) copied, 16.7213 s, 64.2 MB/s
dd if=/dev/zero of=/dev/sdd bs=512k
dd: writing `/dev/sdd': No space left on device 2049+0 records in 2048+0 records out 1073741824 bytes (1.1 GB) copied, 16.5371 s, 64.9 MB/s
dd if=/dev/zero of=/dev/sdd bs=1M
dd: writing `/dev/sdd': No space left on device 1025+0 records in 1024+0 records out 1073741824 bytes (1.1 GB) copied, 16.9271 s, 63.4 MB/s
I attached another 1gb disc, /dev/sde, and then created a new md device.
mdadm --create /dev/md0 --level=0 -n 2 /dev/sd[d-e]
Repeating the above tests - obviously this is now over 2gb of disc;
dd if=/dev/zero of=/dev/md0 bs=128k
dd: writing `/dev/md0': No space left on device 16384+0 records in 16383+0 records out 2147352576 bytes (2.1 GB) copied, 22.8826 s, 93.8 MB/s
dd if=/dev/zero of=/dev/md0 bs=256k
dd: writing `/dev/md0': No space left on device 8192+0 records in 8191+0 records out 2147352576 bytes (2.1 GB) copied, 22.2414 s, 96.5 MB/s
dd if=/dev/zero of=/dev/md0 bs=512k
dd: writing `/dev/md0': No space left on device 4096+0 records in 4095+0 records out 2147352576 bytes (2.1 GB) copied, 21.7883 s, 98.6 MB/s
dd if=/dev/zero of=/dev/md0 bs=1M
dd: writing `/dev/md0': No space left on device 2048+0 records in 2047+0 records out 2147352576 bytes (2.1 GB) copied, 20.9595 s, 102 MB/s
dd if=/dev/zero of=/dev/md0 bs=2M
dd: writing `/dev/md0': No space left on device 1024+0 records in 1023+0 records out 2147352576 bytes (2.1 GB) copied, 21.9773 s, 97.7 MB/s
I attached another 1gb disc, /dev/sdf, and then stopped md0 and replaced it with a new md device.
mdadm -S /dev/md0 mdadm --zero-superblock /dev/sd[d-e] mdadm --create /dev/md0 --level=0 -n 3 /dev/sd[d-f]
Repeating the above tests - obviously this is now over 3gb of disc;
dd if=/dev/zero of=/dev/md0 bs=128k
dd: writing `/dev/md0': No space left on device 24575+0 records in 24574+0 records out 3221028864 bytes (3.2 GB) copied, 30.4635 s, 106 MB/s
dd if=/dev/zero of=/dev/md0 bs=256k
dd: writing `/dev/md0': No space left on device 12288+0 records in 12287+0 records out 3221028864 bytes (3.2 GB) copied, 30.0373 s, 107 MB/s
dd if=/dev/zero of=/dev/md0 bs=512k
dd: writing `/dev/md0': No space left on device 6144+0 records in 6143+0 records out 3221028864 bytes (3.2 GB) copied, 29.601 s, 109 MB/s
dd if=/dev/zero of=/dev/md0 bs=1M
dd: writing `/dev/md0': No space left on device 3072+0 records in 3071+0 records out 3221028864 bytes (3.2 GB) copied, 30.083 s, 107 MB/s
dd if=/dev/zero of=/dev/md0 bs=2M
dd: writing `/dev/md0': No space left on device 1536+0 records in 1535+0 records out 3221028864 bytes (3.2 GB) copied, 29.4059 s, 110 MB/s
Time to test with some redundancy, so I stopped md0 and replaced it with a new md device as follows;
mdadm -S /dev/md0 mdadm --zero-superblock /dev/sd[d-f] mdadm --create /dev/md0 --level=5 -n 3 /dev/sd[d-f]
After waiting for the resync to finish (which took about 2 minutes), I repeated the above tests - obviously this is now over 2gb-ish of disc;
dd if=/dev/zero of=/dev/md0 bs=128k
dd: writing `/dev/md0': No space left on device 16384+0 records in 16383+0 records out 2147352576 bytes (2.1 GB) copied, 39.8346 s, 53.9 MB/s
dd if=/dev/zero of=/dev/md0 bs=256k
dd: writing `/dev/md0': No space left on device 8192+0 records in 8191+0 records out 2147352576 bytes (2.1 GB) copied, 36.197 s, 59.3 MB/s
dd if=/dev/zero of=/dev/md0 bs=512k
dd: writing `/dev/md0': No space left on device 4096+0 records in 4095+0 records out 2147352576 bytes (2.1 GB) copied, 35.9269 s, 59.8 MB/s
dd if=/dev/zero of=/dev/md0 bs=1M
dd: writing `/dev/md0': No space left on device 2048+0 records in 2047+0 records out 2147352576 bytes (2.1 GB) copied, 35.9784 s, 59.7 MB/s
dd if=/dev/zero of=/dev/md0 bs=2M
dd: writing `/dev/md0': No space left on device 1024+0 records in 1023+0 records out 2147352576 bytes (2.1 GB) copied, 36.8876 s, 58.2 MB/s
mdadm -S /dev/md0 mdadm --zero-superblock /dev/sd[d-f] mdadm --create /dev/md0 -n 3 --level=10 --layout=f2 /dev/sd[d-f]
After waiting for the resync to finish (which took about 2 minutes), I repeated the above tests - obviously this is now over 2gb-ish of disc;
dd if=/dev/zero of=/dev/md0 bs=128k
dd: writing `/dev/md0': No space left on device 12287+0 records in 12286+0 records out 1610416128 bytes (1.6 GB) copied, 30.449 s, 52.9 MB/s
dd if=/dev/zero of=/dev/md0 bs=256k
dd: writing `/dev/md0': No space left on device 6144+0 records in 6143+0 records out 1610416128 bytes (1.6 GB) copied, 29.9024 s, 53.9 MB/s
dd if=/dev/zero of=/dev/md0 bs=512k
dd: writing `/dev/md0': No space left on device 3072+0 records in 3071+0 records out 1610416128 bytes (1.6 GB) copied, 30.4268 s, 52.9 MB/s
dd if=/dev/zero of=/dev/md0 bs=1M
dd: writing `/dev/md0': No space left on device 1536+0 records in 1535+0 records out 1610416128 bytes (1.6 GB) copied, 29.5757 s, 54.5 MB/s
dd if=/dev/zero of=/dev/md0 bs=2M
dd: writing `/dev/md0': No space left on device 768+0 records in 767+0 records out 1610416128 bytes (1.6 GB) copied, 31.57 s, 51.0 MB/s
It seems that using RAID does have its advantages over a single disc, even on EBS. However, I would have expected RAID 10 F2 to have similar performance to RAID 0, rather than the results seen.
I will retest soon with some other benchmarks, and configurations with more Volumes!
Russell
