{"id":3392,"date":"2023-09-02T02:06:20","date_gmt":"2023-09-01T21:06:20","guid":{"rendered":"https:\/\/jumpshare.com\/blog\/?p=3392"},"modified":"2023-09-02T02:06:20","modified_gmt":"2023-09-01T21:06:20","slug":"deep-learning-on-aws-graviton2-nvidia-tensor-t4g-for-as-low-as-free-with-cuda-12-2","status":"publish","type":"post","link":"https:\/\/jumpshare.com\/blog\/deep-learning-on-aws-graviton2-nvidia-tensor-t4g-for-as-low-as-free-with-cuda-12-2\/","title":{"rendered":"Deep Learning on AWS Graviton2, NVIDIA Tensor T4G for as Low as Free with CUDA-12.2"},"content":{"rendered":"
\n
\n
\"\"<\/figure>\n
\n
<\/figure>\n

Note:<\/strong> The \u201cas low as free\u201d tagline is based on g5g.xlarge spot instance rates, which have been as low as $0.1519\/hr.<\/em><\/p>\n

Introduction<\/h2>\n

The world we live in today heavily relies on artificial intelligence. From vacuum bots to sales support, from self-driving cars to disease detection, from finding the content you want to consume to translating from a foreign language to your native one. AI is behind every great product out there, and the need for an efficient, cost-effective, and scalable deep learning architecture has never been more critical.<\/p>\n

The G5g instances powered by Amazon\u2019s own Graviton2 processor and also feature NVIDIA T4G Tensor Core GPUs are a cost-effective alternative to Intel\u2019s and AMD\u2019s powered instances for deploying deep learning applications.<\/p>\n

The Dilemma<\/h2>\n

AWS offers robust, powerful, cost-effective architecture for running artificial intelligence and deep learning tasks. One of the advantages is the option to use spot instances, which are far more cost-effective at times and up to 70% cheaper than on-demand instances.<\/p>\n

For example, the spot pricing history for the g5g.xlarge<\/code> instance in various \u201cus-east\u201d zones ranged from $0.1720<\/strong> to $0.1519<\/strong> per hour for the past three months. These rates are tempting, but at the time of writing, no official Amazon Linux 2023<\/strong> Deep learning AMI is available for the Amazon G5g<\/strong> instances family. Setting up the environment can be cumbersome: finding drivers, the correct dev toolchain, and a pre-compiled PyTorch module supporting the latest DL toolkit<\/p>\n

\"\"<\/p>\n

Spot price history for g5g.xlarge for the last three months.<\/em><\/p>\n

Navigating the Challenge\u200a\u2014\u200aA How-To\u00a0Guide<\/h2>\n

This aims to bridge the gap by offering comprehensive step-by-step instructions suitable for newcomers and seasoned data scientists. The goal is to enable you to leverage these state-of-the-art technologies at a meager cost without the hassle of finding the right driver and packages for the G5g family. Eventually, we will compile all the individual steps into a single script that will further streamline the process.<\/p>\n

1. Launching an\u00a0Instance<\/h3>\n

For setting up an instance, we\u2019ll use g5g.4xlarge<\/code> instance. The idea behind using a more powerful instance is to accelerate compilation time. We will launch the build instance with the AWS Command Line Interface (aws cli).<\/p>\n

First, set the following environment variables:<\/p>\n