I often spot trends by looking for common patterns in the questions reporters ask me. In most cases, they are much more in touch with the market than I am and are a good data point.
First, if this happens, it probably won’t last long. Second, other viable options should be considered. Of course, the outlook is doom and gloom, with concerns that if businesses can’t get these processors for use in on-premises systems or in the cloud, they won’t be able to take advantage of the productive AI revolution. On demand.
Fake problem?
I’m the first to admit that generative AI systems are complex and processor-intensive. So the assumption is that they must rely on highly specialized hardware to perform tasks that were once the exclusive domain of the human imagination. People think generative AI needs GPUs or even more specialized processing like quantum computing.
Are these assumptions always true? Is this another custom system where special components are needed at very special prices?
GPUs were originally developed for rendering graphics in video games but have become effective in artificial intelligence due to their highly parallel nature. They can perform thousands of operations simultaneously. This fits perfectly with the tasks required by neural networks, the critical technology of generative AI. This is a technical fact that people who design and build productive AI systems (like yours, really) should carefully consider.
Tensor Processing Units (TPUs), on the other hand, are custom-developed, application-specific integrated circuits designed specifically for Google’s TensorFlow. TensorFlow is an open source machine learning framework that has been around for a while. TPUs aid machine learning processes because they are designed for forward and backward propagation. These are the processes used to train neural networks. I don’t see TPUs as much of an issue as GPUs when it comes to cost. But they are often interconnected, so they are worth mentioning here.
Those of you who build and deploy these systems know that no matter what AI framework you use, most of the processing and time is spent training models from masses of data. Consider, for example, OpenAI’s GPT-4 or Google’s BERT models, which have billions of parameters. Training such models without dedicated processors can take an impractically long time.
Are dedicated processors always needed?
GPUs greatly increase performance, but they do so at a significant cost. Also, for those chasing carbon points, GPUs consume a significant amount of electricity and generate a significant amount of heat. Do the performance gains justify the cost?
CPUs are the most common type of processor in computers. They’re everywhere, including everything you used to read this article. CPUs can perform a wide variety of tasks and have fewer cores compared to GPUs.
However, they have advanced control units and can execute a wide variety of instructions. This versatility means they can handle AI workloads, such as use cases that need to leverage all types of AI, including generative AI.
How much do you really need to pay?
CPUs are more cost-effective in terms of initial investment and power consumption for smaller organizations or people with limited resources. But even for businesses with lots of resources, these can still be a more cost-effective choice.
Also, artificial intelligence is developing. With the latest developments in artificial intelligence algorithms, new developments such as SLIDE (Sub-Linear Deep Learning Engine) are also occurring. This technology claims to train deep neural networks faster on CPUs than on GPUs under certain conditions. They use hashing techniques and reduce memory access costs.
Also consider field-programmable gate arrays (FPGAs). These processors can be programmed after production to perform specific tasks, such as artificial intelligence, much more efficiently. Additionally, associative processing units (APUs) specialize in pattern recognition and perform associative memory tasks, allowing certain types of neural network applications to run faster.
There are many examples where non-GPU processors are much more cost-effective. So why is it that when it comes to generative AI, or just AI in general, the answer is always GPUs? I’m not sure it should be.