As a developer, I would like to offer my view here:
>Why Agentic AI requires more CPU:
>Orchestration of Tasks: CPUs act as the brain that will decide what tools to call, which tasks to perform in which order, and monitor the Agent’s progress towards a goal.
= In agentic task like coding, LLM decides which tool to call. CPU just executes instructions from LLM.
>Latency during Tool Calling: In an agentic workflow, CPU latency can account for up to 90% of the agent’s overall latency. Hence, the CPU is now critical for Agents to complete work on time.
= Most of agentic task is to get data from to LLM or write data to somewhere. Depending on the task, it can be network or storage bound (reading files, DB, or web). Task is rarely CPU bound. Only task like compilation is CPU bound.
> Continuous Operations: Agents may often run 24/7, necessitating always-on Server CPUs.
= CPU always run 24/7 on server. Nothing new here.
Thanks for your comment. Really insightful and very helpful for us to learn more about this trend. Are you seeing any other reason why server CPU prices are suddenly going up - inspite of the rise of ARM CPUs (which have taken share from AMD and Intel).
I think the core idea is still valid: AI data-center build-out increases server CPU demand. After all, new data centers still need CPUs to run. However, the proportional increase is likely to be smaller than that of GPUs and memory/storage, since CPUs are typically neither the primary bottleneck nor the main performance driver for model training or inference.
One supporting scenario is that, as agents become more powerful, more tasks will move to the cloud. Agents can open their own browsers, write code, and compile or test it remotely instead of relying on the user’s PC. Even today, I can code on my mobile phone and let the agent compile and test everything in the cloud. At scale, this could meaningfully expand server CPU demand in the future.
As a developer, I would like to offer my view here:
>Why Agentic AI requires more CPU:
>Orchestration of Tasks: CPUs act as the brain that will decide what tools to call, which tasks to perform in which order, and monitor the Agent’s progress towards a goal.
= In agentic task like coding, LLM decides which tool to call. CPU just executes instructions from LLM.
>Latency during Tool Calling: In an agentic workflow, CPU latency can account for up to 90% of the agent’s overall latency. Hence, the CPU is now critical for Agents to complete work on time.
= Most of agentic task is to get data from to LLM or write data to somewhere. Depending on the task, it can be network or storage bound (reading files, DB, or web). Task is rarely CPU bound. Only task like compilation is CPU bound.
> Continuous Operations: Agents may often run 24/7, necessitating always-on Server CPUs.
= CPU always run 24/7 on server. Nothing new here.
Thanks for your comment. Really insightful and very helpful for us to learn more about this trend. Are you seeing any other reason why server CPU prices are suddenly going up - inspite of the rise of ARM CPUs (which have taken share from AMD and Intel).
I think the core idea is still valid: AI data-center build-out increases server CPU demand. After all, new data centers still need CPUs to run. However, the proportional increase is likely to be smaller than that of GPUs and memory/storage, since CPUs are typically neither the primary bottleneck nor the main performance driver for model training or inference.
One supporting scenario is that, as agents become more powerful, more tasks will move to the cloud. Agents can open their own browsers, write code, and compile or test it remotely instead of relying on the user’s PC. Even today, I can code on my mobile phone and let the agent compile and test everything in the cloud. At scale, this could meaningfully expand server CPU demand in the future.