Discussion about this post

User's avatar
Gab's avatar

As a developer, I would like to offer my view here:

>Why Agentic AI requires more CPU:

>Orchestration of Tasks: CPUs act as the brain that will decide what tools to call, which tasks to perform in which order, and monitor the Agent’s progress towards a goal.

= In agentic task like coding, LLM decides which tool to call. CPU just executes instructions from LLM.

>Latency during Tool Calling: In an agentic workflow, CPU latency can account for up to 90% of the agent’s overall latency. Hence, the CPU is now critical for Agents to complete work on time.

= Most of agentic task is to get data from to LLM or write data to somewhere. Depending on the task, it can be network or storage bound (reading files, DB, or web). Task is rarely CPU bound. Only task like compilation is CPU bound.

> Continuous Operations: Agents may often run 24/7, necessitating always-on Server CPUs.

= CPU always run 24/7 on server. Nothing new here.

2 more comments...

No posts

Ready for more?