AT&T has significantly restructured its AI orchestration to manage the processing of 8 billion tokens daily by shifting from large reasoning models to a system centered around small language models (SLMs), achieving a 90% reduction in costs. This adaptation, driven by Andy Markus and his team, involves using a multi-agent stack with the LangChain framework, where “super agents” direct smaller, task-specific “worker” agents. This approach not only enhanced efficiency and reduced latency but also aligns with recent trends in enterprises adopting SLMs for domain-specific tasks to maintain accuracy and improve scalability. Through this innovation, AT&T has implemented the Ask AT&T Workflows for over 100,000 employees, facilitating both professional and non-technical users in task automation and AI-driven software development, further demonstrating the flexibility and cost-effectiveness of their rearchitected AI system.
AT&T cuts costs 90% by rearchitecting AI orchestration
