Fastly unveils AI Accelerator for Azure AI & ChatGPT

Fastly has announced the general availability of its AI Accelerator, expanding support to Microsoft Azure AI Foundry alongside OpenAI ChatGPT.

Fastly, known for its edge cloud platforms, has made its Fastly AI Accelerator widely accessible, addressing key performance and cost issues in the development of Large Language Model (LLM) generative AI applications. The tool promises to enhance efficiency, offering an average of nine times faster response times, reflecting improvements noted during its beta phase.

Underlining the goals behind this development, Kip Compton, Chief Product Officer at Fastly, stated: "AI is helping developers create so many new experiences, but too often at the expense of performance for end-users. Too often, today's AI platforms make users wait. With Fastly AI Accelerator we're already averaging 9x faster response times and we're just getting started. We want everyone to join us in the quest to make AI faster and more efficient."

The implementation of the AI Accelerator requires developers to update their applications with a new API endpoint, a task that typically involves a single line of code alteration. This setup allows Fastly AI Accelerator to utilise its semantic caching abilities effectively, reducing the need for repeated calls to the AI provider and leveraging the Fastly Edge Cloud Platform to cache responses, thus improving performance and cost-efficiency.

Dave McCarthy, Research Vice President of Cloud and Edge Services at IDC, commented on the significance of this tool: "Fastly AI Accelerator is a significant step towards addressing the performance bottleneck accompanying the generative AI boom. This move solidifies Fastly's position as a key player in the fast-evolving edge cloud landscape. The unique approach of using semantic caching to reduce API calls and costs unlocks the true potential of LLM generative AI apps without compromising on speed or efficiency, allowing Fastly to enhance the user experience and empower developers."

Existing customers of Fastly can integrate the AI Accelerator directly via their Fastly accounts. The company states that its edge cloud platform offers a combination of speed, security, and programmability, enabling organisations to provide enhanced online experiences. Prestigious brands such as Reddit, Neiman Marcus, Universal Music Group, and SeatGeek are among those that trust Fastly to improve internet experiences.