How Edge-Based LLMs Could Alleviate AI Data Center Strain

Hey there,

Edge-based LLMs, running directly on devices like smartphones and laptops, offer a solution to alleviate the strain on AI data centers. By reducing training costs, latency, and energy consumption, and enhancing privacy, edge AI could shift the future of AI processing away from massive data centers. Companies like Huawei and Samsung are leading the way with smaller, efficient models.

TODAY’S ROADMAP

• Current Happenings in AI World

• Smartphones Lead the Charge in Reducing Data Center Burden with Edge AI

• Create Custom Presentations Effortlessly with Gamma's AI Tools

• 5 AI Tools to Boost Your Productivity

  and more…

NEWS

Current Happenings in AI World

Leveling the Field: OpenAI is offering $1M in API credits to developers in lower-income countries, who often lack access to cutting-edge AI models.

Ad Space: The AI search engine Perplexity may soon introduce ads to compete with Alphabet.

Video Shift: Black Forest Labs, the startup behind Grok’s popular image generator, is raising $100M to launch its first text-to-video model.

Meta Insights: Sources told The Information that WhatsApp is the leading platform for Meta’s new AI features, with Facebook close behind.

AI UPDATE

Smartphones Lead the Charge in Reducing Data Center Burden with Edge AI

The issue of AI’s impact on data center power has been widely discussed. A promising solution is the use of “LLMs on the edge,” which allows AI systems to run directly on PCs, tablets, laptops, and smartphones.

The clear advantages of LLMs on the edge include reducing the cost of training these models, decreasing latency in querying, enhancing user privacy, and improving reliability.

By easing the pressure on data centers and lowering processing power needs, LLMs on the edge could potentially eliminate the necessity for massive, energy-intensive AI data center factories. But is this approach truly feasible? As discussions about moving the LLMs that support generative AI to the edge grow, we examine whether this shift can genuinely alleviate data center strain. Smartphones Lead the Way in Edge AI

Michael Azoff, chief analyst for cloud and data center research at Omdia, notes that the fastest-moving AI-on-the-edge use case is lightweight LLMs on smartphones.

Huawei has developed various sizes of its LLM Pangu 5.0, with the smallest version integrated into its HarmonyOS smartphone operating system. Devices like the Huawei Mate 30 Pro 5G run this system.

Samsung has created the Gauss LLM, used in Samsung Galaxy AI, which operates in its flagship Samsung S24 smartphone. Its AI features include live translation, voice-to-text conversion, note summarization, search by circling, and photo and message assistance. Samsung has also begun mass production of its LPDDR5X DRAM semiconductors. These 12-nanometer chips process memory workloads directly on the device, enabling faster operation with storage devices to handle AI workloads more efficiently.

Overall, smartphone manufacturers are striving to make LLMs smaller. Instead of ChatGPT-3’s 175 billion parameters, they aim to reduce them to around two billion parameters.

Intel and AMD are also involved in AI at the edge. AMD is developing notebook chips capable of running 30 billion-parameter LLMs locally at speed. Similarly, Intel has assembled a partner ecosystem to develop AI PCs. These AI-enabled devices may be more expensive than regular models, but the price difference is expected to decrease significantly as adoption increases.

“The expensive part of AI at the edge is mostly on the training,” Azoff told Data Center Knowledge. “A trained model used in inference mode does not need expensive equipment to run.”

AI AT JOB

Create Custom Presentations Effortlessly with Gamma's AI Tools

• Sign up for Gamma

• Paste text, upload a file, or submit a prompt for Gamma to build from (chose the prompt option)

• Describe the slide content, then select the number of slides and preferred language

• You can also use its AI to find or create images for the slides*

• Click ‘Generate’

• If necessary, adjust colors and design

• Finally, present and share!

AI MEETS PRODUCTIVITY

5 AI Tools to Boost Your Productivity

• Wispr: Use your voice to type three times faster than your keyboard — anytime, anywhere.

• Nodeland: Transform your notes into interactive mind maps with AI.

• Spinach AI: Your Meeting Copilot - takes accurate meeting notes, drafts recap emails, and adds action items to Jira, Asana, Monday, Trello, or ClickUp. Try it here.

• Filmora: Effortlessly create with a fully-equipped video editing suite powered by AI.

• PDF2Audio: Convert PDFs into an audio podcast, lecture, or summary with this open-source tool.

PROMPT OF THE DAY

In-Depth Match Analysis: Manchester United vs Chelsea Unfolds

prompt: I want you to act as a football commentator. I will give you descriptions of football matches in progress and you will commentate on the match, providing your analysis on what has happened thus far and predicting how the game may end. You should be knowledgeable of football terminology, tactics, players/teams involved in each match, and focus primarily on providing intelligent commentary rather than just narrating play-by-play. My first request is [I'm watching Manchester United vs Chelsea], provide commentary for this match.

Provide insightful football commentary, analyzing match progress, tactics, and predicting outcomes, starting with Manchester United vs Chelsea.

AI CRAFTED IMAGES

Water colour

Prompt:
watercolor and acrylic painting of a beautiful nomadic woman wearing a top hat with tall fancy jewelry, in the aesthetic style of Alexander Jansson, Anna Dittman, and Camilla d'Errico

✨🙌✨ We are at your service

We appreciate you taking the time to read this.

See you in the next one!

Warm regards,

Team AI Junction

P.S. Enjoyed the newsletter? Feel free to pass it along to your friends and colleagues here. Your thoughts and feedback are valuable to us.