With Intel to secure, efficient and reliable LLMs

In my last blog post on the topic of large language models (LLMs), I asked (and answered) how companies can “build” their own AI applications based on existing LLMs without neglecting important business principles. These include, in particular, data protection, the reliability and timeliness of the underlying data.

A way out of the possible dilemmas is called Retrieval Augmented Generation (RAG)a technique that combines existing language models such as ChatGPT with your own databases in the best possible way. Today's article will show what other possibilities exist, with the help of Intel and its technology partners.

First of all, it should be said that Intel is very aware of its responsibility regarding generative AI and all its effects and therefore takes it very seriously. For this reason, the chip manufacturer is continuously making extensive efforts to reduce the risks posed by existing and frequently used language models and other AI data as much as possible. This is often done with technology partners such as Prediction Guard, Seekr, Storm Reply and Winning Health Technology, which we will be talking about today. This list can of course be extended at will.

Prediction Guard: More data protection in AI applications

With the startup company Prediction Guard, Intel has brought on board the concentrated knowledge of data security and data quality of AI applications, as Daniel Whitenack and his team have been an important part of the Intel Liftoff program for AI startups for quite some time. What is special about the Prediction Guard solution is its relatively easy integration into existing AI enterprise solutions that, on the one hand, want to use the power of existing LLMs, but on the other hand, deal with existing data.

Prediction Guard can be used to examine existing corporate applications for potential vulnerabilities related to the use of the underlying LLMs. This is done simply using line-based queries, as known from applications such as ChatGPT. Does the language model used conflict with applicable law? Are there any potential trade secrets being revealed? These and other questions can be answered interactively using simple questions.

Seekr: AI platform implementation and podcast analysis

With seerAlign and seenrFlow, complete AI platforms and applications can be created and brought to life with relatively little know-how, without companies having to think about the “how”. It is not without reason that multinational companies such as Oracle and Babbel rely on the possibilities offered by both Seekr solutions offer. In a first step, seerFlow creates a secure AI application based on the available data. This happens on any hardware platform that has been designed and optimized for seenrFlow. What's special about it is the ability to further refine the application after it has been initially created using simple line commands.

SeekrAlign, on the other hand, is aimed at advertisers, publishers and marketplace operators who want to expand their reach in a safe and reliable way using suitable podcasts and other media formats. They are supported in the best possible way by AI functions such as Seekr Civility Score and with the highest level of transparency.

Storm Reply: Optimized AWS instances for best possible inference

As a long-standing AWS Premier Consulting Partner, Storm Reply knows exactly what is important in public cloud services. One service involves implementing large language models (LLMs) on Amazon Elastic Compute Cloud (EC2) C7i and C7i Flex instances. These use 4th generation Intel Xeon processors in combination with Intel libraries that are specifically designed for creating and managing language models (LLMs). Storm Reply also supports the Intel GenAI platform and the open source LLaMA model (Large Language Model Meta AI). This enables RAG-based AI applications.

Winning Health Technology: Adapted LLMs for Healthcare

The entire healthcare sector can be one of the winners when it comes to the use of generative artificial intelligence. However, this requires large language models (LLMs) that require powerful computing platforms, which are often lacking in the healthcare sector. A language model called WiNGPT, developed by Winning Health Technology, takes this into account.

This healthcare-specific LLM has been specifically adapted and optimized for use on Intel-based mainframes. The result is a language model that infers 3x faster on 5th generation Intel Xeon processors than on the same 3rd generation CPU. A key reason for this is the intensive use of certain AI accelerators such as Intel AMX (Advanced Matrix Extensions).

Disclaimer: Intel commissioned me to write and publish this blog post. I had almost free rein in designing the content.