Did you know you can extract brand awareness data from large-language models (LLMs)?
Since LLMs contain vast amounts of information about different brands, we can measure their popularity, reach, and sentiment.
Randomised LLM Outputs
In a nutshell, LLMs work by guessing the next word in a sentence based on the probability of it showing up.
In this great LLM generation example from NVIDIA, you can see that there are many options for the next word, and often one of those is the most common:
By default, many LLMs pick results with a degree of randomness to make the output more varied and interesting. For example, if we run this prompt on pizza toppings:
And then run the same prompt again on a fresh chat:
Notice that on both lists we have these same bad pizza toppings:
- Anchovies
- Pineapple
- Tuna
- Jalapenos
- Egg
- Clam (?!)
But these pizza toppings are completely unique on the two duplicate prompts:
- Sausage
- Onion
- Mushroom
- Olives
- Pickles
- Banana (!)
- Sardines
Removing ‘randomness’ from LLM results
The key is to modify the prompt behind the scenes so that the LLM settings eliminate randomness for reliable outputs.
The easiest place to do this is on OpenAI’s playground where you can use ChatGPT in a stable manner. Look for the ‘Temperature’ setting, and set it to zero for the least random output you can achieve:
On other LLMs you’ll need to edit the request of the API payload (via curl or Python), such as this one for Google Gemini Pro:
Extracting Brand Awareness Data from LLMs
Now we can get non-randomised reliable data from LLMs, you can craft prompts to extract valuable insights towards Public Relations (PR) or Search Engine Optimization (SEO).
For brand awareness metrics, you could use a location and topic-based prompt such as:
Measuring a brand’s reach can reveal opportunities for SEO with a prompt such as:
This can in turn, lead down to more niche prompts that combine a topic with different brand sentiments such as:
As LLMs become more prominent and intertwined with search engines, it’s important to understand your brand’s place and how well your competitors are doing.
This will provide you with golden opportunities to improve your brand prevalence, brand perception, and how you can improve your content to answer people’s concerns ahead of time.
Get the most out of your data
If you’d like to find out more about how Hallam can help you measure your brand awareness, please get in touch.