Product Category Memo #5: LLM-Infused Products
Product Category Memo #5: LLM-Infused Products
In-depth analysis of products that are built using Large Language Models (LLMs)
Hello friends,
Welcome to the 5th edition of Product Category Memo. The goal of this segment is to do an in-depth analysis of a specific product category in AI, ML, and Data. We’ll talk about product description, what users expect from this product, landscape of products, pricing mechanics, and growth strategy.
In this post, we’ll talk about:
Why do we need products that use Large Language Models (LLMs)
Who needs this product and what do they want
What products are competing in this market
What factors drive pricing
How these products acquire customers
This post is not about LLMs. This post is about products that use LLMs to address a specific use case.
If you have a question, submit it here and I’ll get back to you with my thoughts. Subscribe to this newsletter to receive it in your inbox every week:
I wanted to a human and a robot writing together, so I asked DALL-E to generate an image for me. And this is what it came up with. Let’s dive in.
Why do we need products that use LLMs?
Products that need to understand text input or generate text output tend to benefit from using LLMs. These products make us faster/better by automating part of the work. They use LLMs to solve a specific problem.
Copy AI is a good example of a product that uses LLMs to generate text for its users. GitHub Copilot is a good example of a product that generates code for its users.
Who needs this product and what do they want?
These products are usually built by machine learning practitioners, but software engineers can build them too. The users of these products could be anybody e.g. copywriters, marketers, HR, customer support, software engineers.
We now have APIs available that mask the complexity of LLMs. This helps software engineers use LLMs to build products. They can ping an API and get a response.
ML practitioners can go a step further and use an open source LLM that can be refined further with a domain-specific dataset. Open source models have leveled the playing field. So in order to build differentiation, ML practitioners fine-tune those models using domain-specific datasets that are proprietary. This gives them an edge in the market.
The users need a product that fits within their workflow. Let's consider the example of copywriters. They need a product that's integrated into the tool they use to write and edit their content. The same is true with developers. They need a product that's integrated into the IDE they use to write their code.
What use cases can be addressed with LLMs?
There are two big themes here:
Applying LLMs to text
Applying LLMs to code
We can apply LLMs to text in the following ways:
Generating text: You can ask an AI system to generate a paragraph or even an entire blog post. You can also ask it to generate product documentation.
Enhancing existing text: You can ask an AI system to enhance something you've already written.
Summarizing text: You can ask an AI system to summarize product feedback or customer reviews from thousands of users.
Searching through text: You can ask an AI system to organize all your text corpus and make it searchable in plain english.
Translation: You can ask an AI system to convert text from one language to another while maintaining the context. This can be used for real-time speech translators as well.
Conversing with humans: You can ask an AI system to talk to a customer and understand why they're contacting customer support before routing it to a human.
We can apply LLMs to code in the following ways:
Generating code: You can ask an AI system to generate code by telling it what you need in plain english. You can also ask it to generate shell commands, regex, SQL queries, and websites.
Optimizing existing code: You can ask the AI system to review your existing code and suggest optimizations. You can also ask it to optimize database queries and assist devops engineers as they deploy their code into production.
What products are competing in this market?
Here are the categories of products that are competing in this market:
Open source models:
BLOOM and DistilBERT - Hugging Face
OPT-175B, RoBERTa, and XLM-RoBERTa - Meta
GPT-Neo, GPT-J, and GPT-NeoX - EleutherAI
XLNet, BERT - CMU and Google
DeBERTa - Microsoft
APIs that provide access to LLMs:
OpenAI GPT-3
Cohere
Products for general purpose writing:
Writer
Lex
Compose AI
Wordtune
AI-Writer
Rytr
Products for copywriting and marketing:
Copy AI
Jasper
Regie AI
Copysmith
Anyword
Smart Copy (by Unbounce)
Products for sales emails:
Smartwriter AI
Reply
Lavender
Lyne AI
Products that can write code and be your pair-programmer:
GitHub Copilot (built using OpenAI Codex)
Replit Ghostwriter
Visual Studio IntelliCode
Amazon CodeWhisperer
Tabnine
Kite
Products that can generate documentation:
Mintlify
Stenography
Products to generate SQL queries:
Cogram
Products that generate shell commands:
Warp
What factors drive pricing?
Depending on the sub-sector, the pricing is dependent on a subset of the following metrics:
Size of the model being used
Volume of text analyzed
Processing speed needed
Amount of compute power used
Number of API calls
Number of details extracted from the input text
Number of users
How do these products acquire customers?
These products can use the following ways to acquire customers:
Freemium model: These products need individual users to adopt. That's how a product makes its way into an organization. Once it's in, you can approach the company directly to sign a bigger deal.
Integration capabilities: A product needs to play well with all the tools along the value chain. If your product doesn't integrate well, it won't be adopted by users.
Built-in collaboration: Once a user starts using a product, they should be able to invite their team members. Inherently collaborative products tend to gain adoption quickly.
If you are getting value from this newsletter, consider subscribing and sharing with your friends: