Welcome to Infinite Curiosity, a weekly newsletter that explores the intersection of Artificial Intelligence and Startups. Tech enthusiasts across 200 countries have been reading what I write. Subscribe to this newsletter for free to receive it in your inbox every week:
Hello friends,
As we enter 2023, I wanted to share my 15 AI predictions for the year. Here’s the tldr list:
LLMs will change the nature of search
Generative AI infrastructure companies will be on the rise
Consumer-facing AI applications will become popular
GPT-4 will focus on reducing the cost of servicing for OpenAI rather than increasing the number of model parameters
Data volume as a moat will end
Foundation model for the industrial world will emerge
AI hardware supply chain will become a geopolitical flashpoint
On-device AI will take off big time as privacy laws get stricter
Grand unified model architecture for cloud and on-device applications
Demand for compute power will skyrocket
A "model streaming" service will emerge
The volume of AI-created video on social platforms will explode
AI-enabled social engineering will rise
"Self-driving" DevOps will rise
Distributed model training will start taking shape
That’s a fortune teller looking into the future to make predictions about AI. This post is different though. Let’s dive in.
1. LLMs will change the nature of search
2023 will mark the biggest challenge to the concept of search ever since Google first ascended to the throne. Let's look at the premise of this problem. No company will be able to challenge Google head on when it comes to using a search tool. They have too much mindshare, too many resources, and too much market dominance. So how can a new company approach it? By attacking the underlying problem.
Why do people use a search engine? To get answers to their questions. Google has built a product to understand the user's intent and bring up relevant web pages. The user then has to click on a couple of them. And figure out what's the right answer for them. But what if you just show the answer right there. There's no need to click anything. OpenAI's ChatGPT has demonstrated that a large number of people are willing to use a different type of tool to get answers to their questions.
2. Generative AI infrastructure companies will be on the rise
Now that OpenAI has opened the floodgates, many developers are building AI applications. This means they need tooling to build their applications. Big future winners in AI infrastructure will be launched this year.
3. Consumer-facing AI applications will become popular
AI has long been used in the backend to enhance software applications. The consumer-facing applications have been toy applications designed to do non-critical things e.g. replacing your face with a cat in a live video. One of the biggest points why DALL-E or ChatGPT became popular is that it focused on the consumer. These tools are spectacularly simple to use, which got the masses interested in trying it out. And it seems to be working.
4. GPT-4 will focus on reducing the cost of servicing for OpenAI rather than increasing the number of model parameters
OpenAI has a lot of momentum. Lot of users are coming to DALL-E and ChatGPT. But it's costing them a lot of money to service these users. Up until now, OpenAI focused on creating a great user experience without regards to the cost (thanks to Microsoft for bankrolling their effort with essentially a blank check). Now OpenAI is at a point where they need to service these users and keep the costs under control. Or else they'll go bankrupt.
How will they reduce the cost? By making inference more effective. This means faster algorithms, hardware optimization, and infrastructure optimization. The public won't directly see too many of these changes, but it's groundbreaking research nonetheless to enable AI for the masses.
5. Data volume as a moat will end
There was a time when the volume of data you have served as a real moat. Foundation models changed that. They are trained on huge quantities of data. And they are open sourced. What will the new moat look like? The data generated due to product usage. This is proprietary data that nobody else has e.g. all the user actions on TikTok. You can use this data to continually refine the foundation model and make it your own. That will become the new moat.
6. Foundation model for the industrial world will emerge
Text and image data have received a lot of attention from the AI world. But they haven't unlocked the potential for industrial applications. That's because the data is not as voluminous, not as standardized, and the use cases are not as replicable. That can all change with a foundation model that helps people build basic use cases.
7. AI hardware supply chain will become a geopolitical flashpoint
AI needs specialized hardware to run. The companies that control this supply chain are poised to dominate big time. And nation-states are very aware of this. If your national security depends on procuring this hardware, then you would want to make sure every single company in that supply chain resides within friendly geographies. For parts of the supply chain that are under risk, the nation-states will provide strong monetary incentives to set up their operations in friendly geographies.
8. On-device AI will take off big time as privacy laws get stricter
Data laws are not yet uniform around the world, so AI applications need to thread the needle carefully. Moving the data from the device to the cloud is becoming risky. But people need access to AI applications. AI models are bulky, so they can't run at the same level on edge devices e.g. mobile phones. So on-device AI research will go further to make sure the applications can run on the devices without having to move the data to the cloud.
9. Grand unified model architecture for cloud and on-device applications
Historically the AI models had to come in two versions -- one for the cloud and one for the on-device one. The cloud model is bigger and stronger. The on-device model is smaller and more efficient. But the biggest advantage of doing it on-device is you'll save a fortune in data bandwidth costs. Not to mention speed, privacy laws, and better user experience. AI research will continue in this direction to make sure we have an architecture that can be deployed universally.
10. Demand for compute power will skyrocket
AI work comprises of two things -- training and inference. Training refers to using available data to build a model and inference refers to using that model to provide outputs e.g. predictions. This AI work need a lot of compute power. The compute power available on the market won't be able to meet the demand. People will have to find new ways to generate compute power.
11. A "model streaming" service will emerge
Infrastructure is getting standardized. This will allow a platform of AI models to emerge just like Spotify. This platform will provide infrastructure for AI builders to build models. They will make these models available for consumption via the platform's API. Customers access all these models through a single interface like how we listen to songs on Spotify. And then the revenue will be split based on relative usage e.g. similar to Spotify's revenue model. This platform won't be a two-sided marketplace connecting buyers and sellers.
12. The volume of AI-created video on social platforms will explode
One-way consumption is different than two-way interaction. If I chat with a robot back-and-forth, it may not be the same as talking to a person. My brain will treat this interaction differently. But content consumption on social platforms like TikTok or Instagram is different. People care about being entertained. Does it matter if a video is human-created or AI-created?
A content creator can figure out what's the best prompt to give to the AI system and generate entertaining videos. It's the quality of the end product that matters to the consumer. There's a whole class of creatives who can think of great stories/concepts/ideas, but can't convert that to great videos. Why? Because they don't have the budget, technical knowhow, or post-production bandwidth. But now AI will empower them to participate in this economy.
13. AI-enabled social engineering will rise
Hackers will create deep-fake videos using AI to target people. This allows to be better at social engineering. They use these fake videos to get people to perform specific actions or divulge confidential information.
14. "Self-driving" DevOps will rise
DevOps refers to the field that combines software development and IT operations. It basically refers to managing software infrastructure. Developers use DevOps tools to monitor servers, monitor applications, manage the deployment process, QA, take specific actions to debug, and more. Many tasks in DevOps can/will be automated using AI.
15. Distributed model training will start taking shape
There are only 4-5 companies in the world that have the resources to train a large AI model. To mount a challenge, large number of individual computers can come together to form a cluster. This cluster can pool computing resources together and train a large AI model. What's the incentive for an individual to participate?
One way to do it would be to use blockchain to create crypto assets. Earning a token can be an incentive. If you don't want to use crypto, you can use another incentive mechanism like "karma points" to help people increase their social standing. Or they can rent out this compute cluster and distribute revenue accordingly to the participants. The company that can figure out a real incentive for people to contribute compute power to this cluster will win in this segment.
If you are getting value from this newsletter, consider subscribing for free and sharing with your friends: