#058 Midweek Special - China’s AI Race Heats Up: Alibaba and Tencent Invest $342M in Zhipu, Meet DataGPT, The AI Analyst That Lets You Talk To Your Data, All About LLM Economics.
Fresh & Hot curated AI happenings in one snack. Never miss a byte 🍔
This snack byte will take approx 6 minutes to consume.
AI BYTE # 1 📢 : Meet DataGPT, The AI Analyst That Lets You Talk To Your Data
⭐ Data analysis is a crucial skill for any business, but it can also be tedious, time-consuming and complex.
Traditional Business Intelligence (BI) tools often require manual work, custom dashboards and technical expertise to get insights from data.
But what if there was a better way to access and understand your data?
Enter DataGPT, a California-based startup that came out of stealth today with the launch of its new AI Analyst, a conversational chatbot that helps teams understand the what and why of their datasets by communicating in natural language.
DataGPT combines the creative, comprehension-rich side of a self-hosted large language model (LLM) with the logic and reasoning of its proprietary analytics engine, executing millions of queries and calculations to determine the most relevant and impactful insights.
This includes almost everything, right from how something is impacting the business revenue to why that thing happened in the first place.
DataGPT targets the static nature of traditional BI tools, where one has to manually dive into custom dashboards to get answers to evolving business questions.
With DataGPT, you can simply ask any question in natural language and get instant, analyst-grade results. You can also explore your data visually using the Data Navigator, a more traditional version where you get visualizations showing the performance of key metrics and can manually drill down through any combination of factors.
DataGPT claims to deliver a new data experience that is faster, cheaper and more powerful than existing solutions. Its lightning cache database is 90 times faster than traditional databases.
It can run queries 600 times faster than standard BI tools while reducing the analysis cost by 15 times.
DataGPT has already attracted several enterprise customers, including Mino Games, Plex, Product Hunt, Dimensionals and Wombo. These companies have been able to use the chatbot to accelerate their time to insights and ultimately make critical business decisions more quickly.
DataGPT plans to open source its database in the near future and expand its analytical capabilities to cover more ground, such as cohort analysis, forecasting and predictive analysis.
However, it will be interesting to see how DataGPT stands out in the market.
Over the past year, a number of data ecosystem players, including data platform vendors and BI companies, have made their Gen AI play to make consumption of insights easier for users.
DataGPT believes that it has an edge over its competitors by combining the left-brained tasks of deep data analysis and interpretation with the right-brained tasks of contextual comprehension and humanization.
AI BYTE # 2 📢 : China’s AI Race Heats Up: Alibaba and Tencent Invest $342M in Zhipu.
⭐ Chinese tech giants Alibaba and Tencent have invested a staggering $342 million in the AI startup Zhipu, which focuses on Gen AI.
This strategic investment, part of a broader trend of capital influx into the AI sector, signifies China’s determination to assert its dominance in AI innovation.
Zhipu is not the only AI startup that has attracted substantial investment from major players. Another AI startup, Baichuan, which is positioning itself as a rival to Zhipu in the AI race, received $300 million from a similar group of investors.
Both Zhipu and Baichuan aim to compete with established players like OpenAI and Google in the development of advanced AI models.
Zhipu has made significant strides by securing government approval for a public rollout in August. Subsequently, it released an open-source AI model and introduced a chatbot named Qingyan. These developments demonstrate the company’s commitment to innovation and its ambition to compete globally.
The US-China AI competition has far-reaching implications for both countries and the world. AI is expected to transform industries across the board, potentially ushering in new economic growth.
However, the technology also has government and military applications, adding complexity to the already tense Washington-Beijing relationship.
The United States recently tightened restrictions on Chinese access to advanced chips essential for training and running AI models. This move challenges Chinese AI developers who may need to explore homegrown alternatives.
Washington has also been expanding its blacklist of restricted firms to include AI chip design companies, further complicating the landscape.
The significant investment by Alibaba and Tencent in Zhipu underscores the escalating competition in the Chinese AI sector. With the backing of major players, Zhipu and its peers are determined to challenge global AI giants like OpenAI and Google.
The stakes are high, as AI innovation is poised to transform industries and economies worldwide. As China and the United States vie for supremacy in AI, the implications extend beyond technology, impacting geopolitics and global economic dynamics.
This race is a testament to AI’s critical role in shaping our world’s future.
What are your thoughts on this topic? Do you think China can catch up with or surpass the United States in AI?
AI BYTE # 3 📢 - LLM Economics: A Guide to Compare the Pricing and Performance of Different Language Models.
⭐ Language models (LLMs) are transforming the way we interact with data and information. They can generate natural language texts, answer questions, summarize documents, and much more. However, not all LLMs are created equal.
There are many factors that affect their cost and performance, such as size, context length, retrieval methods, and fine-tuning.
In this post, I will share some insights from a recent report by AIM Research, titled “LLM Economics - A Guide to Generative AI Implementation Cost”.
The report provides a detailed analysis and cost estimates for using different LLMs in businesses. It also features a handy LLM Calculator on MachineHack that allows you to compare the pricing and performance of various LLMs.
One of the main findings of the report is that using a customized version of OpenAI’s language model is ten times more expensive than using the standard one.
OpenAI’s models are highly rated, but they also come with a hefty price tag. For example, it cost OpenAI over $100 million to train GPT-4, which has 175 billion parameters and can handle up to 32K tokens of context length.
The cost for using the GPT-4 API is $0.03 per 1,000 tokens for input and $0.06 per 1,000 tokens for output for the 8K context model, and $0.06 per 1,000 tokens for input and $0.12 per 1,000 tokens for output for the 32K context model.
To illustrate the cost difference, let’s take a use case of summarizing Wikipedia to half its size. Wikipedia has 6 million articles, and each article is around 750 words. That equals to 1000 tokens because three fourths of a word is equal to one token, so in total it translates to 6 billion tokens. When we reduce the size of Wikipedia by half, we will be left with 3 billion tokens as output.
The cost variations for summarization among different models reveal significant differences in pricing structures.
GPT-4 with 32K context length demands a higher investment at $720,000, while its 8K counterpart costs $360,000.
GPT-3.5 Turbo is a more budget-friendly option at $15,000, and Llama 2 offers competitive pricing at $4,000.
Another factor that affects the cost and performance of LLMs is the retrieval method.
Retrieval Augmented Generation (RAG) is a technique that combines retrieval models, which can quickly identify relevant passages from a large corpus of data, with generative models for more context-aware responses. This can be beneficial in scenarios where quick access to relevant information is crucial.
The indexing cost for OpenAI Embeddings (AdaV2) + OpenAI GPT 3.5 Turbo is $600. While the daily cost for 1 million queries is $3400.
On the other hand, the indexing cost for e5-large-v2 Embedder + Falcon-40B is $83 and the daily cost for 1 million queries is $1415.
The fine-tuning cost is another aspect that needs to be considered when choosing an LLM. Fine-tuning is the process of adapting a pre-trained model to a specific domain or task by adjusting its parameters. This can improve the accuracy and relevance of the model’s outputs.
The fine-tuning cost for OpenAI Curie 13B is $7,000 and daily cost for 1 million queries is $26,400. On the other hand, the fine-tuning cost for LLaMa-v2-7b is $1365 and inference cost for 1 million queries is $470.
As you can see from these examples, there is no one-size-fits-all solution when it comes to selecting an LLM. It depends on your use cases and budget constraints.
However, it is important to understand the factors that play a role in deciding the pricing and performance of different LLMs.
This is how people ignored each other before Smartphones.