On November 30th, 2022—more than two years after GPT-3’s release—OpenAI launched ChatGPT, an online chatbot powered by GPT-3. As a result, ChatGPT became an immediate hit—literally speaking. Within five days, the service had acquired one million users. (in contrast, it took Instagram 2.5 months, Spotify 5 months, and Twitter 2 years to cross the million-user mark.)
.jpg?width=880&height=568&name=shutterstock_2248134023%20(1).jpg)
In the first half of 2023, the Nasdaq Composite, the most widely-tracked technology-heavy stock index in the United States, advanced by 32.74% or 3,400.93 points. Though that’s the index’s best start to a calendar year in four decades, it’s still 2,269.52 points or another 16.5% gain away from eclipsing its current all-time high, which it set in November 2021.
We’ve talked at length about the tech market’s 2022 woes over the past year, and it’s clear that few corners of the industry were left untouched. Accompanying public tech stocks’ 33.1% decline last year was the IPO market, which froze over in 2022.
Venture capital posted a similarly poor showing, with many growth- and late-stage private startups feeling the brunt of the valuation crunch. Digital assets like crypto and NFTs likewise fared poorly, entering a fourth “crypto winter” in 2022.
Nonetheless, though public and private tech companies alike took a tumble in 2022, tech itself arguably advanced at a nosebleed rate (which, tangentially, is loosely another reason why the stock market is not the economy).
OpenAI: From GPT-1 to Today
For the artificial intelligence (AI) industry in particular, last year was a watershed moment. This was OpenAI—a research laboratory founded in 2015 by roughly a dozen machine learning experts and tech industry heavyweights like Elon Musk and Ilya Sutskever.
Headed by 38-year-old former Loopt co-founder (and later YCombinator President and very briefly Reddit CEO) Sam Altman, the AI startup’s research mainly focuses on reinforcement learning. Commonly abbreviated as RL, this feedback-based machine learning technique trains models through repeated trial and error.
In the eight years since its inception, OpenAI has debuted a number of products beneficial to both AI researchers and the broader public. In 2016, the company—then organized as a nonprofit—released OpenAI Gym, a development toolkit that allowed researchers to train reinforcement learning algorithms.
A year later, the startup followed up with RoboSumo, a simulated environment that enabled one (or multiple) virtual robots to engage in competitive self-play. OpenAI found that this process allowed “simulated AIs to discover physical skills like tackling, ducking, faking, kicking, catching, and diving for the ball, without explicitly designing an environment with these skills in mind.”
In 2018, the company began to focus its research on generative pre-trained transformers (GPTs), a type of large language model (LLM) based on transformer architecture that is pre-trained using databases of unlabeled text.
OpenAI released its first model, GPT-1, on June 11th, 2018, and demonstrated in a 12-page paper that the model could perform “discriminative tasks such as question answering, semantic similarity assessment, entailment determination, and text classification, improving the state of the art [i.e., the top-performing model or method at the time] on 9 of the 12 datasets we study.”
Following GPT-1’s success, the startup introduced further LLMs based on the GPT framework. On February 11th, 2019, the company released GPT-2, a model trained on 7,000 unpublished works of fiction. A scaled-up version of its predecessor, GPT-2, had “over an order of magnitude more parameters than GPT” and could “zero-shot[] to state of the art performance on 7 out of 8 tested language modeling datasets.” In other words, GPT-2 could “zero-shot”—or solve a task that it had never encountered before—better than models that had been explicitly trained on those tasks.
On June 11th, 2020, OpenAI unveiled GPT-3. At 175 billion parameters, the model was over 100 times larger than GPT-2, which has 1.5 billion parameters. It was also significantly larger than any other LLM then available on the market—about 10 times larger than Microsoft’s 17 billion-parameter Turing-NLG and 16 times the size of Google’s 11 billion-parameter T5 model.
Its capabilities were equally impressive. For instance, in a word unscrambling task (e.g., “lpaemxe” would unscramble into “example”), GPT-3 achieved greater than 45% accuracy under one-shot learning conditions (i.e., where the model is given a single prior example before being asked to perform the task) and 60% accuracy when offered ten prior examples.
This stood in stark contrast to models with 13 billion parameters, which achieved about 6% accuracy under one-shot learning conditions and 10% accuracy under learning conditions with 10 examples.
ChatGPT Comes to Life
For the first several months, GPT-3 was available on an invite-only basis. Once given access, users could test the model using OpenAI Playground, a web-based interface.
But Playground was primarily geared toward developers. While it wasn’t impossible for laypeople to figure out how to interact with GPT-3, the interface was far from intuitive.
GPT-3 also costs money to use: first $0.06 per 1,000 tokens (which is equivalent to about 750 words) and later—beginning in the third quarter of 2022—$0.02 per 1,000 tokens. Users who secured an invite were given $18 in free credits—enough for casual experimentation, but little more.
Then, on November 30th, 2022—more than two years after GPT-3’s release—OpenAI launched ChatGPT, an online chatbot powered by GPT-3. Unlike Playground, the service’s dialogue-based design made it intuitive to use. Better yet, it was entirely free to use.
By day 40, ChatGPT had 10 million users, and within two months, its userbase had again increased by tenfold to 100 million. Users marveled at ChatGPT’s many capabilities: it could write code, pen fanfiction, answer questions on technical topics, compose emails, serve as a Socratic tutor, brainstorm taglines…and also, uh, offer breakup advice?
GPT-4 and Labor Force Disruptions
Though ChatGPT is free to use, it’s expensive for OpenAI to operate. And as the chatbot’s userbase grew, the company’s server bills began to pile up.
The Information, a technology and business-focused magazine, reported that ChatGPT costs the company $700,000 a day to operate. Altman, meanwhile, estimates that it costs OpenAI “probably single-digits cents” per ChatGPT query. When multiplied across hundreds of millions of active users and dozens of chats per user per day, that’s a figure that easily exceeds tens of millions of dollars per month.
Suffice to say, OpenAI needed a way to monetize ChatGPT. On March 14th, 2023, the startup released GPT-4, the successor model to GPT-3. While the company has been secretive about the model’s specifications, third-party outlets like Semafor—a site founded by New York Times and Bloomberg News columnists—claim that GPT-4 has 1 trillion parameters, making it about 5 times larger than GPT-3. Altman has, however, shared one detail: the model didn’t come cheap, and cost the company an estimated $100 million to train.
For these reasons, it’s unsurprising that GPT-4 isn’t free to use. Unlike GPT-3, which powers the free version of ChatGPT, the GPT-4 version of the chatbot is only available to users who subscribe to ChatGPT Plus. Launched in February 2023, the $20-per-month service promises “faster response times” and gives subscribers “priority access to new features and improvements”. Presumably, if future versions of the company’s GPT models are released, ChatGPT Plus subscribers will be first in line.
But is there a reason to subscribe? While “less capable than humans in many real-world scenarios,” OpenAI claims that GPT-4 “exhibits human-level performance on various professional and academic benchmarks.”
On March 27th, 2023, the startup followed up with a paper that characterized GPT-4 as a general-purpose technology—those that impact an entire economy and which have uses across multiple industries and sectors. They argue that GPTs have the potential to disrupt the workforce, warning that “up to 49% of workers could have half or more of their tasks exposed to [automation from] LLMs.”
Interestingly, forms of work once thought to be resistant to automation—like law, mathematics, journalism, or software engineering—appear to be most affected by the proliferation of general-purpose LLMs. Page 16 of OpenAI’s working paper suggests that legal clerks, writers, web designers, clinical data managers, news reporters, quantitative financial analysts, climate change data analysts, and 80 other occupations were “fully exposed” to LLM software. This stands in contrast to the traditional narrative, which tends to view unskilled labor as the subset of work most prone to automation.
What Does This Mean for the AI Industry?
Beyond the labor force, ChatGPT is also making waves within the tech industry. Market heavyweights like Google and Apple, formerly self-assured about their dominance and confident in the defensibility of their competitive moats, are now worried that their time as king of the hill could be coming to an end.
“Google has no moat” in AI, confessed Luke Sernau, a senior software engineer at the company. His leaked internal memo claims that the search engine giant isn’t “positioned to win this [AI] arms race,” but surprisingly goes on to admit that “neither is OpenAI.”
Sernau argues instead that a “third faction”—namely open-source AI platforms—will come out ahead of both Google and OpenAI, explaining that open-source models are “faster, more customizable, more private, and pound-for-pound more capable” than closed-source models like GPT-4.
Specifically, open-source models focus on data quality rather than quantity. This makes them smaller and thus more affordable to train and deploy. Sernau gives examples of developers running LLMs on their smartphones, fine-tuning custom LLMs that can be trained “within hours on a single RTX 4090 [GPU]”, and deploying fully-custom LLMs that bypass ChatGPT’s restrictions against dangerous or explicit content.
He contends that nearly “anyone with an idea can generate [an LLM that is]…largely indistinguishable from ChatGPT”, concluding that OpenAI’s hostile “posture relative to open-source” means that “alternatives can and will eventually eclipse them.”
If so, that would be a surprising outcome for a startup so broadly celebrated by the public over the past year—one showing that, if nothing else, the hype and reality surrounding AI are two wildly different things.
At The Spaventa Group, we’re plugged into the latest developments in AI. From the newest LLMs to up-and-coming open-source alternatives, our analysts are deeply familiar with the industry and its happenings.
We’re committed to backing some of the most promising AI companies in the world, and we invite you to partner with us as investors in our curated alternative funds, which offer you diversified exposure to an exclusive selection of top technology startups.
Get in touch with us today to see how we can help you diversify with alternative investments.
