OpenAI is an artificial intelligence research body that aims to develop and promote user-friendly AI beneficial to humanity. Launched in 2015, OpenAI is backed by Elon Musk, Peter Thiel, Microsoft, Infosys, and other investors, thus raising over $1 billion in funding.
OpenAI has released machine learning and AI products, including:
OpenAI has three major NLP (natural language processing) model releases. They are GPT, GPT-2, and GPT-3. They also have other models apart from those three.
GPT or GPT-1 was the first NLP model released by OpenAI. It was launched in 2018 and trained using a large BooksCorpus dataset. At the release time, GPT performed better than most other trained, supervised models. It could answer questions well and perform sentiment analysis with good zero-shot learning performance.
Announced in 2019, GPT-2 is the direct successor to GPT. GPT-2 contained over ten times the parameters of the original GPT, and it was trained with over ten times the amount of data used for GPT.
GPT-2 was trained using a dataset of 8 million web pages, containing about 1.5 billion parameters. GPT-2 is adept at generating AI text of superb quality, outperforming other NLP models trained on domain-specific datasets.
GPT-2 performed well on language tasks like reading, comprehension, answering questions, summarization, and translation. Needing only a simple human prompt, GPT-2 could generate believable and coherent bodies of text based on human input. It does all these without task-specific training.
But GPT-2 had shortcomings like repetitive sentences in the generated text and poorly generated content on niche topics.
For safety and security reasons, a smaller version of GPT-2 was released to the public. A miniature version of GPT-2 output datasets was also released for researchers.
Announced in 2020, GPT-3 is the latest version of GPT. GPT-3 was trained on over 175 billion parameters compared to 1.5 billion parameters for GPT-2.
In December 2022, OpenAI debuted GPT-3.5 along with ChatGPT. GPT-3.5 is an improved version of GPT-3, but it isn’t a full-fledged independent release. GPT-4 is still expected in the nearest future.
Whisper is an automatic speech-recognition model from OpenAI. Whisper was trained using over 680,000 hours of multilingual data. The NLP model can recognize multilingual speech and automatically identify the correct language. It can also translate speech from multiple languages.
Whisper is adept at recognizing speech despite accents and background noise. It was trained using a diverse dataset, with a third of the dataset comprising of non-English speech.
Whisper is already being used in applications for faster, direct YouTube search that jumps straight to a relevant video segment using speech recognition as opposed to just searching using the video name and description.
In early 2022, OpenAI released InstructGPT. InstructGPT models are better at following human instructions than GPT-3. GPT-3 and other NLP language models require prompt engineering to generate outputs needed by users. But InstructGPT can follow instructions more accurately without generating untruthful or harmful outputs.
Built on GPT-3.5, ChatGPT is an AI chatbot that generates responses to human prompts. It can write essays, write code, solve mathematical problems, write business plans, translate text and conduct a whole lot of tasks.
ChatGPT is currently OpenAI’s most popular AI product. ChatGPT brought in over 1 million users within just five days. To put this into context, it took GPT-3 two years to gain 1 million users. The estimated number of ChatGPT users is 100 Million
A significant limitation of ChatGPT is that it was trained using a 2021 dataset. So, the bot is limited in giving answers to queries about events that happened after 2021.
GPT-3 provides a significant performance boost over GPT-2. The model provides vastly improved performance from GPT-2, especially for generating text for niche topics. It can also generate computer code. GPT-3’s ability to generate computer code has been used for ChatGPT, Codex, and other AI applications.
Then there’s GPT-3.5, a more refined version of GPT-3. GPT 3.5 is good at writing witty, humane text while adapting to the tone and mannerisms of any popular human.
You could tell it to respond like a child, a popular politician, or a musician. It could write poems, write code, and solve mathematical problems. It can write essays, compose emails, take tests, manipulate data, play games, and explain complex things. It can even write a complete business plan with relevant case studies.
A slight criticism of GPT-3.5 is its verbosity compared to earlier models. It tends to generate unnecessarily lengthy responses to prompts. While this might be unnecessary for the average user, it is helpful for writers and those who need AI help for essays and blog posts.
Many users are now building apps based on ChatGPT/GPT3.5. For example, Addy is an AI email assistant that can write emails 10x faster according to your preferred tone and style.
OpenAI API is based on GPT-3. The API grants developer access to OpenAI models like GPT-3, DALL-E, and Codex. Developers can use the API to build intuitive applications.
Codex is a top-level AI code generator. Based on GPT-3, Codex was trained using natural language and several billion lines of code.
Codex can turn simple human input into code, complete lines of code, rewrite codes, and add comments to written code. You can even tell Codex to find useful APIs and libraries for you.
Thanks to its large library of coding languages, you can use Codex to build a wide variety of no-code apps. Codex can translate simple text in English to Python, SQL, JavaScript, HTML, Swift, Typescript, C#, Perl, Shell, Go, etc.
Codex is flexible with syntax and style. It works even better when you specify the language version you want. Codex can provide helpful suggestions when importing libraries and APIs. But you need to be careful with this because sometimes the suggestions might not work optimally for whatever you are building.
Long completion requests in Codex can lead to errors and repetition. Be as specific as possible when using Codex, and you will get impressive results. You can also set stop tokens to limit query sizes. You can use the comment function to explain difficult codes by starting an explanation comment and letting Codex complete the explanation for you.
Nothing concrete has been announced about GPT-4. But industry rumors predict a potential release date of 2023. GPT-3 had 175 billion parameters. If GPT-4 is 10x larger as expected, it will have over 1 trillion parameters. And it will be trained on significantly larger data than GPT-3. A fine-tuned version of ChatGPT might also be released along with GPT-4.
That said, back in 2021, OpenAI disbanded its robotics team. So, don’t expect to see OpenAI’s neural network and NLP models in physical robots so soon. But things can change, especially if OpenAI partners with a robotic company. OpenAI has achieved a lot within seven years. The future is ripe with possibilities for the organization.
What are GPT models?
GPT or Generative Pre-trained Transformer models are AI Neuro-linguistic programming model that uses deep learning to interpret and produce human text. These GPT models were trained using large text datasets.
Is GPT-3 an NLP?
Yes, GPT-3 is an NLP (Natural Language Processing) model.
What model does GPT-3 use?
GPT-3 uses its own language model trained using hundreds of billions of words.
Is GPT-3 based on BERT?
No, GPT-3 isn’t based on BERT. Neither are GPT-2 and GPT. BERT, or Bidirectional Encoder Representations from Transformers, is an NLP model from Google.
Is GPT better than BERT?
While GPT is more popular, with more use cases than BERT, BERT does have the unique advantage of being bidirectional, which means that it can read text both from left-to-right and right-to-left for better language context. BERT is also open-source, unlike GPT.
What is the difference between BERT and GPT?
BERT is an encoder-only model, while GPT is a decoder-only model that uses transformer encoder blocks. BERT was created by Google, while OpenAI created GPT. GPT was trained using a larger dataset than BERT. BERT is bidirectional, while GPT is not. BERT is open source, while GPT is not.
What dataset is GPT-3 trained on?
GPT-3 is trained on a wide variety of data, including Wikipedia, Common Crawl, books, and web texts. It was trained on hundreds of billions of words.
