What Are Generative AI Models?

In partnership with

Securing top talent with our guide is a click away

Deel’s simplified a whole planet’s worth of information. It’s time you got your hands on our international compliance handbook where you’ll learn about:

• Attracting global talent

• Labor laws to consider when hiring

• Processing international payroll on time

• Staying compliant with employment & tax laws abroad

With 150+ countries right at your fingertips, growing your team with Deel is easier than ever.

Get the Guide

In recent years, large language models (LLMs) like ChatGPT have gained widespread attention. From writing poetry to helping plan vacations, these models represent a transformative leap in AI capabilities and their potential to create value across industries.

Foundation Models

Large language models belong to a broader category known as foundation models, a term first introduced by a team at Stanford. They noted a paradigm shift in AI, moving away from task-specific models to more universal frameworks.

Previously, AI systems were built by training separate models for specific tasks using unique datasets. Now, foundation models serve as a general-purpose capability, fine-tuned to handle a wide range of applications.

These models are trained on massive amounts of unstructured data in an unsupervised manner, allowing them to perform various tasks by leveraging their foundational knowledge. For example, they can generate text, analyze sentiment, or classify data with minimal additional training.

Generative AI

The ability of foundation models to predict and generate new outputs—such as completing a sentence—is what categorizes them as generative AI. Although initially designed for generation tasks, these models can be fine-tuned with small amounts of labeled data to perform tasks like sentiment analysis, classification, or named-entity recognition.

Even without labeled data, techniques like prompt engineering allow these models to perform specific tasks. For instance, by prompting a model with a question like "Does this sentence have a positive or negative sentiment?" it can generate an appropriate response based on its foundational training.

Advantages of Foundation Models

1. Performance:
   Foundation models, trained on terabytes of data, significantly outperform task-specific models, particularly in tasks requiring adaptability or when applied to small datasets.

2. Productivity Gains:
   These models require far less labeled data for fine-tuning, leveraging their pre-trained knowledge to deliver results quickly and efficiently.

Disadvantages of Foundation Models

1. High Compute Costs:
   Training and running these models require substantial computational resources, making them expensive for smaller organizations to adopt.

2. Trustworthiness Concerns:
   Since these models are trained on vast amounts of unfiltered data, they may inadvertently learn biases, toxic content, or misinformation. Additionally, the exact datasets used in training are often unknown, raising reliability concerns.

Applications Beyond Language

Foundation models are not limited to text-based tasks and are being applied across multiple domains:

• Vision: Models like DALL-E 2 generate images from text descriptions.

• Code: Tools like Copilot assist in coding by completing scripts.

• Chemistry: Molecule discovery and drug development are being accelerated through foundation models in chemistry.

• Climate Research: Earth Science models use geospatial data to aid climate research and disaster management.