👋🏼 Hey there! Welcome to another roundup edition of Offload, a newsletter for professionals to learn how to build products and automate work with AI.

I think I'm writing too much 🤓 (please let me know in the feedback section). For this week’s edition, I had to reduce the number of sections because the main topic ended up long, but necessary.

Here is what you'll find in the edition:

  • Understanding the AI landscape

  • The best tool out there to create images with super reliable text generation

  • What is context window

  • Create a billboard for you next ad

-Offload This-

Understanding the AI Stack Landscape

No matter your type of work or area of expertise, you must be seeing every day now a new company or technology being launched and people promising that it will kill industry/company X or that it will change it all now.

Below, today's most recent “death certificate” found in Linkedin:

Donald's prediction

Keep calm. Donald may be right, but let us raise our heads and try to make sense of things.

Let's take a look at the landscape.

-

When trying to make sense of any industry, its players and dynamics, it's common practice (and also really helpful) to break it down into layers, or what we call "the stack."

Think of it like a cake: each layer builds on the one below it, and together they create the complete picture.

To do that, we must first decide what type of companies should be part of this landscape.

AI: The New Electricity

It's important to understand that AI is a general-purpose technology (GPT, ironically), like electricity, the internet, or cloud computing.

These technologies don't just create new industries; they transform every industry.

Think about it: we don't say "a company uses electricity to power their lights" anymore because electricity is everywhere.

Similarly, in a short time, saying "a company uses AI to improve customer service" will sound just as obvious.

AI will be woven into the fabric of how business gets done.

With that in mind, what type of companies should be included in our landscape here?

Defining the Landscape

When we talk about the "AI landscape," we're focusing on companies that are delivering AI capabilities, not just using AI tools.

These are the businesses building the infrastructure, creating the models, and developing the applications that make AI possible for everyone else.

Well, there should be only a couple of companies, right? Not quite…

Some people think the AI industry is brand new, but the truth is that many players have been pushing this technology forward for decades.

Take a look at First Mark's Machine Learning and AI landscape below:

what a view

What's different now is that we've hit a macro cycle moment, a breakthrough that suddenly makes a technology mainstream and marks the beginning of a new era.

Just as Microsoft Windows launched the personal computer era in the 1980s, the release of ChatGPT in late 2022 officially marked the beginning of the AI era.

Suddenly, millions of people were using AI for the first time, and businesses everywhere started paying attention.

Looking Beyond the Headlines

When a new technology goes mainstream, we tend to focus on the companies making headlines, like the OpenAIs, Googles, and Microsofts.

But here's the thing: there are hundreds of other companies involved in making AI work.

From the chip manufacturers to the data processors to the specialized application builders, the AI value chain is vast and interconnected.

With that in mind, here's one way to think about the AI stack (keep in mind, this is just one perspective, there are many valid ways to slice this cake):

Let's go over each layer now, with a brief overview of what the players do in each of them and a couple of examples.

1. Infrastructure Layer

This is the bedrock that everything else depends on, and can be divided into at least the following groups:

Hardware

Companies in this group build the specialized processors that provide the massive parallel computing power AI models need for training and inference.

Unlike traditional CPUs designed for sequential processing, AI workloads require chips optimized for matrix operations and floating-point calculations.

NVIDIA is probably the name you know from this group, providing the raw computational power for training large language models. Recently, NVIDIA became the most valuable company in the world, with a market capitalization of over US$4 tri (yes, trillion).

Other players include AMD and Intel, and now Google and Microsoft are also in this space.

Cloud & Storage

These are the players that provide the data centers and cloud platforms that rent access to these expensive chips and provide the infrastructure backbone.

AWS offers the most comprehensive AI infrastructure, and Microsoft Azure provides GPU-accelerated virtual machines and collaborates closely with NVIDIA for AI infrastructure.

Middleware

The software tools that manage and orchestrate AI workloads across multiple machines. This group is mostly unknown to most non-tech professionals.

Kubernetes handles container orchestration, while Modal, Together AI, and Fireworks provide more specialized AI infrastructure management.

2. Data Layer

AI is only as good as the data it learns from. This layer encompasses the entire data pipeline from collection to preparation for model training.

Ingestion & Sourcing

Companies that collect data from various sources and bring it into AI systems.

This includes web scraping platforms like Bright Data, real-time streaming tools like Confluent and Apache Kafka, and data pipeline services like Fivetran. Companies collect data from APIs, IoT sensors, customer interactions, and proprietary databases, often requiring sophisticated infrastructure to handle the volume and variety.

Storage & Management

Infrastructure for storing and organizing vast amounts of structured and unstructured data that AI models consume.

Modern AI requires flexible storage solutions that can handle everything from traditional databases to massive collections of images, videos, and text.

Snowflake and Databricks lead in cloud data platforms, while object storage services provide scalable repositories for training datasets and model artifacts.

Preprocessing & Transformation

The critical step of cleaning, normalizing, and preparing raw data for AI consumption. Raw data is rarely ready for AI training; it needs to be cleaned, transformed, and structured appropriately.

This step often determines the quality of the final AI model.

Apache Spark processes large-scale data transformations. Pandas and NumPy (Python libraries) handle data manipulation and analysis.

Labeling & Annotation

The process of adding human judgments to data so AI models can learn from examples, crucial for supervised learning

Scale AI pioneered human-powered data labeling and provides reinforcement learning from human feedback (RLHF) services to train models like ChatGPT. Labelbox offers comprehensive annotation platforms, while tools like Snorkel enable programmatic labeling approaches.

Recently, both Scale AI and Surge AI were featured in the news for their success (from different perspectives).

3. Model Layer

This layer encompasses the entire machine learning lifecycle from experimentation to production deployment. Modern ML operations treat this as one integrated process rather than separate phases.

Frameworks & Libraries

The foundational tools and programming interfaces that developers use to build AI models.

TensorFlow and PyTorch dominate deep learning, with PyTorch favored for research and TensorFlow for production.

Model Training & Development

Companies and platforms that create the actual AI models, from foundation models to specialized applications.

Perhaps, this layer is the most pop one.

OpenAI created GPT-4 and ChatGPT, setting the standard for large language models. Anthropic develops Claude, focusing on AI safety and helpfulness. Google DeepMind creates Gemini models and leads research in areas like protein folding with AlphaFold. Meta develops and open-sources the Llama family of models. Mistral and Cohere provide alternative foundation models for enterprises.

MLOps, Deployment & Management

Tools that handle the complete operational lifecycle of machine learning, from deployment to monitoring and management.

This group is huge in terms of options/players/initiatives. A couple of important names to know are below.

MLflow has become the standard for managing model lifecycles and deployment pipelines. Weights & Biases provides comprehensive experiment tracking and model monitoring. TensorFlow Serving offers high-performance model serving for production environments. Hugging Face makes it simple to deploy NLP models.

4. Application Layer

This is where AI capabilities become tangible value for end users. Applications take the raw power of AI models and package them into intuitive, problem-solving tools.

Here, things can get a little mixed up because many “traditional” software players are making the move towards being AI first.

For our landscape exercise here, we consider only players that can provide the AI experience should be considered here.

Analytics

Platforms that help organizations extract insights from data using AI-powered analysis and visualization.

Tableau and Power BI have integrated AI capabilities for automated insights and natural language querying. Palantir specializes in complex data integration and analysis for enterprises and government.

Enterprise Applications

AI capabilities embedded directly into business software that companies already use daily across multiple departments.

Microsoft 365 Copilot integrates generative AI into Office applications, while Salesforce Einstein provides AI-powered insights within CRM workflows. Slack AI and Notion AI assist with communication and knowledge management in collaborative workspaces.

Horizontal Applications

AI tools that solve common problems across multiple industries and use cases.

Grammarly provides writing assistance, Canva offers AI-powered design tools, and n8n and Zapier enable workflow automation. Lindy provides AI assistants for business processes, while image generation tools like Midjourney and DALL-E create visual content from text descriptions.

Vertical Applications

AI solutions built specifically for particular industries or functions, incorporating deep domain knowledge and specialized workflows.

Harvey AI provides legal research and document analysis for law firms. Cursor and Windsurf are AI-native code editors for software development. Tempus applies AI to cancer diagnosis in healthcare, while Veeva offers solutions designed specifically for pharmaceutical companies.

Services

Consulting, implementation, and support services that help organizations successfully deploy and scale AI initiatives.

Traditional consulting firms like Deloitte, McKinsey, and Accenture provide AI strategy consulting, while specialized firms like Slalom focus on hands-on implementation.

These services are crucial because implementing AI successfully requires not just technology but also change management, training, and strategic planning.

Phew, you've passed by them all 🥳

Finding Your Place in the Stack

Understanding where you want to play in this stack and how the layers connect is crucial for anyone building and living in the AI space.

The era has just begun, and there's room for innovation at every level.

In the upcoming insight editions, we continue looking at the landscape but reflecting on where the money is flowing and parties interested in that flow.

-Tool of the Week-

🛠️ Tool of the Week: Ideogram.ai

What it does: an AI image generator that creates high-quality visuals from text prompts. It stands out for its ability to generate images with accurate written text embedded in the visuals, making it ideal for posters, ads, and branding content.

Why it’s worth checking out:

Text rendering is sharp and reliable, unlike many other image generators
Easy-to-use interface with fast, visually impressive results

A few drawbacks:

⚠️ Limited customization of typography (font style, size, placement)
⚠️ Image resolution and export options are basic compared to professional design tools.

My take:
I really liked the reliability of the text generation, something other tools always fell short for me.

For now, I find this tool more interesting than Midjourney for marketing and content purposes. So, I'm going to use it more from now on.

-Learn AI-

What is Context Window?

The idea of a context window is relatively simpler to understand compared to other things covered here, like vector databases and RAG, but it is still super important to the world of AI.

Here we go:

Imagine you’re talking with a colleague, and you're keeping track of the last few things they said to keep the conversation flowing. That mental “buffer” helps you reply thoughtfully and maybe even bring up something mentioned earlier.

For an AI language model, the context window is like that buffer. It’s the chunk of text or data the model can hold onto and use to come up with a response.

Think of it as the model’s short-term memory.

When you toss a question or prompt its way, it doesn’t just zero in on the last sentence. It looks at everything in its context window, like your prompt, past exchanges, and sometimes even some background info it’s trained to use.

The size of this window decides how much the model can juggle at once, and it’s a huge part of how well it gets you.

Here is an interactive app I created to exemplify how the context window works. Drag the sliders to determine the context window size and see how it impacts the AI output/summary (I'm using Artifacts from Claude to create this app, in case you are wondering).

How does it work?

The context window is measured in tokens, which are the building blocks of language for LLMs. In a previous edition, we covered what tokens are, but to recap: a token can be a word, part of a word, or even punctuation, basically the little bits that make up text.

For example, “I love AI” might break into about four tokens: “I”, “love”, “AI”, and a space or punctuation.

The context window is the max number of tokens the model can handle at once. This covers both the input (your prompt or question) and the output (the model’s response).

So, if a model’s got a context window of 8,000 tokens, it can deal with up to 8,000 tokens of input and output combined before it starts “forgetting” the earlier stuff.

Here’s an easy way to think about it: imagine the context window as a whiteboard. The model can jot down a bunch of notes, but once it’s full, it’s gotta erase the oldest stuff to make room for new info. The bigger the whiteboard, the more the model can keep track of without losing the plot.

Why does it matter?

The size of the context window is a big deal for a few reasons:

  1. Longer chats: A bigger context window lets the model keep up with long conversations without dropping the ball. If you ask it to summarize a 10-page document or keep a multi-turn chat going, a bigger window helps it hold onto more details.

  2. Better understanding: More context means a better grasp of what you’re saying. If you give the model a complex prompt with tons of background, a larger window lets it take it all in, leading to sharper, more relevant responses.

  3. Creative freedom: Want the model to write a 1,000-word story or dig into a huge dataset? A larger context window gives it the space to work with all the details you throw in, plus room to whip up a solid response.

  4. Fewer “wtf?” moments: Ever had a chatbot totally miss the mark because it forgot the first half of your question? A bigger context window cuts down on those moments by letting the model keep more of the conversation in focus.

Bigger isn’t always better

You might be thinking, “why not give the model an infinite context window?”

Well, a larger context window needs more computing power, which means more energy, more time, and sometimes more cost.

It’s like trying to read a 1,000-page book in one go. Sure, it’s doable, but it takes a lot of effort, or in this case, processing power.

Plus, not every task needs a giant context window. If you’re just asking, “What’s the weather today?” the model doesn’t need to hold onto a novel’s worth of context to give you a good answer.

Smaller windows can be quicker and more efficient for short, simple questions.

How do context windows vary?

Different LLMs have different context window sizes, kinda like comparing phone storage. Bigger is often cooler, but it depends on the model’s setup. For example:

  • Older models, like early GPT versions, had context windows of around 2,000 to 4,000 tokens, which worked for short chats but could feel cramped for big tasks.

  • Modern models often rock much larger windows, some handling 128,000 tokens or more! That’s enough to process whole books or super long chats.

  • Some specialized models might have smaller windows built for specific jobs, like quick Q&A or real-time translation.

Here is a summary table of the state of the context window per model:

What does this mean for you

It shapes how you interact with models.

If you are using LLM chats to complete small tasks occasionally, using a free model which will have a small context window is ok. It will suffice for you.

But, since we are builders here, that's not going to be enough. Choosing models according to the purpose of the build is something that can lead to success or failure.

From previous editions:

-Fun with AI-

Creating billboards with text for your next ad

Ideogram.ai is great for images with text. I've made this one below to remind me of the important things in life. You must do it too.

To create your billboard, go to Ideogram.ai

  1. Copy and paste this prompt below, and adapt the text in brackets

A striking nighttime lifestyle advertisement featuring a bustling cityscape illuminated by a colossal billboard showcasing a steaming cup of coffee. The billboard, prominently positioned above a wet, reflective sidewalk, displays the text ["Remember to buy coffee. Wife gets mad."] in a bold, slightly distressed font, while the warm glow from the coffee cup contrasts with the cool blues of the urban skyline. Soft, ambient light washes over the scene, highlighting the billboard’s message and subtly suggesting the convenience and necessity of coffee, with the faint outline of a recognizable coffee chain logo subtly integrated into the bottom corner. The overall mood is playful and relatable, using humor to connect with a stressed-out audience and reinforce the brand's position as the go-to solution for a quick caffeine fix.

Billboards are just the tip of the iceberg. You can go WAY beyond that.

A cool thing about ideogram.ai is that they offer a “magic prompt” option … that is, they know people suck at prompting and auto-adjust the prompt to a better version.

Here is another example I did, based on the amazing style of Eduardo Kobra:

A photograph of a towering building facade in São Paulo, Brazil, showcasing a vibrant geometric mural in the style of Eduardo Kobra. The mural bursts with bold colors and abstract shapes, seamlessly integrating the hand-painted message ["I like my coffee to taste like my life: bitter"] in expressive, dynamic lettering that curves and flows with the design. The building stands against a clear, bright blue sky, framed by the silhouettes of neighboring urban structures, and textured concrete details add depth. Sunlight illuminates the mural, highlighting the intricate patterns and casting subtle shadows across the wall.

The result:

-Any feedback-

Before you go, I’d love to know what you thought of today's newsletter so I can improve it for you. 🙃

Keep Reading

No posts found