How Technology Training Changed: Five Waves From Hadoop to AI

I have spent years building technical training outlines for enterprise teams: the one-page summaries of what a group of engineers, analysts or managers should be able to do after a course. Recently I went back through them. The archive covers 249 files, which collapse to about 208 distinct outlines once you merge versions and formats, and 195 of those map cleanly to a technology theme.

Reading them in order is a strange experience. It is a record of what companies were afraid of not knowing, year by year. This post walks through five waves, shows what my own outlines say about each, and puts real market data next to them. I have removed every client name. This is about the technology, not the accounts.

How I counted, and what this is not

Before the charts, the limits, because they matter.

• Dates are file dates. The earliest part of my archive was bulk-loaded in late 2019, so "2019-20" really means "2019 and earlier". The Hadoop-era peak (roughly 2012 to 2018) is mostly before my archive starts, so you will only see its tail.
• Counts are outlines, not learners. One outline can be delivered many times. These charts show what I was asked to teach, not how many people were trained.
• Themes are my own classification, based on titles and content keywords. Machine learning and data science are counted under big data and analytics, not under the AI wave. A handful of domain-specific outlines (finance, compliance, payroll) are left out of the theme charts.
• The 2023 to 2025 window is thin (16 outlines), because I wrote fewer in those years. Do not over-read it.

The shape of the whole archive

Three things jump out. In 2019-20, 27% of my new outlines were about big data and analytics and 30% were about cloud and DevOps. By 2021-22, software delivery and digital transformation took over at 53%. Then in 2026, 60% of everything I wrote was about AI, up from none in 2019-20.

Each of those shifts matches a wave the industry went through, with training following the money by a year or two.

Wave 1: Big data and Hadoop (the tool that outlived its hype)

The big data wave was driven by raw volume. IDC's 2018 forecast had the global datasphere growing from 33 zettabytes in 2018 to 175 zettabytes by 2025. Companies bought clusters first and asked about skills second.

The training ask was tool-shaped: Hadoop administration, Spark, NiFi, Greenplum, Elasticsearch, messaging platforms. In 2019 the market signalled the turn. Cloudera and Hortonworks, the two biggest Hadoop distributors, merged in an all-stock deal valued at about $5.2 billion, and the third major vendor, MapR, said in May 2019 it would shut down unless it found a buyer. Analysts tied the decline to managed cloud services and cheap object storage replacing the cluster you had to run yourself.

My own outlines show only the tail. Exactly one is a pure Hadoop course (administration). The rest of the big data stack lived on as modules inside data engineering programs in 2021-22, as Spark, NiFi and Greenplum, and by 2026 none of my outlines leans on any of them. What survived was the layer underneath: SQL and visualization. Tableau and Power BI appear in 6, 8, 4 and 7 outlines across the four periods. Lesson: tool-specific skills have a short half-life. The concepts under them (data modeling, SQL, how to read a chart) do not.

Wave 2: Cloud migration and DevOps (training for the lift, then for the run)

The money followed the migration. Gartner forecast worldwide public cloud end-user spending at $257.5 billion for 2020 and $723.4 billion for 2025, roughly 2.8 times in five years (two different Gartner releases, so treat the exact gap as indicative).

Training in this wave started with fundamentals and certifications (Linux, AWS core, DevOps foundations, CI/CD with Jenkins), then moved to containers. Docker and Kubernetes show up in 12 outlines in 2019-20, still 8 in 2021-22, and come back in 2026 with 6, but the 2026 versions include networking and troubleshooting workshops, not just "introduction to containers". The question changed from "how do we get there" to "how do we keep it running". AWS and Azure keep a steady presence across every period (6, 13, 5, 14). Lesson: cloud is not a wave that ended. It became the floor every other wave stands on.

Wave 3: Digital transformation (the hiring-program years)

IDC forecasts worldwide digital transformation spending of nearly $3.9 trillion by 2027. In practice, for a training provider, this wave looked like large hiring programs: full-stack developers, mobile developers, API developers and automation specialists trained in bulk for a business that needed delivery capacity fast.

It was the biggest slice of my archive in 2021-22 at 53%. It was also the wave with the clearest boom and bust inside it. Robotic process automation (RPA) appears in 1 outline in 2019-20 and 4 in 2021-22, then in none after that. Java and .NET followed a gentler version of the same curve (6, 12, 2, 2). Lesson: a skill category can look permanent at its peak. If you are planning a multi-year training budget around one tool, plan around the job, not the product.

Wave 4: Cloud data platforms (the quiet one)

Between the big data bust and the AI wave, data engineering rebuilt itself on managed platforms. Snowflake and dbt appear in 2, 1, 3 and 5 outlines across the four periods. Databricks goes 0, 0, 2, 5.

The content also matured. The early Snowflake outlines were fundamentals. The 2026 ones include cost optimization and AI features, because by then teams had the platform and had also met the bill. This is the wave that quietly prepared the ground for AI: the data the models need lives here.

Wave 5: The AI wave (and why it is a different kind of training)

Adoption data tells the story faster than anything I can write. In the Stack Overflow Developer Survey, the share of respondents using or planning to use AI tools went from 70% in 2023 to 76% in 2024 and 84% in 2025.

Business adoption followed. McKinsey's survey shows organizations using AI in at least one business function rising from 50% in 2022 to 88% in 2025, with generative AI going from 33% in 2023 to 79% in 2025. McKinsey also reports that only about 7% say AI is fully scaled, which is the gap training is supposed to close.

In my archive, 37 of my 62 outlines from 2026 are about AI. They differ from every earlier wave in four ways.

1. The audience moved. Earlier waves trained developers, administrators and data engineers. The AI outlines are written for finance teams, project managers, marketing teams, procurement, business analysts and data analysts, often with "no coding required" in the first lines.
2. The format shortened. In 2019-20, 1 of 70 outline titles said "workshop". In 2026, it was 29 of 62.
3. The risk is part of the lesson. In earlier waves governance was usually a closing module, if it appeared at all. As the chart further down shows, guardrails, validation and human review now appear in most AI outlines I write.
4. The tools reshuffle within quarters. Major platforms renamed and repackaged their AI products several times in 2026 alone, so an outline can be out of date before the first class.

In the topics chart above, prompt engineering is the most common topic, but guardrails, hallucination checks and human review appear in roughly three of every four AI outlines I wrote this year, and cost awareness and evaluation are not far behind. What buyers are really asking for is not "how to use the tool". It is "how to use it without creating a new risk".

What I would tell a learning leader planning for the next wave

• Fund concepts before products. The Hadoop and RPA curves show how fast a tool can lose its market while the skills underneath (SQL, process thinking, data modeling) stay useful.
• Treat AI training as workflow training. The outlines I would defend start from the learner's real task (a variance report, a contract redline, a status update) and end with a reviewable output, not a tool tour.
• Budget for refresh. Plan to revisit AI content quarterly, not yearly.
• Build governance into the first lab, not the last slide. People should know what they are allowed to do, and what must be checked, before they start experimenting.
• Measure outputs, not attendance. Ask every participant to leave with one working, documented use case, so you can see whether usage continues after the course.
• Expect broader audiences. The World Economic Forum expects 39% of workers' core skills to change by 2030, and the group that most needs training is no longer only engineers.

Where this leaves us

Every wave in my archive had the same pattern. A technology gets big, the market panics about skills, training boom follows, and then the tooling consolidates and the durable skills survive. In my archive, the AI wave is moving faster than any of the previous ones, and it is reaching a wider audience. That makes hands-on, role-specific, governed training more valuable, not less.

If you are planning training for a team, I would be glad to compare notes. Explore our hands-on workshops or get in touch.

Sources: Gartner press releases (17 Nov 2020, 19 Nov 2024); IDC Data Age 2025 (Dec 2018) and Worldwide Digital Transformation Spending Guide (Nov 2023); Stack Overflow Developer Surveys 2024 and 2025; McKinsey Global Survey on the state of AI; World Economic Forum Future of Jobs Report 2025; trade coverage of the 2019 Cloudera and Hortonworks merger. Outline counts come from my own training archive.