Did Google kill itself with Transformer?

Google was trying to build ChatGPT in 2012 😏

Or more specifically, a team inside Google was trying to build an AI for Google Search so that when people typed a question in the search box, the AI would directly give the answer instead of showing ten blue links to the user.

But the AI that they were using had a bad memory. It kept forgetting the user’s question. Answering it was far away.

The team tried different methods to increase the AI’s memory. Didn’t work. 🙄

One guy on the team, Jakob Uszkoreit, had an idea: instead of trying to increase this AI’s memory, why don’t we make another type of AI from scratch that has a strong memory?

No one took Jakob’s idea seriously.

But Jakob didn’t give up.

He found a couple of people in Google who found this idea interesting: Illia Polosukhin(AI scientist), Ashish Vaswani from Google Brain, Niki Parmar from Google Search, Llion Jones from Google Research, Łukasz Kaiser and his intern, Aidan Gomez.

Together, they started working to turn Jakob’s idea(new type of AI) into reality.

They called it: Transformer.

They quickly built the prototype version of transformer and used it to make a translator.

For strong memory, they designed the transformer to look at texts in one shot instead of word-by-word. So the transformer remembered what it read earlier.

The translator performed quite well. Full speed ahead. ⚡

But then they hit a wall.

Making the prototype version of transformer was easy. But they didn’t know how to make it perfect.

The problem was, these people were brilliant and smart, but to build a new type of AI, you need a legend, duh! You need lisan al gaib.

Enter Noam Shazeer.

Noam was a legend at Google. He was one of the earliest employees, built Google’s ad system, was a principal engineer, and had written countless research papers on AI.

One day in 2017, he was walking in Google’s Building 1965 when he heard some people talking about a new type of AI.

He was amazed. He joined Team Transformer immediately.

On his first day, he threw away the code that the team had written. 💁‍♂️

He came back with his own code for the transformer. 😎

After that, the team perfected the idea of transformer and wrote a research paper about transformers called Attention Is All You Need.

Google execs did not understand the paper, but a small company called OpenAI did.

They used transformer to create an AI called ChatGPT, where T stood for Transformer.

And the rest is history. 💹

Happy to know this post has been reposted 115k times on LinkedIn 😊