Turning words into numbers that capture meaning

orig. “Efficient Estimation of Word Representations in Vector Space” · Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean

Machine Learning Beginner 3 min read Written, reviewed by Marginalia Editorial

In the margin

Heads up

This paper showed you can turn words into lists of numbers where "king minus man plus woman" lands near "queen".

What's going on

Computers need numbers, not words. Word2Vec learns to place each word at a point in space so that words used in similar ways sit close together. A surprising result was that simple arithmetic on these points captured real relationships, like capital cities to their countries. It learned all of this just by reading lots of text and predicting nearby words.

Why it matters

Turning words into meaningful numbers, called embeddings, became a building block for nearly all language AI that followed. The same idea now applies to images, products, and papers. It is one of the most practical ideas a newcomer can pick up early.

Source

Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean, Google

View on arXiv ↗ Read the PDF ↗

Still fuzzy on a sentence?

Paste it and we'll explain it even more simply.

Test your understanding

Member notes Sign in ↗

ME Marginalia Editorial TEAM

The editorial team picked this paper because it is worth your time, and an AI drafted this note. We check each one and rewrite it when the draft misses the mark.