What Is Co-occurrence?

How Co-occurrence Works

Co-occurrence rests on one deceptively simple idea, the distributional hypothesis: words that appear in similar contexts tend to have similar meanings. If you never knew what “tesgüino” meant but saw it turn up beside “drink,” “corn,” “bottle,” and “drunk,” you would infer it is an alcoholic beverage — from its company, not a definition. Co-occurrence is the formal version of that inference: count how often terms show up near each other, and the counts start to encode meaning.

In practice a system slides a window across a corpus and tallies, for each word, which other words fall inside that window. Collected across millions of sentences, these tallies form a co-occurrence matrix: each row is a word, each column a possible neighbor, and each cell the number of times the two appeared together. A word’s row becomes a vector describing it by its neighbors, and two words with similar rows can be treated as similar in meaning — which is why cosine similarity over those rows is a natural measure of relatedness and a direct route to semantic similarity.

Those raw counts can be compressed into dense vector embeddings through matrix factorization, and this is exactly the lineage of count-based embedding methods. It also matters beyond words: when a brand consistently co-occurs with the concepts and entities it wants to be known for, search and AI systems pick up the association from the pattern itself, with no hyperlink required.

Example of Co-occurrence

The definitive worked example comes from the 2014 paper that introduced GloVe, by Jeffrey Pennington, Richard Socher, and Christopher Manning at Stanford. Its opening table uses co-occurrence to tease apart two related words, ice and steam, by watching how each co-occurs with a set of probe words drawn from a corpus.

The raw probabilities alone are noisy, so the paper looks at their ratio. For the probe word solid, the probability of seeing it near ice is 0.00019 and near steam just 0.000022 — a ratio of 8.9, correctly flagging that “solid” is a property of ice, not steam. Flip to gas, and the ratio inverts to about 0.085, because gas belongs with steam. For water, related to both, the ratio is close to balanced at 1.36, and for an unrelated word like fashion it sits near 0.96, essentially neutral. The pattern is striking: a plain count of which words appear together, run through a ratio, cleanly recovers real semantic relationships without anyone ever labeling them.

That table is the seed of the whole GloVe method. Its authors argued that these co-occurrence ratios, not individual probabilities, carry the meaning, and built a model that learns word vectors specifically to reproduce them. The result is a vector space where the geometry between words mirrors the co-occurrence statistics of the corpus they were drawn from — meaning distilled straight out of context.

Frequently Asked Questions

The Bottom Line

Co-occurrence turns the company a word keeps into evidence of what it means. By tallying which terms cluster together across a large body of text, systems learn associations no dictionary spells out. It is the raw material behind count-based embeddings and a quiet force in how search engines decide what a brand or page is genuinely about, built from consistent context rather than from keywords or links alone.