Introduction to NLP

1. A Brief History of NLP and the Role of Language in the Modern World

Natural Language Processing (NLP) emerged at the intersection of linguistics, computer science, and artificial intelligence. Early NLP systems (1950s–1970s) relied heavily on hand‑crafted rules and symbolic processing. These systems struggled to scale, as language is vast, ambiguous, and context-dependent.

In the 1980s and 1990s, statistical NLP became dominant due to increased computational power and the availability of large corpora. Methods such as n‑gram models, Hidden Markov Models (HMMs), and probabilistic parsers transformed NLP into a data-driven discipline.

Today, language drives modern technology: search engines, translation systems, chatbots, recommendation engines, and digital assistants all rely on NLP. The shift toward neural networks, word embeddings, and transformers has enabled models to capture semantic meaning, contextual nuance, and long‑range dependencies, making NLP central to modern AI.

2. Denotational Semantics and the Challenges of Symbol-Based Meaning

Denotational semantics assumes that a word directly refers to its meaning. For example, the word “tree” should ideally map to the concept of a tree (or an icon, definition, etc.). Early NLP methods attempted this using lexical databases such as WordNet, which organizes words into synsets (sets of cognitive synonyms).

While WordNet offers structured semantic relationships, it suffers from several limitations:

• Synonym errors (e.g., listing “proficient” as a synonym of “good,” which is imprecise).
• Meaning is static and does not adapt to context.
• No graded similarity (all synonyms are treated equally).
• Cannot model metaphorical or domain-specific usage.

These shortcomings motivated the shift from symbolic systems to distributional semantics, which underlies modern word vectors.

3. One‑Hot Vectors and Localist Representations

A one‑hot vector is a binary vector used to represent categorical items such as words in a vocabulary.

For example, if the vocabulary is:

["apple", "banana", "cat", "tree"]

Then the word “cat” is encoded as: