subject: Latent Semantic Indexing Is Not For Dinosaurs [print this page] Heard about latent semantic indexing? Ever wondered what latent semantic indexing is? True, you probably haven't given latent semantic indexing a lot of thought, unless you're on the geek end of the spectrum. But there are a lot of us geeks around and you probably wouldn't even be reading an article on latent semantic indexing unless you had some geekish tendencies. I won't tell. But I will tell you about latent semantic indexing, as it's a key component of search engine optimization. And SEO is a really important part of any internet marketing campaign that's going to be successful.
I find it easiest to describe latent semantic indexing by doing a semantic analysis on the term. That is, break the term down into the meaning of each word and then build it back up to a meaning of the entire term. This is really appropriate way to understand LSI, as semantic analysis is actually a preliminary step to latent semantic indexing itself.
After all, that term semantic is right there in LSI, so the first thing to know about latent semantic indexing is that it is all about the meanings of the words and terms in an article being indexed. Clearly, an LSI process will be creating an index of a list of words and terms, and an index is just an ordered form of a list. A semantic indexing means that the order of the list will be according to the meaning of the terms.
And that's where it gets interesting. How does one order a list of words according to their meaning? Don't words just mean what they mean? How can you assign some kind of numerical ordering to all those words? Well, it is a little arbitrary to assign a numerical value to the meaning of words, but it's also quite practical. And it's not something that is entirely made up on the fly. Linguists have long used the term "value decomposition" to describe how the meaning of a word can be arrived at. Whenever you're analyzing something in that much detail, it lends itself to some form of ordering, and thus ranking, or indexing. And it is the details that bring out the latent meanings of the words, thus giving full meaning to the term latent semantic indexing.
For example, the word dinosaur. The word can refer to a variety of animals that walked the earth millions of years ago. But if someone calls their computer a dinosaur, they're not calling it a meat-eater, but merely old. So when a search engine performs latent semantic indexing on an article with the word dinosaur, it needs to be aware of the many parts of the meaning of the word, and assign to it the part that makes the most sense in the current context. In the case of a dinosaur computer, latent semantic indexing would use the "old" part of dinosaur and then look for additional terms to support the theme of aging.
