Another duplicate content thread with a different perspective at WebmasterWorld. Google has been awarded a patent for a 2001 submission of an application named Methods and apparatus for estimating similarity. The patent is a relatively short read compared to others, here is the abstract:
A similarity engine generates compact representations of objects called sketches. Sketches of different objects can be compared to determine the similarity between the two objects. The sketch for an object may be generated by creating a vector corresponding to the object, where each coordinate of the vector is associated with a corresponding weight. The weight associated with each coordinate in the vector is multiplied by a predetermined hashing vector to generate a product vector, and the product vectors are summed. The similarity engine may then generate a compact representation of the object based on the summed product vector.
It is old but can be a fun read for some.
Forum discussion at WebmasterWorld.
