In data analysis, cosine similarity is a measure of similarity between two non-zero vectors defined in an inner product space. Cosine similarity is the cosine of the angle between the vectors; that is, it is the dot product of the vectors divided by the product of their lengths. It follows that the cosine similarity does not depend on the magnitudes of the vectors, but only on their angle. The cosine similarity always belongs to the interval
[-1,1].
[0,1]
For example, in information retrieval and text mining, each word is assigned a different coordinate and a document is represented by the vector of the numbers of occurrences of each word in the document. Cosine similarity then gives a useful measure of how similar two documents are likely to be, in terms of their subject matter, and independently of the length of the documents.[1]
The technique is also used to measure cohesion within clusters in the field of data mining.[2]
One advantage of cosine similarity is its low complexity, especially for sparse vectors: only the non-zero coordinates need to be considered.
Other names for cosine similarity include Orchini similarity and Tucker coefficient of congruence; the Otsuka–Ochiai similarity (see below) is cosine similarity applied to binary data.
The cosine of two non-zero vectors can be derived by using the Euclidean dot product formula:
A ⋅ B =\left\|A\right\|\left\|B\right\|\cos\theta
Given two n-dimensional vectors of attributes, A and B, the cosine similarity,, is represented using a dot product and magnitude as
cosinesimilarity=SC(A,B):=\cos(\theta)={A ⋅ B\over\|A\|\|B\|}=
| |||||||||
| }{ |
| n | |
| \sqrt{\sum\limits | |
| i=1 |
| 2}} | |
| {A | |
| i |
⋅
| n | |
| \sqrt{\sum\limits | |
| i=1 |
| 2}} | |
| {B | |
| i |
},
where
Ai
Bi
i
A
B
The resulting similarity ranges from -1 meaning exactly opposite, to 1 meaning exactly the same, with 0 indicating orthogonality or decorrelation, while in-between values indicate intermediate similarity or dissimilarity.
For text matching, the attribute vectors A and B are usually the term frequency vectors of the documents. Cosine similarity can be seen as a method of normalizing document length during comparison. In the case of information retrieval, the cosine similarity of two documents will range from
0\to1
If the attribute vectors are normalized by subtracting the vector means (e.g.,
A-\bar{A}
ifA=[A1,
| T, | |
| A | |
| 2] |
then\bar{A}=\left[
| (A1+A2) | , | |
| 2 |
| (A1+A2) | |
| 2 |
\right]T,
soA-\bar{A}=\left[
| (A1-A2) | , | |
| 2 |
| (-A1+A2) | |
| 2 |
\right]T.
When the distance between two unit-length vectors is defined to be the length of their vector difference then
Nonetheless the cosine distance[3] is often defined without the square root or factor of 2:
cosinedistance=DC(A,B):=1-SC(A,B).
It is important to note that, by virtue of being proportional to squared Euclidean distance, the cosine distance is not a true distance metric; it does not exhibit the triangle inequality property — or, more formally, the Schwarz inequality — and it violates the coincidence axiom. To repair the triangle inequality property while maintaining the same ordering, one can convert to Euclidean distance or angular distance . Alternatively, the triangular inequality that does work for angular distances can be expressed directly in terms of the cosines; see below.
The normalized angle, referred to as angular distance, between any two vectors
A
B
When the vector elements may be positive or negative:
angulardistance=D\theta:=
| \arccos(cosinesimilarity) | |
| \pi |
=
| \theta | |
| \pi |
angularsimilarity=S\theta:=1-angulardistance=1-
| \theta | |
| \pi |
Or, if the vector elements are always positive:
angulardistance=D\theta:=
| 2 ⋅ \arccos(cosinesimilarity) | |
| \pi |
=
| 2\theta | |
| \pi |
angularsimilarity=S\theta:=1-angulardistance=1-
| 2\theta | |
| \pi |
Another effective proxy for cosine distance can be obtained by
L2
In biology, there is a similar concept known as the Otsuka–Ochiai coefficient named after Yanosuke Otsuka (also spelled as Ōtsuka, Ootsuka or Otuka,[5] Japanese: 大塚 弥之助)[6] and Akira Ochiai (Japanese: 落合 明),[7] also known as the Ochiai–Barkman[8] or Ochiai coefficient,[9] which can be represented as:
K=
| |A\capB| | |
| \sqrt{|A| x |B| |
A
B
|A|
A
In a recent book,[11] the coefficient is tentatively misattributed to another Japanese researcher with the family name Otsuka. The confusion arises because in 1957 Akira Ochiai attributes the coefficient only to Otsuka (no first name mentioned)[7] by citing an article by Ikuso Hamai (Japanese: 浜井 生三),[12] who in turn cites the original 1936 article by Yanosuke Otsuka.[6]
The most noteworthy property of cosine similarity is that it reflects a relative, rather than absolute, comparison of the individual vector dimensions. For any positive constant
a
V
V
aV
Cosine similarity is related to Euclidean distance as follows. Denote Euclidean distance by the usual
\|A-B\|
\|A-B\|2=(A-B) ⋅ (A-B)=\|A\|2+\|B\|2-2(A ⋅ B)
by expansion. When and are normalized to unit length,
\|A\|2=\|B\|2=1
2(1-\cos(A,B)).
In short, the cosine distance can be expressed in terms of Euclidean distance as
DC(A,B)=
| \|A-B\|2 | |
| 2 |
when \|A\|2=\|B\|2=1
The Euclidean distance is called the chord distance (because it is the length of the chord on the unit circle) and it is the Euclidean distance between the vectors which were normalized to unit sum of squared values within them.
Null distribution: For data which can be negative as well as positive, the null distribution for cosine similarity is the distribution of the dot product of two independent random unit vectors. This distribution has a mean of zero and a variance of
1/n
n
n
The ordinary triangle inequality for angles (i.e., arc lengths on a unit hypersphere) gives us that
|~\angle{AC}-\angle{CB}~|\le~\angle{AB}~\le~\angle{AC}~+~\angle{CB}~.
\cos(\angle{AC}-\angle{CB})\ge\cos(\angle{AB})\ge\cos(\angle{AC}+\angle{CB}).
\cos(A,C) ⋅ \cos(C,B)+\sqrt{\left(1-\cos(A,C)2\right) ⋅ \left(1-\cos(C,B)2\right)}\geq\cos(A,B),
\cos(A,B)\geq\cos(A,C) ⋅ \cos(C,B)-\sqrt{\left(1-\cos(A,C)2\right) ⋅ \left(1-\cos(C,B)2\right)}.
This form of the triangle inequality can be used to bound the minimum and maximum similarity of two objects A and B if the similarities to a reference object C is already known. This is used for example in metric data indexing, but has also been used to accelerate spherical k-means clustering[17] the same way the Euclidean triangle inequality has been used to accelerate regular k-means.
A soft cosine or ("soft" similarity) between two vectors considers similarities between pairs of features.[18] The traditional cosine similarity considers the vector space model (VSM) features as independent or completely different, while the soft cosine measure proposes considering the similarity of features in VSM, which help generalize the concept of cosine (and soft cosine) as well as the idea of (soft) similarity.
For example, in the field of natural language processing (NLP) the similarity among features is quite intuitive. Features such as words, n-grams, or syntactic n-grams[19] can be quite similar, though formally they are considered as different features in the VSM. For example, words “play” and “game” are different words and thus mapped to different points in VSM; yet they are semantically related. In case of n-grams or syntactic n-grams, Levenshtein distance can be applied (in fact, Levenshtein distance can be applied to words as well).
For calculating soft cosine, the matrix is used to indicate similarity between features. It can be calculated through Levenshtein distance, WordNet similarity, or other similarity measures. Then we just multiply by this matrix.
Given two -dimension vectors
a
b
\begin{align} \operatorname{soft\cosine}1(a,b)=
| ||||||||||
|
| N | |
| \sqrt{\sum\nolimits | |
| i,j |
sijbibj}}, \end{align}
where .
If there is no similarity between features (for), the given equation is equivalent to the conventional cosine similarity formula.
The time complexity of this measure is quadratic, which makes it applicable to real-world tasks. Note that the complexity can be reduced to subquadratic.[20] An efficient implementation of such soft cosine similarity is included in the Gensim open source library.