What is a matrix factorization?
In math, a mechanism for finding the matrixes whose dot product approximates a target matrix.
matrix factorization explained in plain English
In math, a mechanism for finding the matrixes whose dot product approximates a target matrix. In recommendation systems, the target matrix often holds users' ratings on items. For example, the target matrix for a movie recommendation system might look something like the following, where the positive integers are user ratings and 0 means that the user didn't rate the movie: Black Panther --- 3.0 0.0 | 0.0 5.0 | 1.0 0.0 | The movie recommendation system aims to predict user ratings for unrated movies. For example, will User 1 like Black Panther? One approach for recommendation systems is to use matrix factorization to generate the following two matrixes: - A user matrix, shaped as the number of users X the number of embedding dimensions. - An item matrix, shaped as the number of embedding dimensions X the number of items. For example, using matrix factorization on our three users and five items could yield the following user matrix and item matrix:
The dot product of the user matrix and item matrix yields a recommendation matrix that contains not only the original user ratings but also predictions for the movies that each user hasn't seen. For example, consider User 1's rating of Casablanca, which was 5.0. The dot product corresponding to that cell in the recommendation matrix should hopefully be around 5.0, and it is:
More importantly, will User 1 like Black Panther? Taking the dot product corresponding to the first row and the third column yields a predicted rating of 4.3:
Example
Matrix factorization typically yields a user matrix and item matrix that, together, are significantly more compact than the target matrix.
People also read
- bag of words
A representation of the words in a phrase or passage, irrespective of order.
- bidirectional language model
A language model that determines the probability that a given token is present at a given location in an excerpt of text based on the preceding and following text.
- cross-entropy
A generalization of Log Loss to multi-class classification problems.
- dimension reduction
Decreasing the number of dimensions used to represent a particular feature in a feature vector, typically by converting to an embedding vector.
- dimensions
Overloaded term having any of the following definitions: The number of levels of coordinates in a Tensor.
- distillation
The process of reducing the size of one model (known as the teacher) into a smaller model (known as the student) that emulates the original model's predictions as faithfully as possible.
- embedding layer
A special hidden layer that trains on a high-dimensional categorical feature to gradually learn a lower dimension embedding vector.
- embedding space
The d-dimensional vector space that features from a higher-dimensional vector space are mapped to.
- embedding vector
Broadly speaking, an array of floating-point numbers taken from any hidden layer that describe the inputs to that hidden layer.
- encoder
In general, any ML system that converts from a raw, sparse, or external representation into a more processed, denser, or more internal representation.