What is a Bayesian neural network?
A probabilistic neural network that accounts for uncertainty in weights and outputs.
Bayesian neural network explained in plain English
A probabilistic neural network that accounts for uncertainty in weights and outputs. A standard neural network regression model typically predicts a scalar value; for example, a standard model predicts a house price of 853,000. In contrast, a Bayesian neural network predicts a distribution of values; for example, a Bayesian model predicts a house price of 853,000 with a standard deviation of 67,200. A Bayesian neural network relies on Bayes' Theorem to calculate uncertainties in weights and predictions. A Bayesian neural network can be useful when it is important to quantify uncertainty, such as in models related to pharmaceuticals. Bayesian neural networks can also help prevent overfitting.
Example
Practitioners refer to bayesian neural network when building, training, or evaluating machine learning systems. It appears in research papers, product documentation, and technical discussions about AI capabilities and limitations.
People also read
- Backpropagation
The process that tells a neural network which internal settings caused an error and how to adjust them, working backwards through layers.
- embedding layer
A special hidden layer that trains on a high-dimensional categorical feature to gradually learn a lower dimension embedding vector.
- full softmax
Synonym for softmax.
- generative model
Practically speaking, a model that does either of the following: - Creates (generates) new examples from the training dataset.
- gradient boosting
A training algorithm where weak models are trained to iteratively improve the quality (reduce the loss) of a strong model.
- Gradient Descent
The method by which a model gradually improves by making small adjustments after each mistake, moving toward better performance.
- input layer
The layer of a neural network that holds the feature vector.
- logistic regression
A type of regression model that predicts a probability.
- minimax loss
A loss function for generative adversarial networks, based on the cross-entropy between the distribution of generated data and real data.
- sigmoid function
A mathematical function that "squishes" an input value into a constrained range, typically 0 to 1 or -1 to +1.