What is a Neural Architecture Search?
A technique for automatically designing the architecture of a neural network.
Neural Architecture Search explained in plain English
A technique for automatically designing the architecture of a neural network. NAS algorithms can reduce the amount of time and resources required to train a neural network. NAS typically uses: - A search space, which is a set of possible architectures. - A fitness function, which is a measure of how well a particular architecture performs on a given task. NAS algorithms often start with a small set of possible architectures and gradually expand the search space as the algorithm learns more about what architectures are effective. The fitness function is typically based on the performance of the architecture on a training set, and the algorithm is typically trained using a reinforcement learning technique. NAS algorithms have proven effective in finding high-performing architectures for a variety of tasks, including image classification, text classification, and machine translation.
Example
Practitioners refer to neural architecture search when building, training, or evaluating machine learning systems. It appears in research papers, product documentation, and technical discussions about AI capabilities and limitations.
People also read
- generative AI
An emerging transformative field with no formal definition.
- Long Short-Term Memory
A type of cell in a recurrent neural network used to process sequences of data in applications such as handwriting recognition, machine translation, and image captioning.
- mixture of experts
A scheme to increase neural network efficiency by using only a subset of its parameters (known as an expert) to process a given input token or example.
- pooling
Reducing a matrix (or matrixes) created by an earlier convolutional layer to a smaller matrix.
- Attention
A mechanism that lets a model focus on the most relevant parts of its input when producing an output, weighting what matters most in context.
- auto-regressive model
A model that infers a prediction based on its own previous predictions.
- autoencoder
A system that learns to extract the most important information from the input.
- automatic evaluation
Using software to judge the quality of a model's output.
- bag of words
A representation of the words in a phrase or passage, irrespective of order.
- BERT
A model architecture for text representation.