Continual backpropagation is a technique that selectively reinitializes low-utility units in a neural network. It uses a utility measure called contribution utility, which is based on the value of connections and weights in the network. This measure is updated over time to evaluate the importance of units in the network. Continual backpropagation helps in maintaining plasticity by adding random units from the initial distribution in each new task. This method combines conventional backpropagation with selective reinitialization to continually inject random units and prevent immediate reinitialization of new units.
In the specific Continual ImageNet environment, the database consists of 1,000 classes, each with 700 images. The network architecture used includes convolutional layers followed by fully connected layers. This setting challenges deep learning models to continually adapt to new tasks and maintain performance over multiple passes through the training set. By utilizing methods like continual backpropagation with specific hyperparameters and weight regularizations, the network can improve plasticity and enhance learning across tasks.
In another complex environment like the class-incremental CIFAR-100, the learning system incrementally gains access to more classes over time. The network architecture includes deep residual networks with various hyperparameters to address issues like weight decay, hyperparameter tuning, and mini-batch challenges. Various techniques such as selective reinitialization and utility measures help mitigate loss of plasticity and improve overall performance in continuing learning scenarios.
Further studies explore the robustness of loss of plasticity in scenarios such as the Slowly-Changing Regression problem and reinforcement learning environments like Ant-v3, Hopper-v3, and Walker-v3. By analyzing the effects of methods like L2 regularization, Dropout, online normalization, Shrink and Perturb, and Adam in these environments, researchers aim to understand the impact of different algorithms on mitigating loss of plasticity and enhancing continual learning capabilities. Future research directions include investigating the lottery ticket hypothesis, enhancing methods for continual plasticity preservation, and exploring the connections between noise injection and neural network training performance.