Experiments
From Robin
Line 12: | Line 12: | ||
=== Results === | === Results === | ||
=== Conclusion === | === Conclusion === | ||
+ | |||
+ | |||
+ | |||
= Search for the first path? = | = Search for the first path? = |
Revision as of 12:33, 8 November 2017
Contents |
Capacity Increase
Research question
Can we estimate the decline in needed capacity for each new sub-task learned from the curriculum? How "much" capacity is needed to learn a new meme?
Hypothesis
Previous studies show a decline in needed capacity for each new sub-task (cite: Progressive Neural Networks-paper). If a metric can be defined for measuring the capacity change, we expect the results to confirm this.
Suggested experiment
Implemented experiment
Results
Conclusion
Search for the first path?
Research question
Is there anything to gain from performing a proper search for the first path versus just picking a random path and training the weights?
Hypothesis
I think performance will have the same asymptote, but it will be reached in fewer training iterations. The only thing that might be influenced by this path selection is that the modules in PathNet might have more interconnected dependencies. Maybe the layers are more "independent" when the weights are updated as part of multiple paths? This might be important for transferability when learning future tasks.
Suggested experiment
Performing multiple small multi-task learning scenarios. Two tasks should be enough, but it is necessary to show that modules are reused in each scenario. Test both picking a path and the full-on search for a path and compare convergence time for the second task.
Implemented experiment
- Information about the execution of the experiment
Results
- Plots and table contents showing experiment results
Conclusion
- What does the results tell me