|
|
(6 intermediate revisions not shown) |
Line 1: |
Line 1: |
- | == Capacity Increase ==
| |
- | === Research question ===
| |
- | Can we estimate the decline in needed capacity for each new sub-task learned from the curriculum?
| |
- | How "much" capacity is needed to learn a new meme?
| |
| | | |
- | === Hypothesis ===
| |
- | Previous studies show a decline in needed capacity for each new sub-task (cite: Progressive Neural Networks-paper).
| |
- | If a metric can be defined for measuring the capacity change, we expect the results to confirm this.
| |
- |
| |
- |
| |
- | === Suggested experiment ===
| |
- | === Implemented experiment ===
| |
- | === Results ===
| |
- | === Conclusion ===
| |
- |
| |
- |
| |
- |
| |
- |
| |
- |
| |
- | == Search for the first path? ==
| |
- | === Research question ===
| |
- | Is there anything to gain from performing a proper search for the first path versus just picking a random path and training the weights?
| |
- |
| |
- | === Hypothesis ===
| |
- | I think performance will have the same asymptote, but it will be reached in fewer training iterations. The only thing that might be
| |
- | influenced by this path selection is that the modules in PathNet might have more interconnected dependencies. Maybe the layers are more
| |
- | "independent" when the weights are updated as part of multiple paths? This might be important for transferability when learning future tasks.
| |
- |
| |
- | === Suggested experiment ===
| |
- | Performing multiple small multi-task learning scenarios. Two tasks should be enough, but it is necessary to show that modules are reused in each scenario.
| |
- | Test both picking a path and the full-on search for a path and compare convergence time for the second task.
| |
- |
| |
- | === Implemented experiment ===
| |
- | * Information about the execution of the experiment
| |
- |
| |
- | === Results ===
| |
- | * Plots and table contents showing experiment results
| |
- |
| |
- | === Conclusion ===
| |
- | * What does the results tell me
| |
- |
| |
- |
| |
- |
| |
- |
| |
- |
| |
- | == Format ==
| |
- | === Research question ===
| |
- | * What do I want to test/know/research?
| |
- |
| |
- | === Hypothesis ===
| |
- | * What do I think my results will be and why?
| |
- |
| |
- | === Suggested experiment ===
| |
- | * How is the experiment to be performed?
| |
- | * What metrics are going to be used?
| |
- |
| |
- | === Implemented experiment ===
| |
- | * Information about the execution of the experiment
| |
- |
| |
- | === Results ===
| |
- | * Plots and table contents showing experiment results
| |
- |
| |
- | === Conclusion ===
| |
- | * What does the results tell me
| |