|
|
| (4 intermediate revisions not shown) |
| Line 1: |
Line 1: |
| - | == Capacity Increase ==
| |
| - | === Research question ===
| |
| - | Can we estimate the decline in needed capacity for each new sub-task learned from the curriculum?
| |
| - | How "much" capacity is needed to learn a new meme?
| |
| | | | |
| - | === Hypothesis ===
| |
| - | Previous studies show a decline in needed capacity for each new sub-task (cite: Progressive Neural Networks-paper).
| |
| - | If a metric can be defined for measuring the capacity change, we expect the results to confirm this.
| |
| - |
| |
| - | === Suggested experiment ===
| |
| - | === Implemented experiment ===
| |
| - | === Results ===
| |
| - | === Conclusion ===
| |
| - |
| |
| - | == Search for the first path? ==
| |
| - | === Research question ===
| |
| - | Is there anything to gain from performing a proper search for the first path versus just picking a random path and training the weights?
| |
| - |
| |
| - | === Hypothesis ===
| |
| - | I think performance will have the same asymptote, but it will be reached in fewer training iterations. The only thing that might be
| |
| - | influenced by this path selection is that the modules in PathNet might have more interconnected dependencies. Maybe the layers are more
| |
| - | "independent" when the weights are updated as part of multiple paths? This might be important for transferability when learning future tasks.
| |
| - |
| |
| - | === Suggested experiment ===
| |
| - | Performing multiple small multi-task learning scenarios. Two tasks should be enough, but it is necessary to show that modules are reused in each scenario.
| |
| - | Test both picking a path and the full-on search for a path and compare convergence time for the second task.
| |
| - |
| |
| - | === Implemented experiment ===
| |
| - | * Information about the execution of the experiment
| |
| - |
| |
| - | === Results ===
| |
| - | * Plots and table contents showing experiment results
| |
| - |
| |
| - | === Conclusion ===
| |
| - | * What does the results tell me
| |
| - |
| |
| - |
| |
| - |
| |
| - |
| |
| - |
| |
| - | ----
| |
| - |
| |
| - | == Format ==
| |
| - | === Research question ===
| |
| - | * What do I want to test/know/research?
| |
| - |
| |
| - | === Hypothesis ===
| |
| - | * What do I think my results will be and why?
| |
| - |
| |
| - | === Suggested experiment ===
| |
| - | * How is the experiment to be performed?
| |
| - | * What metrics are going to be used?
| |
| - |
| |
| - | === Implemented experiment ===
| |
| - | * Information about the execution of the experiment
| |
| - |
| |
| - | === Results ===
| |
| - | * Plots and table contents showing experiment results
| |
| - |
| |
| - | === Conclusion ===
| |
| - | * What does the results tell me
| |
