User:Martijho

From Robin

(Difference between revisions)
Jump to: navigation, search
(Multitask Evolution with Cartesian Genetic Programming)
 
(39 intermediate revisions not shown)
Line 1: Line 1:
-
== Key "words" ==  
+
= Current draft =
-
* Super neural network
+
 
-
* Evolved sub-models from a larger set of parameters
+
Current draft of the thesis can be found by following this link
-
* Multitask learning
+
:[https://www.dropbox.com/s/063csu4cwxalo5v/Current%20master%20thesis%20draft%2030.4.pdf?dl=0 Current thesis draft]
-
* No catastrophic forgetting
+
Updated as of 30.04.2018
-
* Embedded transfer learning
+
 
 +
 
 +
; Presentations
 +
: [https://docs.google.com/presentation/d/1I3ObuFmTDMSaoST_G589VSaOIcRpst7osPGR-XxvyOs/edit?usp=sharing| Presentation of PathNet and research questions]
 +
: [https://docs.google.com/presentation/d/1QdQnJfcUNPkWDMSZ2dnVgq2qeHI5mQBge0E9WFBAgtM/edit?usp=sharing| Transfer learning in SNNs: (tl;dr) + First-path-experiments]
 +
: [https://docs.google.com/presentation/d/1Mh5z-AoWE9t0YtXMfJMIm7WHXWDVJkMEqZQvoOmUgtI/edit?usp=sharing| Transfer learning in SNNs: Search-experiments]
 +
: [https://docs.google.com/presentation/d/1jqZsRLzdY9ylh7p3nDj029sNSkalh1YmIEXOkBnhKok/edit?usp=sharing| ML for the cool kids]
 +
 
 +
= Thesis structure and notes =
 +
A seperate page describing outline and section structure of the thesis.
 +
: [[Martijho-PathNet-thesis|Thesis structure and outlines]]
 +
 
 +
= Experiments =
 +
A separate page describing research questions and the experiments proposed to answer them.
 +
: [[Martijho-PathNet-Experiments|Experiments]]
 +
 
 +
 
 +
= Terms to use in thesis =
 +
*; Plastic Neural Network
 +
: NN that change topology or connectivity according to learning algorithm
 +
*; In-silico
 +
: performed by computer
 +
*; Modular Super Neural Network
 +
: DNN consisting of modules of smaller NNs,
 +
*; Task-specific meme
 +
: Smallest concise unit of knowledge required to perform some task
 +
: Example - Task = Pick something up. Meme = Ability to bend index finger 
 +
*; Memetics
 +
: Study of information in an analogy to Darwinian evolution.
 +
*; Transferability
 +
: Ability to transfer/reuse knowledge between task
 +
*; Saturation in PathNet
 +
: Most or all modules are trained and locked to backpropagation
 +
*; Embedded transfer learning
 +
: Knowledge transfer capability is incorporated into the machine learning structure (PathNet, PNNs)
 +
*; Catastrophic forgetting
 +
: Forgetting previously known task when fine-tuning parameters
 +
*; Evolved sub-models
 +
: Using GAs to evolve paths through a larger set of parameters (PathNet functionality)
 +
 
 +
= Thoughts on Thesis =
 +
- Search for the first path is unnecessary? The search is over good permutations of parameters from the network at the same time
 +
the parameters are trained for the first time. In other words: does the search provide a significant increase in transferability or any measurable increase in performance over just picking a random path and training it for a set amount of iterations?
 +
 
 +
- When training on a saturated PathNet, it might be quicker to preprocess the data for each path (view it as feature extraction) since there is no backpropagation except for
 +
in the final task-specific softmax layer
 +
 
 +
- When training on a curriculum and decrease in batch size for each increase in the task difficulty might make sense.
 +
Easy examples have little "nuance" in-between datapoints so large batch size might increase convergence speed.
 +
Equivalently, complex tasks later on in the curriculum might have a lot of "detail" which will be drowned out if the batch size is kept constant.
== Thesis problem specification ==   
== Thesis problem specification ==   
Line 13: Line 62:
** What would, in that case, constitute a "saturated PathNet"?  
** What would, in that case, constitute a "saturated PathNet"?  
** Is there a learning advantage to be had from this kind of learning?  
** Is there a learning advantage to be had from this kind of learning?  
 +
* Is there a measurable increase in performance by searching over optimal "first paths" instead of just training a selected segment of the PathNet?
 +
 +
= PathNet Implementation =
 +
The pathnet is implementet using Keras with a tensorflow backend, in a object oriented structure with a high level of mudularity.
 +
 +
Pathnet layers are represented as subclasses of Layer. Currently only DenseLayer is implemented.
 +
These contain all modules in the layer and functionality for providing a log of layer-information (used for saving pathnet to disc), merging selected modules from the layer with a new model,
 +
temporarily storing weights in the layer and loading them back (used during backend session reset).
 +
Task-objects contain the unique softmax layer, a potential optimal path as well as functionality for providig log (again: saving pathnet to disc), applying unique layer to a new model.
 +
 +
A PathSearch class contain all implemented search algorithms (currently tournament and a simple evolutionary search are implemented). This class use a provided pathnet object which provides
 +
paths (genotypes) and models (for fitness evaluation). The search metods returns a optimal path along side a history-structure that are used in the Analytics class.
 +
Here, test results are stored and plotted.
=== PathNet structure ===  
=== PathNet structure ===  
Small structure to reduce computational requirements.  
Small structure to reduce computational requirements.  
-
* (3 layers 10-20 modules of small affine MLPs)
+
: (3 layers 10-20 modules of small affine MLPs)
=== Test scenario ===
=== Test scenario ===
Must be fairly quick to provide one episode. Small input dimentionality to reduce necessary capacity of PathNet structure and computational time.  
Must be fairly quick to provide one episode. Small input dimentionality to reduce necessary capacity of PathNet structure and computational time.  
-
Scenario must also be easy to divide into subtasks.
+
The scenario must also be easy to divide into subtasks.
* OpenAI gym?  
* OpenAI gym?  
** LunarLander:  
** LunarLander:  
Line 30: Line 92:
-
== Who cites PathNet? ==  
+
= Who cites PathNet? =
-
=== ''[https://arxiv.org/pdf/1703.10371.pdf Born to Learn]'' ===
+
''[https://arxiv.org/pdf/1703.10371.pdf Born to Learn]''
EPANN - Evolved Plastic Artificial Neural Networks
EPANN - Evolved Plastic Artificial Neural Networks
Mentions Pathnet as an example of where evolution where
Mentions Pathnet as an example of where evolution where
Line 39: Line 101:
evolution and deep learning approaches.
evolution and deep learning approaches.
-
=== ''[https://arxiv.org/pdf/1706.00046.pdf Learning time-efficient deep architectures with budgeted super networks]'' ===
+
''[https://arxiv.org/pdf/1706.00046.pdf Learning time-efficient deep architectures with budgeted super networks]''
Mentions PathNet as a predecessor in the super neural network family
Mentions PathNet as a predecessor in the super neural network family
-
=== '' [https://arxiv.org/pdf/1708.07902.pdf Deep Learning for video game playing]'' ===
+
'' [https://arxiv.org/pdf/1708.07902.pdf Deep Learning for video game playing]''
Reviewing recent deep learning advances in the context  
Reviewing recent deep learning advances in the context  
of how they have been applied to play different types of video games
of how they have been applied to play different types of video games
-
=== ''[http://ceur-ws.org/Vol-1958/IOTSTREAMING2.pdf Evolutive deep models for online learning on data streams with no storage]'' ===
+
''[http://ceur-ws.org/Vol-1958/IOTSTREAMING2.pdf Evolutive deep models for online learning on data streams with no storage]''
Pathnet is proposed alongside PNNS as a way to deal with changing environments. It is mentioned that both PathNet and progressive networks show good results on sequences of tasks and are a good alternative to fine-tuning to accelerate learning.  
Pathnet is proposed alongside PNNS as a way to deal with changing environments. It is mentioned that both PathNet and progressive networks show good results on sequences of tasks and are a good alternative to fine-tuning to accelerate learning.  
-
=== ''[https://openreview.net/pdf?id=H1XLbXEtg Online multi-task learning using active sampling]'' ===
+
''[https://openreview.net/pdf?id=H1XLbXEtg Online multi-task learning using active sampling]''  
Cites Progressive Neural Networks for multitask learning
Cites Progressive Neural Networks for multitask learning
-
=== ''[http://juxi.net/workshop/deep-learning-rss-2017/papers/Xu.pdf Hierarchical Task Generalization with Neural Programs]'' ===
+
''[http://juxi.net/workshop/deep-learning-rss-2017/papers/Xu.pdf Hierarchical Task Generalization with Neural Programs]''
Mentions PathNet as way of reusing weights
Mentions PathNet as way of reusing weights
-
=== ''[https://arxiv.org/pdf/1702.02217.pdf Multitask Evolution with Cartesian Genetic Programming]'' ===
+
 
 +
''[https://arxiv.org/pdf/1702.02217.pdf Multitask Evolution with Cartesian Genetic Programming]''  
Mentions PathNet in a list of systems that use evolution as tool in multitasking
Mentions PathNet in a list of systems that use evolution as tool in multitasking

Current revision as of 10:17, 30 April 2018

Contents

Current draft

Current draft of the thesis can be found by following this link

Current thesis draft

Updated as of 30.04.2018


Presentations
Presentation of PathNet and research questions
Transfer learning in SNNs: (tl;dr) + First-path-experiments
Transfer learning in SNNs: Search-experiments
ML for the cool kids

Thesis structure and notes

A seperate page describing outline and section structure of the thesis.

Thesis structure and outlines

Experiments

A separate page describing research questions and the experiments proposed to answer them.

Experiments


Terms to use in thesis

  • Plastic Neural Network
NN that change topology or connectivity according to learning algorithm
  • In-silico
performed by computer
  • Modular Super Neural Network
DNN consisting of modules of smaller NNs,
  • Task-specific meme
Smallest concise unit of knowledge required to perform some task
Example - Task = Pick something up. Meme = Ability to bend index finger
  • Memetics
Study of information in an analogy to Darwinian evolution.
  • Transferability
Ability to transfer/reuse knowledge between task
  • Saturation in PathNet
Most or all modules are trained and locked to backpropagation
  • Embedded transfer learning
Knowledge transfer capability is incorporated into the machine learning structure (PathNet, PNNs)
  • Catastrophic forgetting
Forgetting previously known task when fine-tuning parameters
  • Evolved sub-models
Using GAs to evolve paths through a larger set of parameters (PathNet functionality)

Thoughts on Thesis

- Search for the first path is unnecessary? The search is over good permutations of parameters from the network at the same time the parameters are trained for the first time. In other words: does the search provide a significant increase in transferability or any measurable increase in performance over just picking a random path and training it for a set amount of iterations?

- When training on a saturated PathNet, it might be quicker to preprocess the data for each path (view it as feature extraction) since there is no backpropagation except for in the final task-specific softmax layer

- When training on a curriculum and decrease in batch size for each increase in the task difficulty might make sense. Easy examples have little "nuance" in-between datapoints so large batch size might increase convergence speed. Equivalently, complex tasks later on in the curriculum might have a lot of "detail" which will be drowned out if the batch size is kept constant.

Thesis problem specification

Studying the behaviour of super neural networks when saturated with subtasks from the same domain such as in a curriculum learning scenario. Include research questions such as

  • Can we estimate the decline in needed capacity for each new sub-task learned from the curriculum?
  • Could a PathNet saturated with optimized paths for tasks from a curriculum provide one/few-shot learning?
    • What would, in that case, constitute a "saturated PathNet"?
    • Is there a learning advantage to be had from this kind of learning?
  • Is there a measurable increase in performance by searching over optimal "first paths" instead of just training a selected segment of the PathNet?

PathNet Implementation

The pathnet is implementet using Keras with a tensorflow backend, in a object oriented structure with a high level of mudularity.

Pathnet layers are represented as subclasses of Layer. Currently only DenseLayer is implemented. These contain all modules in the layer and functionality for providing a log of layer-information (used for saving pathnet to disc), merging selected modules from the layer with a new model, temporarily storing weights in the layer and loading them back (used during backend session reset). Task-objects contain the unique softmax layer, a potential optimal path as well as functionality for providig log (again: saving pathnet to disc), applying unique layer to a new model.

A PathSearch class contain all implemented search algorithms (currently tournament and a simple evolutionary search are implemented). This class use a provided pathnet object which provides paths (genotypes) and models (for fitness evaluation). The search metods returns a optimal path along side a history-structure that are used in the Analytics class. Here, test results are stored and plotted.

PathNet structure

Small structure to reduce computational requirements.

(3 layers 10-20 modules of small affine MLPs)

Test scenario

Must be fairly quick to provide one episode. Small input dimentionality to reduce necessary capacity of PathNet structure and computational time. The scenario must also be easy to divide into subtasks.

  • OpenAI gym?
    • LunarLander:
      • Hover
      • Land safely
      • Land in goal
      • Land in goal quickly


Who cites PathNet?

Born to Learn EPANN - Evolved Plastic Artificial Neural Networks Mentions Pathnet as an example of where evolution where used to train a network on multiple tasks. "While these results were only possible through significant computational resources, they demonstrate the potential of combining evolution and deep learning approaches.

Learning time-efficient deep architectures with budgeted super networks Mentions PathNet as a predecessor in the super neural network family

Deep Learning for video game playing Reviewing recent deep learning advances in the context of how they have been applied to play different types of video games

Evolutive deep models for online learning on data streams with no storage Pathnet is proposed alongside PNNS as a way to deal with changing environments. It is mentioned that both PathNet and progressive networks show good results on sequences of tasks and are a good alternative to fine-tuning to accelerate learning.

Online multi-task learning using active sampling Cites Progressive Neural Networks for multitask learning

Hierarchical Task Generalization with Neural Programs Mentions PathNet as way of reusing weights

Multitask Evolution with Cartesian Genetic Programming Mentions PathNet in a list of systems that use evolution as tool in multitasking

Personal tools
Front page