Unity
From Robin
Contents |
Installation
- Download and install Unity : unity.com
For coding in Unity and Python it is recommended to use Visual Studio Code but you can of course use your preferred IDE and editor.
- Install Unity ML-Agents
In the Unity editor, go to window->package manager. Inside the package manager search for ML agents in the Unity Registry. Install the most recent ML Agents package.
Documentation
Using Unity ML-Agents
- Build a standalone Unity Executable
You can run a Unity build (executable) from the Unity ML agents python scripts where this build is a standalone version of the game (robot simulation). You will have to create this build from the Unity editor using by adding an active scene (the scene that contains the ML agent) to the Scenes in build. It is explained in more detail here.
3. Using the python scripts
The 4 python scripts included in this repository are the bare minimum of what you would need to run a standard evolutionary algorithm. To get started, make sure to make a virtual environment and install the python packages from the requirements.txt. You can find more information on how to do this here
3.1 UnityMLInterface The Unity ML agents script contains code that allows you to open a Unity executable of your project and send motor commands to the environment. Note that you will have to change the PATH variable to point to your standalone build of the robot simulation.
3.2 EvolutionaryAlgorithm The Evolutionary Algorithm script contains a minimal example of how you can Distributed Evolutionary Algorithms in Python (DEAP). You will have to pass references of the environment, the evaluation function, the individual, and the controller to the evolutionary algorithm.
ea = EvolutionaryAlgorithm.EvolutionaryAlgorithm(env, evaluate,EvolutionaryAlgorithm.Individual, SmallWorldNeuralNetwork.SmallWorldNeuralNetwork) 3.3 SmallWorldNeuralNetwork The SmallWorldNeuralNetwork is a simple type of neural network that can be evolved using the evolutionary algorithm. The network serves as an example but have a look at it to hack in your own type of controller.
3.4 EvolvingUnityMLAgents The EvolvingUnityMLAgents script combines the above implementations to evolve the behavior of the robot. This example runs on a single thread and will have to be turned into a multithreaded solution for your experiments.
3.5 Unity ODRI package The Open Dynamic Robot Initiative Quadruped package can be imported into Unity by going to Assets->Import Package->Custom Package. It requires ML agents version 1.0.7.
Unity timing / indeterminism fixing (how to get your simulator deterministic)
MonoBehaviour.Update() and MonoBehaviour.FixedUpdate() are common script methods to use with C# in Unity — primarily to capture functionality you want to occur at every frame in your simulation.
In order to time-delay your C# functions/methods to be called in your simulation (e.g. to make time-based rules for how long robots have to wait before doing something in Unity), you can (most easily, but not robustly) use the MonoBehaviour.Invoke() function and pass in the amount of seconds you want your robot to wait. Another approach to this (slightly more complex, but definitely doable (ask Frank) and much more stable) is to use a Unity Coroutine.
What MonoBehaviour.Update() and MonoBehaviour.Invoke() have in common (apart from being overused) is that they both depend on the thread in Unity that has to do with graphics and frame rate; so if your computer is working hard and is heating up, leading to reduced frame rate, your Update() and Invoke() functions will consequently be affected, and will (at least if time is critical for your simulation) hence lead to simulator indeterminism (i.e. you will not get identical end results from identical starting points). Conversely, MonoBehaviour.FixedUpdate() is, and Coroutines can be, dependent on fixed time in Unity. That your functionality runs on fixed time (as opposed to the first mentioned graphics thread) means that Unity keeps detailed track of time — even though your simulation might take longer in real life due to a warm computer and lower frame rate.
This latter way, with MonoBehaviour.FixedUpdate() and Coroutines, is *the* way to go if you want to get rid of indeterminism and inconsistent timing in your simulation, and if you want to get the same results from your Unity simulation given the same input-variables, hyperparameters, as well as the same software/code. Trust me, it can save you months of worthless debugging.
Some *minor* ordering indeterminisms/inconsistencies in your simulation can also occur by using MonoBehaviour.Update() in too many individual Unity GameObjects simultaneously; that is, Unity can switch up which GameObjects's Update() function it calls first if they are to be called simultaneously.
