Unity

From Robin

(Difference between revisions)
Jump to: navigation, search
(→Unity timing / indeterminism fixing (how to get your simulator deterministic))
 
(29 intermediate revisions not shown)
Line 1: Line 1:
== Installation ==
== Installation ==
-
* Download and install Unity : unity.com
+
=== Download and install Unity ===
-
For coding in Unity and Python it is recommended to use Visual Studio Code but you can of course use your preferred IDE and editor.
+
[https://www.unity.com Official Unity Website]
-
* Install Unity ML-Agents
+
=== Installing Unity ML-Agents ===
-
In the Unity editor, go to window->package manager. Inside the package manager search for ML agents in the Unity Registry. Install the most recent ML Agents package.
+
-
== Documentation ==
+
In the Unity editor, go to ''window->package manager''. Inside the package manager search for ML agents in the Unity Registry. Install the most recent ML Agents package. ML-Agents examples can be downloaded from the [https://github.com/Unity-Technologies/ml-agents/blob/main/docs/Installation.md ML-Agents Github]
 +
 
 +
=== Setting up the Python API ===
 +
 
 +
''pip install mlagents''
-
[https://github.com/Unity-Technologies/ml-agents Unity ML-Agents Github Page]
 
-
[https://docs.unity3d.com/Manual/index.html Unity Documentation]
 
== Using Unity ML-Agents ==  
== Using Unity ML-Agents ==  
-
** Build a standalone Unity Executable
+
=== Running ML-Agents through the Python API ===
-
You can run a Unity build (executable) from the Unity ML agents python scripts where this build is a standalone version of the game (robot simulation). You will have to create this build from the Unity editor using by adding an active scene (the scene that contains the ML agent) to the Scenes in build. It is explained in more detail here.
+
 +
You can interact with Unity ML-Agents by [1] opening Unity through python (''env = UnityEnvironment(side_channels=[])''), [2] resetting the environment (''env.reset()''), [3] stepping the environment forward until a new input from python is required (''env.step()''), [4] getting the state of the agent (with any sensory information; ''env.get_steps(behavior_name: str)'') and [5] setting the actions for agents (''env.set_actions(behavior_name: str, action: ActionTuple)''). For more information, please consult the guides below.
-
3. Using the python scripts
+
[https://github.com/Unity-Technologies/ml-agents/blob/main/docs/Python-API.md Python API]
-
The 4 python scripts included in this repository are the bare minimum of what you would need to run a standard evolutionary algorithm. To get started, make sure to make a virtual environment and install the python packages from the requirements.txt. You can find more information on how to do this here
+
-
3.1 UnityMLInterface
+
[https://github.com/Unity-Technologies/ml-agents/blob/main/docs/Python-API-Documentation.md Python API Documentation]
-
The Unity ML agents script contains code that allows you to open a Unity executable of your project and send motor commands to the environment. Note that you will have to change the PATH variable to point to your standalone build of the robot simulation.
+
-
3.2 EvolutionaryAlgorithm
+
== Documentation ==
-
The Evolutionary Algorithm script contains a minimal example of how you can Distributed Evolutionary Algorithms in Python (DEAP). You will have to pass references of the environment, the evaluation function, the individual, and the controller to the evolutionary algorithm.
+
-
ea = EvolutionaryAlgorithm.EvolutionaryAlgorithm(env, evaluate,EvolutionaryAlgorithm.Individual, SmallWorldNeuralNetwork.SmallWorldNeuralNetwork)
+
[https://github.com/Unity-Technologies/ml-agents Unity ML-Agents Github Page]
-
3.3 SmallWorldNeuralNetwork
+
[https://docs.unity3d.com/Manual/index.html Unity Documentation]
-
The SmallWorldNeuralNetwork is a simple type of neural network that can be evolved using the evolutionary algorithm. The network serves as an example but have a look at it to hack in your own type of controller.
+
-
3.4 EvolvingUnityMLAgents
+
== Troubleshooting ==
-
The EvolvingUnityMLAgents script combines the above implementations to evolve the behavior of the robot. This example runs on a single thread and will have to be turned into a multithreaded solution for your experiments.
+
-
3.5 Unity ODRI package
+
Although it is in principle quite easy to set up a Unity ML-Agents environment, in practice there can be a few tricky bugs either with Unity failing to start, version incompatibilities, or dependencies. If you're having specific issues with Unity and/or ML-Agents, for now, contact Frank.
-
The Open Dynamic Robot Initiative Quadruped package can be imported into Unity by going to Assets->Import Package->Custom Package. It requires ML agents version 1.0.7.
+
-
== Unity timing / indeterminism fixing (how to get your simulator deterministic) ==
+
=== Getting rid of too many log messages in the terminal ===
-
[[Unity-Timing-David | David's tips on getting deterministic timing]]
+
Depending on your experiments, it can be the case that you'll see that Unity is sending a bit too many messages. To limit the number of log messages, go to ''Project Settings->Player->Other Settings->Stack Trace'' and adjust the settings accordingly.
-
MonoBehaviour.Update() and MonoBehaviour.FixedUpdate() are common script methods to use with C# in Unity — primarily to capture functionality you want to occur at every frame in your simulation.
+
== Notes on deterministic behavior ==
-
In order to time-delay your C# functions/methods to be called in your simulation (e.g. to make time-based rules for how long robots have to wait before doing something in Unity), you can (most easily, but not robustly) use the MonoBehaviour.Invoke() function and pass in the amount of seconds you want your robot to wait. Another approach to this (slightly more complex, but definitely doable (ask Frank) and much more stable) is to use a Unity Coroutine.
+
To ensure determinism, it seems that the physics of a scene needs to be reset. This can be done by loading a scene whenever you are evaluating something. Note that when coupling Python and Unity in the ''Editor Mode'', experiments will not be deterministic. You will have to build and run an executable of your project to get deterministic behavior. One can subscribe a function to the ''Academy.Instance.OnEnvironmentReset'' event in Unity that loads the scene containing the robot. You can't just reset the scene then you will lose the connection between python and ml-agents. You then have to create a "singleton" that is not reset when the scene is reloaded. The idea is to have a "RobotManager" with a single tone that keeps on and registers all the side channels. Robotmanager does very little other than maintain the connection, in another script you can have the agent script as usual.
 +
PS: Since you reload the scene, it will also create a "new" agent, it behaves the same but it will get a different id recommended way to retrieve the observations:
 +
    decisionSteps,other = self.env.get_steps(individual_name)
 +
    obs = decisionSteps.obs # <---- Beste måten å gjøre det på når det er en agent
 +
    end_position = obs[0][0][:3] # Henter observasjonene til agent 0
-
What MonoBehaviour.Update() and MonoBehaviour.Invoke() have in common (apart from being overused) is that they both depend on the thread in Unity that has to do with graphics and frame rate; so if your computer is working hard and is heating up, leading to reduced frame rate, your Update() and Invoke() functions will consequently be affected, and will (at least if time is critical for your simulation) hence lead to simulator indeterminism (i.e. you will not get identical end results from identical starting points). Conversely, MonoBehaviour.FixedUpdate() is, and Coroutines can be, dependent on fixed time in Unity. That your functionality runs on fixed time (as opposed to the first mentioned graphics thread) means that Unity keeps detailed track of time — even though your simulation might take longer in real life due to a warm computer and lower frame rate.
+
How to declare a singelton:
 +
    public class RobotManager : MonoBehaviour
 +
    {
 +
        // singleton
 +
        private static RobotManager instance;
 +
        public static RobotManager Instance { get { return instance; } }
 +
    }
-
This latter way, with MonoBehaviour.FixedUpdate() and Coroutines, is *the* way to go if you want to get rid of indeterminism and inconsistent timing in your simulation, and if you want to get the same results from your Unity simulation given the same input-variables, hyperparameters, as well as the same software/code. Trust me, it can save you months of worthless debugging.
+
[https://docs.nvidia.com/gameworks/content/gameworkslibrary/physx/guide/Manual/BestPractices.html#determinism  On PhysX determinism]
-
Some *minor* ordering indeterminisms/inconsistencies in your simulation can also occur by using MonoBehaviour.Update() in too many individual Unity GameObjects simultaneously; that is, Unity can switch up which GameObjects's Update() function it calls first if they are to be called simultaneously.
+
[[Unity-Timing-David | David's tips on getting deterministic timing]]

Current revision as of 16:52, 20 February 2024

Contents

Installation

Download and install Unity

Official Unity Website

Installing Unity ML-Agents

In the Unity editor, go to window->package manager. Inside the package manager search for ML agents in the Unity Registry. Install the most recent ML Agents package. ML-Agents examples can be downloaded from the ML-Agents Github

Setting up the Python API

pip install mlagents


Using Unity ML-Agents

Running ML-Agents through the Python API

You can interact with Unity ML-Agents by [1] opening Unity through python (env = UnityEnvironment(side_channels=[])), [2] resetting the environment (env.reset()), [3] stepping the environment forward until a new input from python is required (env.step()), [4] getting the state of the agent (with any sensory information; env.get_steps(behavior_name: str)) and [5] setting the actions for agents (env.set_actions(behavior_name: str, action: ActionTuple)). For more information, please consult the guides below.

Python API

Python API Documentation

Documentation

Unity ML-Agents Github Page Unity Documentation

Troubleshooting

Although it is in principle quite easy to set up a Unity ML-Agents environment, in practice there can be a few tricky bugs either with Unity failing to start, version incompatibilities, or dependencies. If you're having specific issues with Unity and/or ML-Agents, for now, contact Frank.

Getting rid of too many log messages in the terminal

Depending on your experiments, it can be the case that you'll see that Unity is sending a bit too many messages. To limit the number of log messages, go to Project Settings->Player->Other Settings->Stack Trace and adjust the settings accordingly.

Notes on deterministic behavior

To ensure determinism, it seems that the physics of a scene needs to be reset. This can be done by loading a scene whenever you are evaluating something. Note that when coupling Python and Unity in the Editor Mode, experiments will not be deterministic. You will have to build and run an executable of your project to get deterministic behavior. One can subscribe a function to the Academy.Instance.OnEnvironmentReset event in Unity that loads the scene containing the robot. You can't just reset the scene then you will lose the connection between python and ml-agents. You then have to create a "singleton" that is not reset when the scene is reloaded. The idea is to have a "RobotManager" with a single tone that keeps on and registers all the side channels. Robotmanager does very little other than maintain the connection, in another script you can have the agent script as usual. PS: Since you reload the scene, it will also create a "new" agent, it behaves the same but it will get a different id recommended way to retrieve the observations: 

   decisionSteps,other = self.env.get_steps(individual_name)
   obs = decisionSteps.obs # <---- Beste måten å gjøre det på når det er en agent
   end_position = obs[0][0][:3] # Henter observasjonene til agent 0

How to declare a singelton: 

   public class RobotManager : MonoBehaviour
   {
       // singleton
       private static RobotManager instance;
       public static RobotManager Instance { get { return instance; } }
   }

On PhysX determinism

David's tips on getting deterministic timing

Retrieved from "https://robin.wiki.ifi.uio.no/Unity"
Personal tools
Front page