Unity

From Robin

(Difference between revisions)
Jump to: navigation, search
(→Unity timing / indeterminism fixing (how to get your simulator deterministic))
 
(27 intermediate revisions not shown)
Line 1: Line 1:
== Installation ==
== Installation ==
-
* Download and install Unity : unity.com
+
=== Download and install Unity ===
-
For coding in Unity and Python it is recommended to use Visual Studio Code but you can of course use your preferred IDE and editor.
+
[https://www.unity.com Official Unity Website]
-
* Install Unity ML-Agents
+
=== Installing Unity ML-Agents ===
-
In the Unity editor, go to window->package manager. Inside the package manager search for ML agents in the Unity Registry. Install the most recent ML Agents package.
+
-
== Documentation ==
+
In the Unity editor, go to ''window->package manager''. Inside the package manager search for ML agents in the Unity Registry. Install the most recent ML Agents package. ML-Agents examples can be downloaded from the [https://github.com/Unity-Technologies/ml-agents/blob/main/docs/Installation.md ML-Agents Github]
 +
 
 +
=== Setting up the Python API ===
 +
 
 +
''pip install mlagents''
-
[https://github.com/Unity-Technologies/ml-agents Unity ML-Agents Github Page]
 
-
[https://docs.unity3d.com/Manual/index.html Unity Documentation]
 
== Using Unity ML-Agents ==  
== Using Unity ML-Agents ==  
-
** Build a standalone Unity Executable
+
=== Running ML-Agents through the Python API ===
-
You can run a Unity build (executable) from the Unity ML agents python scripts where this build is a standalone version of the game (robot simulation). You will have to create this build from the Unity editor using by adding an active scene (the scene that contains the ML agent) to the Scenes in build. It is explained in more detail here.
+
 +
You can interact with Unity ML-Agents by [1] opening Unity through python (''env = UnityEnvironment(side_channels=[])''), [2] resetting the environment (''env.reset()''), [3] stepping the environment forward until a new input from python is required (''env.step()''), [4] getting the state of the agent (with any sensory information; ''env.get_steps(behavior_name: str)'') and [5] setting the actions for agents (''env.set_actions(behavior_name: str, action: ActionTuple)''). For more information, please consult the guides below.
-
3. Using the python scripts
+
[https://github.com/Unity-Technologies/ml-agents/blob/main/docs/Python-API.md Python API]
-
The 4 python scripts included in this repository are the bare minimum of what you would need to run a standard evolutionary algorithm. To get started, make sure to make a virtual environment and install the python packages from the requirements.txt. You can find more information on how to do this here
+
-
3.1 UnityMLInterface
+
[https://github.com/Unity-Technologies/ml-agents/blob/main/docs/Python-API-Documentation.md Python API Documentation]
-
The Unity ML agents script contains code that allows you to open a Unity executable of your project and send motor commands to the environment. Note that you will have to change the PATH variable to point to your standalone build of the robot simulation.
+
-
3.2 EvolutionaryAlgorithm
+
== Documentation ==
-
The Evolutionary Algorithm script contains a minimal example of how you can Distributed Evolutionary Algorithms in Python (DEAP). You will have to pass references of the environment, the evaluation function, the individual, and the controller to the evolutionary algorithm.
+
-
ea = EvolutionaryAlgorithm.EvolutionaryAlgorithm(env, evaluate,EvolutionaryAlgorithm.Individual, SmallWorldNeuralNetwork.SmallWorldNeuralNetwork)
+
[https://github.com/Unity-Technologies/ml-agents Unity ML-Agents Github Page]
-
3.3 SmallWorldNeuralNetwork
+
[https://docs.unity3d.com/Manual/index.html Unity Documentation]
-
The SmallWorldNeuralNetwork is a simple type of neural network that can be evolved using the evolutionary algorithm. The network serves as an example but have a look at it to hack in your own type of controller.
+
-
3.4 EvolvingUnityMLAgents
+
== Troubleshooting ==
-
The EvolvingUnityMLAgents script combines the above implementations to evolve the behavior of the robot. This example runs on a single thread and will have to be turned into a multithreaded solution for your experiments.
+
-
3.5 Unity ODRI package
+
Although it is in principle quite easy to set up a Unity ML-Agents environment, in practice there can be a few tricky bugs either with Unity failing to start, version incompatibilities, or dependencies. If you're having specific issues with Unity and/or ML-Agents, for now, contact Frank.
-
The Open Dynamic Robot Initiative Quadruped package can be imported into Unity by going to Assets->Import Package->Custom Package. It requires ML agents version 1.0.7.
+
 
 +
=== Getting rid of too many log messages in the terminal ===
 +
 
 +
Depending on your experiments, it can be the case that you'll see that Unity is sending a bit too many messages. To limit the number of log messages, go to ''Project Settings->Player->Other Settings->Stack Trace'' and adjust the settings accordingly.
== Notes on deterministic behavior ==
== Notes on deterministic behavior ==
 +
 +
To ensure determinism, it seems that the physics of a scene needs to be reset. This can be done by loading a scene whenever you are evaluating something. Note that when coupling Python and Unity in the ''Editor Mode'', experiments will not be deterministic. You will have to build and run an executable of your project to get deterministic behavior. One can subscribe a function to the ''Academy.Instance.OnEnvironmentReset'' event in Unity that loads the scene containing the robot. You can't just reset the scene then you will lose the connection between python and ml-agents. You then have to create a "singleton" that is not reset when the scene is reloaded. The idea is to have a "RobotManager" with a single tone that keeps on and registers all the side channels. Robotmanager does very little other than maintain the connection, in another script you can have the agent script as usual.
 +
PS: Since you reload the scene, it will also create a "new" agent, it behaves the same but it will get a different id recommended way to retrieve the observations:
 +
    decisionSteps,other = self.env.get_steps(individual_name)
 +
    obs = decisionSteps.obs # <---- Beste måten å gjøre det på når det er en agent
 +
    end_position = obs[0][0][:3] # Henter observasjonene til agent 0
 +
 +
How to declare a singelton:
 +
    public class RobotManager : MonoBehaviour
 +
    {
 +
        // singleton
 +
        private static RobotManager instance;
 +
        public static RobotManager Instance { get { return instance; } }
 +
    }
 +
 +
[https://docs.nvidia.com/gameworks/content/gameworkslibrary/physx/guide/Manual/BestPractices.html#determinism  On PhysX determinism]
[[Unity-Timing-David | David's tips on getting deterministic timing]]
[[Unity-Timing-David | David's tips on getting deterministic timing]]

Current revision as of 16:52, 20 February 2024

Contents

Installation

Download and install Unity

Official Unity Website

Installing Unity ML-Agents

In the Unity editor, go to window->package manager. Inside the package manager search for ML agents in the Unity Registry. Install the most recent ML Agents package. ML-Agents examples can be downloaded from the ML-Agents Github

Setting up the Python API

pip install mlagents


Using Unity ML-Agents

Running ML-Agents through the Python API

You can interact with Unity ML-Agents by [1] opening Unity through python (env = UnityEnvironment(side_channels=[])), [2] resetting the environment (env.reset()), [3] stepping the environment forward until a new input from python is required (env.step()), [4] getting the state of the agent (with any sensory information; env.get_steps(behavior_name: str)) and [5] setting the actions for agents (env.set_actions(behavior_name: str, action: ActionTuple)). For more information, please consult the guides below.

Python API

Python API Documentation

Documentation

Unity ML-Agents Github Page Unity Documentation

Troubleshooting

Although it is in principle quite easy to set up a Unity ML-Agents environment, in practice there can be a few tricky bugs either with Unity failing to start, version incompatibilities, or dependencies. If you're having specific issues with Unity and/or ML-Agents, for now, contact Frank.

Getting rid of too many log messages in the terminal

Depending on your experiments, it can be the case that you'll see that Unity is sending a bit too many messages. To limit the number of log messages, go to Project Settings->Player->Other Settings->Stack Trace and adjust the settings accordingly.

Notes on deterministic behavior

To ensure determinism, it seems that the physics of a scene needs to be reset. This can be done by loading a scene whenever you are evaluating something. Note that when coupling Python and Unity in the Editor Mode, experiments will not be deterministic. You will have to build and run an executable of your project to get deterministic behavior. One can subscribe a function to the Academy.Instance.OnEnvironmentReset event in Unity that loads the scene containing the robot. You can't just reset the scene then you will lose the connection between python and ml-agents. You then have to create a "singleton" that is not reset when the scene is reloaded. The idea is to have a "RobotManager" with a single tone that keeps on and registers all the side channels. Robotmanager does very little other than maintain the connection, in another script you can have the agent script as usual. PS: Since you reload the scene, it will also create a "new" agent, it behaves the same but it will get a different id recommended way to retrieve the observations: 

   decisionSteps,other = self.env.get_steps(individual_name)
   obs = decisionSteps.obs # <---- Beste måten å gjøre det på når det er en agent
   end_position = obs[0][0][:3] # Henter observasjonene til agent 0

How to declare a singelton: 

   public class RobotManager : MonoBehaviour
   {
       // singleton
       private static RobotManager instance;
       public static RobotManager Instance { get { return instance; } }
   }

On PhysX determinism

David's tips on getting deterministic timing

Retrieved from "https://robin.wiki.ifi.uio.no/Unity"
Personal tools
Front page