Here I will present the solution that works for us. Big thanks to all the helpers mentioned and not mentioned below, all the software developers that created the great tools, and all the others who made this possible.
Our Requirements
At work we want to automatically create builds of our Unity3D applications. Currently we use the continuous integration (CI) middleware Atlassian Bamboo. But the choice of the CI tool is up to you, you could also use Jenkins, Buildbot, TeamCity, or something else. Same with any other tool mentioned here, just use what you like best, or what is best in your context.
Here, Bamboo executes a NAnt task that calls Unity3D with some command line arguments to create a Windows Standalone Player build. Just like this
Bamboo -> NAnt -> Unity3d
Problems
We didn’t want to install DirectX on a Windows server machine and we discovered the Unity command line option -nographics that allows you to build on a headless server. Unfortunately creating a build through Unity3D with the -nographics command line option, not only disables the graphics requirements on the system running the Unity Editor creating the build, but also disables any visuals in the resulting Standalone Player. A black screen is all you will get…
Because we want fully functional builds with graphics, this is not acceptable for our use case. Hence, we decided we need to create the build without the -nographics option. As a consequence we installed DirectX and equipped the server machine with better graphics hardware. Soon we learned that Unity3D has no access to Direct3d9 when run in the context of a service (at that time Bamboo was running as a Windows service). Well, bummer.
But, as an alternative we can run Bamboo with a logged-in user over a Remote Desktop session. Of course it’s not as easy as it sounds at first: You have to stay logged-in into the Remote Desktop session (you may not even minimize the Remote Desktop window, unless you use this workaround, thanks to liortal53 for pointing out, although we did not use this nice trick). Basically this is not very practical. Fortunately there is a crazy workaround:
The Solution
We followed the tracks of a hero named Cygon. He/she deserves praise for asking and solving this problem before us. This made the solution public
http://www.gamedev.net/topic/522951-continuous-integration-and-xna/
http://forums.create.msdn.com/forums/p/24664/138642.aspx
That hero was guided by grandmaster timeBandit that gave the crucial tip at the gentoo forums
[1] http://forums.gentoo.org/viewtopic-p-5475029.html
This lead us to our solution:
We use VirtualBox which runs an instance of Ubuntu. Inside Ubuntu an Upstart script starts a vncserver which in turn runs rdesktop to create a Remote Desktop session to the Windows host server. Just like this
Windows Server 2008 R2 -> VirtualBox -> Ubuntu -> vncserver -> rdesktop -> Bamboo -> NAnt -> Unity3D
Running VirtualBox as a Windows Service
To run VirtualBox as a service at system start we use the free tool vboxctrl (link may occassionally be down, just retry). We did try different ways to setup custom Windows services, but this is not as easy as we thought and this tool made it easy for us. As described in the documentation you name the vboxctrl.exe with the name of the service you want to run, e.g. “Ubuntu rdesktop.exe” and you put the .ini file into the same directory with the similar name “Ubuntu rdesktop.ini”. Inside the .ini file you change the settings to your configuration. Here environment variables like %AAA% did not seem to work in the .ini file, but do setup the variables as indicated on the documentation of vboxctrl (the VirtualBox home, and the VirtualBox user config path). When everything is set up, your instance of Ubuntu will be started at server restarts. This gives us
Windows Server 2008 R2 -> VirtualBox -> Ubuntu
Unfortunately the tray icon tool of vboxctrl did not seem to work. But I don’t think we’ll need it because we should be able to start/stop individual virtual machines through the Windows Server Manager and the services configuration dialog.
Creating the persistent Remote Desktop Session
Inside Ubuntu you configure everything as indicated in [1]. We used a different user name and did not require /etc/conf.d/local.start and /etc/conf.d/local.stop. Instead we used an Upstart configuration that starts the vncserver. Also, in /home/user/.vnc/xstartup we provide the username and password of the Windows user to rdesktop (you have to setup a Windows user that is allowed to login per Remote Desktop and also has the permission to run Bamboo in the cmd console. Bamboo comes with a .bat for that). As always, it is never a good idea to provide plain text passwords in configuration files. Make sure your system is not accessible from the outside, and also not from the inside, by unauthorized users. The user you create for this setup should not have any administration rights. Also on Ubuntu’s side, don’t use an admin user. Create a regular user, e.g. with adduser on the command line.
If you need to test the rdesktop session from inside Ubuntu and you installed the server edition of Ubuntu, you can install xorg and fluxbox (via apt-get, aswell as other tools like vim that you might need). Run fluxbox with the command startx. Rightclick on the fluxbox desktop, navigate to open a terminal and enter your rdesktop command inside. But xorg and fluxbox are not required for this to work. Now we are here
Windows Server 2008 R2 -> VirtualBox -> Ubuntu -> vncserver -> rdesktop
Running Bamboo from the Desktop and not as a Service
Just login via Remote Desktop (from Ubuntu or from another Windows machine) as the user you just configured. In that users Windows Start menu, add the Bamboo console .bat into the Startup entries, so that it is started whenever that user logs on. Congrats! You did it
Windows Server 2008 R2 -> VirtualBox -> Ubuntu -> vncserver -> rdesktop -> Bamboo -> NAnt -> Unity3D
You should be able to start the VirtualBox service and the user you setup should log in automatically via Remote Desktop into your Windows server. As soon as this happens Bamboo should start as a process run by this user. You can inspect the users that are currently logged into the system in the users tab of the task manager application. And you can also check which user is running a process in the same program.
Unless I forgot any critical points, this should give you a nice working solution to automatically create builds with Unity3D.
Again, thanks to all who paved the way!