Unity3D 5.6 video player and UWP

Unity3D just released version 5.6 with brand new video importer and player that I’ve been waiting for a while. In a few years we’ll be able to say “long gone are the days when you needed to install quicktime and brew an entire day’s worth of coffee to import a 360p 30 second video inside Unity”, and I find this beautiful.

Jokes aside, this means we have a lot more support and control over our video files, and dayum, that player’s good. It’s straightforward, it just works, and it still has a lot of options for the best experience. I was effectively surprised to see that video streaming worked out of the box without requiring $200+ plugins with zero support. (it’s not perfect though, we’ll come to that later)

One of the main features I was waiting for was this streaming from local or web addresses, explicitely for the case where you can’t package all your videos inside your project. I shit you not, I did a youtube video player in about 10 lines of code, and it was really fun. Internet streaming is a go, whatever the platform, it’ll just work. Local streaming is a different story. For starters, mobile operating systems’ permissions. Honestly, it’s a good thing we have them, you don’t want any app modifying your storage space on its own. But honestly, the permissions system on UWP (Windows 10 store apps, and Hololens too (and W10M, but who cares)) is b-r-o-k-e-n. I mean, okay, sandboxing complexifies storage operations, but you still have the right to offer an effective API, which UWP never has, in my humble opinion.

One of the problems I met while testing the new Unity3D video player was with these permissions, on the simple action of reading a local video file that’s not in the project, neither in the app’s local state (where everything is permitted and nothing is true). Let’s say it’s in your video library, you have a couple of things to do before you can read your video :

  • add the “Videos Library” capability to your app
  • put said video file in the videos library (it’s harder than you think on a Hololens)

Innocently, you’re going to want to load your file like this because that’s how the video player works with streaming :

var file = await KnownFolders.VideosLibrary.GetFileAsync("video.mp4");
var path = file.Path;
// next lines must be inside the main thread, somehow
// we'll see later, this is just an example
var player = GetComponent<VideoPlayer>();
player.url = "file:///" + path;
player.prepareCompleted += Player_prepareCompleted;
player.Prepare();

That’s where you realize that it doesn’t work, as usually, you’re going to get something like Access Denied while trying to run this. The reason is that the video player prepare must be (I actually have no idea) native and it tends to fuck up permissions. So you’ve got to grant it some special permissions with a special class I haven’t heard much about, but that’ll save your life on UWP : Windows.Storage.AccessCache.StorageApplicationPermissions and more importantly Windows.Storage.AccessCache.StorageApplicationPermissions.FutureAccessList. What this does is it creates a token that acts like a shortcut inside your app context for easier file access. I honestly don’t think it was meant to be used in this particular context, but hey! for lack of a better way, this works. So the idea is that the path you will give to the video player is the token, not the direct path on your drive. Here’s a full example behavior of something that loads a video from your video library :

public class LoadVideo : MonoBehaviour {

    // use this for resync
    Queue<IEnumerator> coroutines = new Queue<IEnumerator>();

    void Start () {
        // we need async because UWP loves async
        // to a point it gets old and boring
        GetFiles();
    }

    async void GetFiles()
    {
        // get your file
        var file = await KnownFolders.VideosLibrary.GetFileAsync("video.mp4");

        // generate a token
        var path = StorageApplicationPermissions.FutureAccessList.Add(file);

        // resync through a coroutine
        coroutines.Enqueue(LoadFile(path));
    }

    IEnumerator LoadFile(string path)
    {
        yield return new WaitForEndOfFrame();
        var player = GetComponent<VideoPlayer>();
        // player still needs a protocol to know where to look
        // here, "file:///" with three '/' because Microsoft
        // seems determined to not do anything like the others
        player.url = "file:///" + path;
        player.prepareCompleted += Player_prepareCompleted;
        player.Prepare();
    }

    private void Player_prepareCompleted(VideoPlayer source)
    {
        // we've got our video, it's loaded in memory, let's play!
        source.Play();
    }

    // Update is called once per frame
    void Update ()
    {
        // this is kind of lame, but quickest hack in the book to get
        // things working when UWP fucks up the main thread
        if (coroutines.Count > 0)
        {
            var action = coroutines.Dequeue();
            StartCoroutine(action);
        }
    }
}

With that little bit of code (and correctly set permissions in your appxmanifest), you get to play 4k hw accelerated videos from your hard drive on your lovely UWP app. Hurray!

This brings me to a final note : where. is. the. documentation? For real though, at the time of writing of this article, 5.6 is RTM and we’re using it on production, but the video player documentation is still a draft gdoc lost behind a link in the official doc. WTF? I don’t mean to criticize Unity’s workflow, and I’ll admit there’s not a lot to explain, but damn, documentation is not something you should forget for a full release on a commercial product. That’ll be all.

Have fun, code safe

Tuxic

Fuck This Jam post mortem : Strategy.min

This week (May 31st – June 6th) happened the Fuck This Jam online gamejam, hosted and created by the awesome dudes over at Vlambeer (Super Crate Box, Luftrauser, Ridiculous Fishing, Nuclear Throne, the awesomeness just goes on…). The point was to create a game in a genre you hate or don’t know much about and put it on itch.io. Since I already had 2 or 3 projects already running at work, I decided now was the right moment to get back into gamedev again. Fuck logic.

So yeah, I made a “RTS” game. I don’t particularly hate RTS, but it just so happens that I’ve never developed one before, mostly because my mentors where like “this is so out of fashion, facebook city builders is the shite”. It had occured to me to actually do a Clash of Clans clone or some stuff like that since I hate these kind of games. I ended up doing things a whole lot differently : a sort of defend your base against a brainless AI RTS with fixed camera, minimal graphics and no resource collection. Thus was born Strategy.min. But as a matter of fact, things went pretty wrong this week, because it seems I cannot lead 20 lives at the same time.


So, what went right:

  • I actually released, which is a thing
  • I had rather solid gamedesign, but I’ll talk more about it later
  • Architecture was really fine, I pushed Unity and Mono’s compiler to its limits
  • I only ever used one outside asset : Share Tech Mono from Google fonts
  • I suck at graphics design, and I ain’t even mad after my ugly units and buildings
  • What I could do with the time I had was rather neat
  • Unity export (y)

What went wrong:

  • Even though I has solid GD, the current released version has almost 3/4 cut out because I didn’t have the time to do it properly. In the intended GD, I had bases, player constructions, research/upgrades and even a basic AI that was totally doable. Problem is I really didn’t get the time for that (see “what went horribly wrong”)
  • Mouse management was actually a pain in the ass because my system is not flexible enough
  • Architecture was fine until I realized I didn’t have the time to do everything. Then things turned ugly. Working in a prototype specialized company surely helps.
  • Flight patterns for planes is rather weird and collision detection is absent
  • Didn’t get time to do sfx and music :'(
  • YOLO : didn’t do a single code versioning on this project

What went horribly wrong:

TIME MANAGEMENT. God I suck at this. For starters, I lost all the week end figuring out what I wanted to do, and I ended up doing game design at 1am on monday morning. Since I have a job during daytime, I had mostly reserved my evenings for code and beer. Things went right on the first night, but later on, there where spontaneous “heroic fantasy tv series” or “wassup dude, wanna hang out”. The point is, when I could have had a shitton of time to dev, I actually spent like 8 to 10 hours on the game only.

Notes to future self:

  • You suck at time management
  • stop wasting time on pointless details when you’re already short on it
  • pizza, you’re eating too much of it

Well, things never go according to plan anyways. I’ll take the time to finish it, I really enjoyed working on it. Thanks to Vlambeer for putting up that event, it feels great to do games again. Download the game for free on Itch.io :

Have fun, code safe,

Tuxic

Unity2d entities in a 3d world, why you shouldn’t do this

Sometimes I have really stupid ideas and a bit of time to waste. The base architecture for my st’hack challenge is an example of that, how I can really push a bad idea pretty far.

As you probably know (I like this formula, I use it a lot), a few months ago, Unity released its native 2d layer, with tons of new components, and a fully integrated workflow with existing 3d tools. That actually caught my interest : can you do a 3d game with the new 2d entities? The st’hack project was the perfect sandbox for this test, as I was into random dungeon generation. So, with my tileset and some Sprite game objects, can I generate a full 3d dungeon. The point is that the Sprite workflow is really simple, and it’s basically a drag and drop to create a 3d transformable entity. If you’ve checked out my executable, you’ll realize that there are no height levels, the whole dungeon is flat. It is still rendered in 3d with Sprite game objects, and as you might have seen, it’s awfully buggy.

Let’s breakdown the problems:

  • sometimes, the dungeon won’t generate. This is actually fairly easy to understand and the base limit to my idea : Unity2d entities, even though 3d objects, limit themselves to 2 axis, X and Y. What happens, at random, is that at object generation, something resets the Z axis which I use for depth placement. This means that every gameobject gets positioned on the 0 value of the Z axis, and since the player starts around [-16,0,-16], well… you can’t see any dungeon (duh)
  • the Sprite class draws on call order, and since the Z axis is not taken in account, there is no notion of depth testing. Meaning that if a Sprite gameobject behind another is drawn after the latter, it gets drawn in front of it. Overlapsing and shit, and this call order is pretty random on the same draw layer. You can in fact organize them with the draw layer feature, but this is totally wrong when it comes to 3d projection, as this needs to be totally dynamic when you modify the view model.
  • shaders. I can write a lot on this, but it actually is a noticeable problem with base 2d workflow in Unity. There is no dynamic lighting notion for Sprite objects by default. No matter what type of light or render type you choose, it won’t do anything on a Sprite object. This can of course be bypassed by changing the draw shader to a manually created diffuse sprite shader (create a new material and select this shader, affect it as material to Sprite Renderer). This should be enough if you remain into a 2d projection, but when it comes to 3d, you get the previous problems plus a shitload of math what the fucks from the Sprite class.

So yeah, it’s pretty impractical. And definitely a bad idea. Don’t do this at home kids.

So, what’s next? I plan on redoing this demo properly, aka the hard way : create true 3d objects with textures instead of sprites, reuse my algorithm, release this open source. And everyone will be happy, and the world will keep turning.

Have fun, code safe.

Tuxic

#sthack : Unity2D, dllimport and Monty Python

On the 14th of march 2014 went on the 4th edition of St’hack, a computer security event in Bordeaux that consists of a whole day of security conferences followed up by a Capture The Flag contest, organized by our very own security guru @agixid. For the 4th year, I’ve been part of the staff, taking care of CTF support at first, then UI design last year. I had the opportunity to create a challenge this year (and hand out pizza and redbull).

Those who know me also know that I really suck at computer security. Idiots would be like “that’s normal, you use MS tech”, but I would be like “I fell more into video games than security these past years”. Yeah, I use Windows as my main OS, deal with it. So I thought about a game concept the powners would have to reverse to find the solution. Here’s what I came up with : a basic hack n’ slash in which the hero has to move from chamber to chamber in the correct direction (much like I don’t remember which zelda) in a limited amount of room changes. If the hero changes too many rooms without finding the correct path, he loses. He can also get killed by cheaty ogres and the level generation is rather buggy (I’ll talk about that in another blog post). There is also no information whatsoever, you are on your own.

DungeonOfLolwat

So, how do you win? I actually made the game with Unity and used a feature only available for pro versions (I think) : dllimport. The feature itself is something available with C# since the dawn of ages, but Unity has decided you have to pay to use it in your games. That appart, it requires you to understand a bit on how .Net (and Mono + Unity by extension) compile their dynamic libraries : it’s managed code, executed on a virtual machine and totally controlled by it. It means that when you compile your C#, VB.Net or managed c++ in Visual Studio, you create an intermediate language (IL) version of your code that the VM will compile on the run so that the CPU understands it. This is called Just In Time (JIT) compiler and is a rather complex and nasty business. Check out this Telerik blog post if you’d like to know more. As a sidenote, .Net is not the only tech using JIT compiling, it’s rather common these days, so haters go home (unless you just hate JIT, which I can understand).

Back to business : IL code means easy decompiling. And when I say easy, I mean that you can get like 99% of your base .Net code (excluding naming if it is obfuscated of course) in a single click in an app like ILSpy. That’s the same with Mono and Unity, at least for Windows (haven’t checked on other platforms, but it shouldn’t be that different, I suppose). You get a bunch of dlls when you build your game, in particular Assembly-CSharp.dll in your Data/Managed. This includes all your non editor/plugins custom classes, which means “hide yo code, hide yo data, there’s a library intruder”. If you open this dll with ILSpy, you’ll probably find all your code barely touched by Unity (they optimize stuff like loops on build time). From this point, if you want to hide something, there are two solutions:

  • hide your data inside gameobjects metadata : this gets compiled into their .assets files at build time and seems like a freaking mess to open. If your data is critical, this is probably where you want to put it.
  • hide your methods inside a non managed library and call it via dllimport : this gives another layer of protection to your code, even though it’s not impermeable. We’ll use this.

So the goal is : create a non managed dll that stocks the flag the powners have to find about, call it in our managed code and generate path in game with that data. Want to check out the challenge? Download it here (Windows only, sorry)! Let’s do a step by step on how to solve this:

  • Find out this is a Unity game : I’m handing out solution here, but during the ctf, it was not specified. There were a few hints though:
    • the exe icon is the default unity icon
    • data/Managed/ has UnityEngine.dll
    • data/Resources/ files have unity in their name
  • Open Data/Managed/Assembly-CSharp.dll with ILSpy.
  • Find the class code for HeroMove, decompile it and get the lines of the dllimport.
  • Find YeOldeContent.dll in data/Plugins/. You can try to open it in ILSpy, but you’re going to be insulted.
  • Two options here:
    • the dev way : develop a .Net app that dllimports the non managed library and get the flag
    • the hacker way : extract the static strings directly from the non managed library

Now there’s a hiccup : the flag is incomplete, you have to interpret it ingame to get the complete flag (cf. directions). There’s also another hiccup if you use the dev way : I’ve put an extra security on the dll function that requires a passkey to get the flag directly. This passkey is inside the metadata, so if you want to go hardcore, try to open the .asset files and get that passkey. Else, you’ll have to replay a famous scene from Monty Python’s Holy Grail.

That’s it, you can witness my noobitude in computer security. Anyways, it was fun to create and I’m glad that some powners actually were able to break it! Regarding game part, I’m going to have to redo it, because it’s really fucking buggy. I tried to transpose Unity2d components in a full 3d world, and it didn’t go well. I’ll make an article about that later, but it was a pretty stupid idea. I used the Dungeon tileset over at OpenGameArt on a sidenote. Really awesome art!

That being said, have fun, code safe.

Tuxic

Unity : metronome like a pro

I’ll be honest : after yesterday’s rant, I really thought about if I should sell you this code through the asset store or not. But hey, I’m not such a bitch, the trick’s pretty simple.

So here’s a nice way to get an acceptably precise metronome, with custom BPM and signature. The purpose is to create a MonoBehavior that you can stick to an entity to count your beats. Let’s start by creating a new C# script (javascript should be straight forward, but I don’t use it, sorry). I named it metronome, because that’s what it is. We’ll add in a few fields that will make sense soon enough :

public int Base;
public int Step;
public float BPM;
public int CurrentStep = 1;
public int CurrentMeasure;

private float interval;
private float nextTime;

First 3 fields should be straightforward if you know a little bit of music theory : signature represented by its base and step amount, and Beats Per Minute. CurrentStep and CurrentMeasure just let us keep track of what step/measure we’re on. Now this is where the trick starts in order to be as precise as possible : interval is the absolute amount of seconds there should be between 2 beats. nextTime is the relative moment of when the beat will actually occur.

I see the guys in the back of the room going all “WTF man”, but it’ll make sense in a second : we are going to use Unity Coroutines. Unity uses Mono, which supports a great load of C# thread operations, including Tasks. The problem is, Unity is not thread safe, and tends to go all “NOPENOPENOPE” when you use threads and Unity objects. This is were the coroutines come in : they are a sort of bastard “multitask” technique that consist in watering down heavy operations between frames. That’s the important word : frame. Basically, a coroutine is a yield return method that disperses its executions on a frame to frame basis. On its simplest form, it works much like a simple component update, as launched coroutines are each called by default once per frame until they are over. The interesting part is the kind of value you can send to your yield return.

UnityGems did a great article on coroutines, and this is what you can send as a return value (shameless copy from their article) :

  • null – the coroutine executes the next time that it is eligible
  • WaitForEndOfFrame – the coroutine executes on the frame, after all of the rendering and GUI is complete
  • WaitForFixedUpdate – causes this coroutine to execute at the next physics step, after all physics is calculated
  • WaitForSeconds – causes the coroutine not to execute for a given game time period
  • WWW – waits for a web request to complete (resumes as if WaitForSeconds or null)
  • Another coroutine – in which case the new coroutine will run to completion before the yielder is resumed

I highlighted the one we are going to enjoy very much : WaitForSeconds. It’s straightforward, give it an amount of time and it will execute when that time is consumed. Let’s write a first coroutine with that!

    IEnumerator DoTick() // yield methods return IEnumerator
    {
        for (; ; )
        {
            Debug.Log("bop");
            // do something with this beat
            yield return new WaitForSeconds(interval); // wait interval seconds before next beat
            CurrentStep++;
            if (CurrentStep > Step)
            {
                CurrentStep = 1;
                CurrentMeasure++;
            }
        }
    }

Simple coroutine : infinite loop that increments CurrentStep and CurrentMeasure. It works pretty fine, but the more discerning readers will have noticed that we never set interval. I’m going to do a simple public method for that, to be able to reset and change my coroutine :

    public void StartMetronome()
    {
        StopCoroutine("DoTick"); // stop any existing coroutine of the metronome
        CurrentStep = 1; // start at first step of new measure
        var multiplier = Base / 4f; // base time division in music is the quarter note, which is signature base 4, so we get a multiplier based on that
        var tmpInterval = 60f / BPM; // this is a basic inverse proportion operation where 60BPM at signature base 4 is 1 second/beat so x BPM is ((60 * 1 ) / x) seconds/beat
        interval = tmpInterval / multiplier; // final interval is modified by multiplier
        StartCoroutine("DoTick"); // start the fun
    }

This goes a bit more into music theory, but I suppose you can deal with that if you’ve read this far. So we get the absolute interval between each beat and store it in our field. You’ll notice I use StartCoroutine and StopCoroutine with a string of the coroutine method name. This method is more expensive but allows us to stop the coroutine at will, which is appreciable. You can call StartMetronome() in your Start(), create an entity and attach the script as a component, and set for example Base and Step to 4, BPM to 120 and launch. In your debug log, you’ll have a nice timed “bop” appearing in an endless loop. Mission accomplished.

Wait, we still have something to fix : on usage you’ll realize that this is precise but not enough. It tends to desynchronize pretty fast (at 120bpm, it’s be off tracks in less than 4 measures) and that’s bad if for instance you’re making a musical game. The reason is simple : coroutines are balanced by frames, and frames have a delta that you don’t really control. The problem is that your interval is fixed, but the WaitForSeconds might just decide that it’s too late to execute at this frame, let’s wait another one or two. Thus the wibbly wobbly bullshit the metronome outputs. This is where nextTime comes in. The purpose is to resync the metronome with effective time scales. The wait interval will thus never be constant. Let’s modify our methods :

    public void StartMetronome()
    {
        StopCoroutine("DoTick");
        CurrentStep = 1;
        var multiplier = Base / 4f;
        var tmpInterval = 60f / BPM;
        interval = tmpInterval / multiplier;
        nextTime = Time.time; // set the relative time to now
        StartCoroutine("DoTick");
    }

    IEnumerator DoTick() // yield methods return IEnumerator
    {
        for (; ; )
        {
            Debug.Log("bop");
            // do something with this beat
            nextTime += interval; // add interval to our relative time
            yield return new WaitForSeconds(nextTime - Time.time); // wait for the difference delta between now and expected next time of hit
            CurrentStep++;
            if (CurrentStep > Step)
            {
                CurrentStep = 1;
                CurrentMeasure++;
            }
        }
    }

This very simple trick allows you to fix this sync problem, as the wait delta will be fixed depending on actual time and expected time. Of course, this is far from perfect, but it does the trick : beats are really precise, even after 20 minutes of play. The rest is up to you. I decided to implement events on tick and on new measure, and you can find my code sample on this gist. Output is as follows with a visual thingy:

Anyways, have fun, code safe.

Tuxic

Unity2D : hail to the king, baby!

So yeah, like I previously stated on the blog, I’ve been testing Unity2D a lot these past days (and a bunch of other stuff, as always).

Overall impression is “GUDDAMITFINALY!1!!!11!”. And Unity delivers. So in the past I wasn’t the first Unity3D supporter, on the main purpose that 3D is not a subject to be treaded lightly, and a hell load of noobs and idiots would start making stuff with the free version. Eventually I got to work a lot with it this year and changed my mind concerning its raw power. I mean, just look at the time I lost on Kerbal Space Program, and this marvel of procrastination is being made with Unity. And workflow is really great, their Entity Component system is awesome. Always had problems dealing with Monodev, but as a MS tech dude, I redirected every single code line to VS. And I frantically never touched a line of Javascript. Still have to do some progress on my problems with this language, and Unity’s implementation is just… weird.

One of the things I’ve always regretted with Unity was the absence of real 2D support. It was until now a 3D tool, and you’d have to hack it to get things to show up in 2D. And the fucking asset store taking advantage of that : no sir, I’m not paying 60 bucks to show sprites on a free game making tool. So being limited to 3D is kind of drastic. Like I said, 3D is hard, people expect so much from it, you can’t just go around making 3D stuff without having previous experiences. Stuff like shaders, LOD and animations just give the creep to any dev that knows, and despite Unity3D being a very well conceived tool, you won’t escape these matters. And god I hate the asset store. This thing is evil. This thing should be held responsible for all the crap shitty mobile devs create. I’m exagerating of course, I just have a problem with these guys saying “oh, Unity doesn’t do that? Let’s go waste $150 on a plugin/asset/code library!” when it’s something as simple as post process shaders or block level design.

Alright, let’s stop ranting for a second.

To sum it up, Unity2D is Unity3D with one dimension less. This means you’ve got SpriteRenderer instead of MeshRenderer, that you’ve got 2D prefabs, 2D physics, 2D post process, 2D goodness! It’s freaking awesome, seriously. I’ve never tried other 2D alternatives like GameMaker, but I suppose they offer the same type of pleasure Unity2D sends your way. Drag&drop your sprites, add 2 or 3 entities, and you’ve got yourself a working prototype. The fabulous idea to natively support spritesheets and atlas textures is just enormous and fantastic, and the animation workflow is fan-tas-tic. Going back to physics, I was really impressed they were able to keep almost all of their 3D conventions, with the ease and simplicity it procured. Of course, basic primitive shapes like rectangles and ellipses are available, but the sprite based collider is art. Creating custom colliders has never been this simple! The greatest thing of this all will remain their 3D/2D mixing : a 2D scene is just an orthographic camera with a bunch of 2D components, but still in a 3D scene. The point is, you can easily merge 3D elements into a 2D scene, and the contrary too, because nothing stops you from using a SpriteRenderer in a 3D Scene.

So to try it out, I recreated the core gameplay of my SpaceShooterz game, and this took me.. at most something like half an hour : background parallax, player spritesheet, enemy + basic move routine, shooting and killing. 30 freaking minutes, and not even trying hard.

So this is great news for indie devs. We get an awesome tool for free and the raw power to create very complex 2D gameplays in no time. Since Unity exposes free exports to iOS and Android now, including the long-awaited Windows Phone and Windows 8, you devs have no excuse! Go make some games! This comes at a cost of course : Unity is heavy. In the last year only, the package went from 400megs to a whole freaking gig. The interface also is a marvelous work of the kraken, but you can get over that. I would recommend a computer with good perfs and a big screen though, your eyes will thank you.

Anyways, have fun!

Tuxic

Unity 2D in progress

Been testing out Unity3D’s new 2D features on the 4.3 update, I’ve got to say I’m quite impressed by the simplicity of the overall thing. Blogging on it really soon!

In the meantime, have a dogemon

Dogemon