Improving AI Performance

Hi all, Steve here!  Here's a look into what I've been working on recently: 

There's been some monstrous performance spikes in our AI, and I've spent time over the last month working to mitigate this issue.  Two significant changes have been made to improve this:

1 - the addition of a pool of Dictionaries to store states, and
2 - making each state item immutable

This vastly reduces object creation, which is slow, and reduces the number of objects in existence, which helps reduce memory fragmentation (another Bad Thing) - which, being C#, is harder to control than in C++.

Dictionary Pool

So what's a Dictionary Pool, you may ask! Well, at its simplest (which this is), it's a big list of pre-made Dictionaries, each with a flag on to indicate if it's being used.

When we want a Dictionary, we grab one from the pool, put it in a list of 'in-use' Dictionaries, and mark it as being used. When the object using it is destroyed, we mark it as free, and put it back in the pool for someone else to use. The key here is to be very strict on who is using your object! It's critical that nobody is using it when it's marked as free, or the whole concept breaks down.

Immutability

And immutable states?  Immutability is another weird software-engineer-y word. In english, it means something that never changes. But that's perhaps misleading when it comes to thinking about objects in Unity - what use is an object you cannot alter?

The answer is that while you can't change the object, you can create a new one to reflect this change. This means you have precisely one instance of any particular combination of object.  For us, this means that instead of creating hundreds of thousands (no exaggeration) of states, all duplicates of one another, we create... about seventy. The overhead of querying a Dictionary for the instance of the state we want is much, much less than creating a tiny object, then destroying it, repeatedly.

These two changes weren't trivial, as such, but were fairly self contained, and only produced a handful of super frustrating bugs that I had to then work against for a week or two.

Back within 60fps, but still a lot of garbage to deal with!

But the good news is that performance improved drastically!  We are now back to 60fps within the editor on my machine, mostly. Which was a relief because I feared for a moment or two that I'd accidentally extended the AI system in such a way that it would never be useful in a finished game, which would have been embarrassing.

-Steve

If you haven’t already - be sure to wishlist Off Grid on Steam - each wishlist makes a big difference to us, and we really appreciate your support!

OFF GRID IS OVER 50% FUNDED IN 24 HOURS!

WOW.  You backers gave us a seriously wild ride yesterday:  35% funded in 5 hours, 40% funded in 7 hours, AND THEN we hit the 50% mark within 21 hours!   

It feels seriously good to be 50% funded within the first 24 hours of the campaign!  Your enthusiasm and support has blown us away.  THANK YOU!

For this first update, we wanted to give you a little background on the inspiration for Off Grid.  As you might be aware, it’s been in the works for quite some time! 

The game was initially inspired when Rich heard Eben Moglen speak about net neutrality and data privacy at the INET conference in 2011.  It struck a cord, and the seed for Off Grid was planted. 

Soon after, Rich teamed up with Pontus, and together they built the first prototype of Off Grid.  The goal was to get people thinking about data.  The root notion was to try and physicalize it -  to make data into something manipulable, something that people would be able to go and play with and see as a thing you can walk around.

But the concept was still a little out there:  this was early 2013, before Snowden and NSA revelations.  

Fast forward to June 2013, and the first ever public unveiling of Off Grid (in a livestreamed interview at Etoo London) just happened to fall nearly 64 years to the day after the publishing of George Orwell's book, 1984, and the same week that Edward Snowden made his first leak.

And all of a sudden, it clicked.

Prior to the first revelations, it was a struggle to explain to anyone why what we were doing with Off Grid was important or would be interesting.  Then, almost overnight, Snowden gave everyone the vocabulary to talk about it.  People understood.

Now, years on, the revelations have just kept rolling, and everyone is much more aware of the issues surrounding data privacy and mass surveillance.    

It is the perfect time for a game like Off Grid:  people are tuned into the effects of data mining and the impact that bad laws around digital rights have on real life. Help us get Off Grid funded, and THEN help us shape the game so we can all try to picture a less dystopian digital tomorrow.

Let’s keep the momentum going - tell your friends!!

THANK YOU! 

The Off Grid Team 

P.S. If you want to chat to us a bit more informally, we regularly hang out in our studio discord - we’d love to see you in there!  Join us:  https://discordapp.com/invite/NDEVwBJ

If you haven’t already - be sure to wishlist Off Grid on Steam - each wishlist makes a big difference to us, and we really appreciate your support!

Dev Blog Post 5.12.2017 - Social Engineering and Distracting Noises

We’ve been working with a bit smaller team for this sprint, so this is going to be slightly shorter blog post than usually. And we’ll switch the format slightly as well, so you’ll get a section from both of us telling about what we’ve been up to…

Steve:

My first month has flown by! It’s been great to get started and become acquainted with Unity, which has impressed me. I started off on Off Grid by poking around the existing code, and playing the demo level, which gives an good overview of where the project is. After that I was forced to do some real work; first of all, I exposed the ‘Noise’ system to Lua, meaning modders will be able to play sounds that the AI will investigate. I’ve also fiddled with the camera, with the Lua setup, the wiki, and a fair few under the hood changes that hopefully will help us create new content more quickly. Exciting times ahead!

Pontus:

Character profiling

We had already previously converted the text files we use to describe each character’s personalities and background information from old XML files into Lua. But as we'de designed some of our data mechanics, and especially the social engineering aspect of hacking and privacy bit further, we realised the profile format we had would not do the job.

So, in this sprint I’ve worked on re-designing the character profiles to use a tag-based system that allows us, and the modders, to easily add pretty much any amount of background information about the characters, and tag that data in a way that lets us then attach those tags to different data points, files and whatever inside the game, and hook it into the AI’s behaviours easily.

(and same as with our mission progression system and many other things, the design goals and explanation how it works ended being much more complicated than the actual implementation, so no need to worry, this is really simple system for modders to use and should be very extensible and flexible for all the crazy weird hacking stories and tools modders might come up with which we never even thought about…)

While doing the required changes for this, I also added few quality-of-life improvements, so the levels can now automatically load character profile files, and the image files used for character colour customization, both from the level’s own folders and from the Common folder used for sharing things that might be used in multiple levels. This means that we don’t need to duplicate character files to each separate level where the same character might appear, and also modders will have easy access to some pre-made characters they can use in their own levels without having to even bother with the actual character profile files until they want to create some new characters of their own.

Sending SMS, with Lua

We are then using those character profiles to generate text messages (and eventually e-mails, and other files as well) that get sent to all the characters, and that the player than intercept, read though, and use to learn about the characters (maybe to figure out how to distract a specific guard, or to help guessing someone’s password, and so on).

This is nothing new, the SMS system has been in place for long time already. But the source file we used for the SMS templates was the last remaining XML file in the game, so of course that had to be converted to Lua as well… If nothing else, it’s more consistent and easier for everyone to deal with the same language throughout the project, but it’s much more human-readable syntax as well.

To take things a bit further than that, we thought that maybe the actual code used to generate the messages could also be moved into Lua. This certainly isn’t something you’d want to mess around for every mod you make, as it’s a bit more complicated than the rest of our mission and character files, and the actual content you see in-game can be changed easily by just creating different character profiles anyway. But exposing that code to the modders might open some interesting opportunities, maybe for localizing the message generation to some language with different grammar than what English has, or to build something more complicated than what the base game needs to go with some awesome mod you are making. We’ll see what happens!

Oh, and of course I made sure both the template file and the actual message generator code can be loaded from both the Common folders as well as from the level-specific folders, just like with the character profiles.

Outro

That’s everything for now! We’re not quite sure yet if we’ll have a short sprint (and one more blog post this year) or if we should just make it a long one and return back to you in 2018! Either way, we’ll make sure to let you now when the next blog post is out!

Off Grid Sprint Update 16.8.2017 - Imma chargin ma lazer

Heyho! It’s been a long Sprint this one due to various galavanting and laser based activities, so apologies for the adjustment to our usual blogging schedule. The upside is - we have packed an incredible amount in to tell you about!!

Develop 

This sprint started with us heading down to Develop conference in Brighton as Harry was giving a talk on ‘Making a hacking game hackable’. The talk went incredibly well and generated lots of indepth questions from the audience.
We had the opportunity to meet up with all kinds of incredible folk and show them the game, including the ever lovely Dan Marshall who was extremely excited by the game and had these lovely things to say about us :D

That one is definitely going in olive wreathed award quotes!
Harry also found time to appear on Keir Miron (of Darkest Dungeon fame)’s podcast 'The Question Bus’ and was interviewed in a corridor at the conference! You can listen to it here.

SHA2017

We have been pushing a bunch of features forward in the game, especially within the modding toolset, and getting it ready for SHA2017, a hacker camp in the Netherlands that we were invited to.  Rich and Harry spoke and ran workshops focusing on how hackers and modders can use our modding to tools to create interesting hacks that reflect real life vulnerabilities.
The talk and workshops went really well - we learned a heck of a lot and we got a fair few people in using the tools.  You can watch the talk here and we’ll be putting up a full blog post on the whole experience in the coming weeks.

Steelcon

In the run up to this, Rich headed up to Steelcon again to gather inspiration (the SHA talk references how one of the first hacking mods we made in Off Grid was based on a Steelcon talk about hacking the Nissan Leaf electric car by Scott Helme last year). There was loads of interesting brain juice this year, including our friend Darren Martyn’s hilarious talk on Hacking ACS Servers for World Domination and this unnerving talk by Ken Munro from Pen Test Partners about the awful vulnerabilities they had found in IoT sex toys…

Mod hacking workshop with Spoonzy

On top of having gotten a good start on testing the modding tools with Dominic during his work experience last month, we took this a step further and got our mate Spoonzy to come in and apply some hacking knowledge to crafting some hackable devices with the modding tools. Spoonzy spent the day with us and came up with some pretty epic ideas for hacks - we made a start on a couple of them and he prompted a few fixes that we got to work on ahead of our workshops at SHA.

New Networking

This included Harry working on switching the networked tools that allow modders to build their own level changes directly into the game over to our own networking code (much faster iteration time!). This took a little wrangling, but as well as speeding up the workflow, it has put in a couple of checks that prevent potential loss of work. The old way used Unity’s built in TCP/IP networking code and made use of the editor’s play button in the modding project - if you have ever done any Unity development you will know that if you leave that on by mistake when working on objects or values in the scene then you will lose your work when you stop play mode on the editor - the new networking code avoids this pitfall entirely. 

Tools!

We wrote an export tool that can now generate clean versions of LevelKit for us to submit to our steam tools section for the game. Meaning that any half-baked dev content is stripped out and our modders have a nice clean project they can work within.

Spawning

We also found that folk wanted to know what direction characters would face when spawned and so now allow users to visualise players and guards spawn rotation with an arrow marker on the mission object.

Templates

We wanted modders to be able to just jump straight into mission and
hackable object scripting if they wanted to, rather than having to do
any geometry or environment work and so made a template scene so modders
can jump into a working mission with hackable devices already in it,
ready to be edited, scripted and added to. It is just a basic couple of
rooms with a few props and a basic laptop to hack into as an example but
it is a good start. The laptop in it even had the SHA wiki on as the front page :P

All your data belong to us

All networked devices now have a data inventory - this is the first step in making characters dynamically populate the devices in the world with their personal data. This way, when you hack a device you are able to get specific information on the NPCs around you who have interacted with that device, building up a piture of their behaviours and routes, and what data or other devices you might be able to manipulate them with.

No Rest for the Wicked

And one for the books… we fixed guards giving up patrolling after roughly 5 patrol points. We found that they had not been given a place to rest and recoup their motivation so they were essentially lazy guards striking due to lack of coffee!

Progress and Affect

Modders can now update the game progress via lua, which means conversations, objectives, or other triggers and interections in your level can affect another mission. For example, finding a hidden space in a level or making specific conversational choices in a set of CryptoChat messages can be used to unlock new levels and side missions within the wider game. Essentially branching narratives and their effects can be set off by any lua triggers in a level.
We also added an Error checking pass on conversation system (making them now more robust). It’ll now be easier to avoid lua compile errors and get your conversation scripts in the game and conversing properly.

Wiki

 

The other thing we have done is some fairly comprehensive work on getting the Off Grid Wiki to a stage where it is useful to players and modders. We now have a bunch of articles which describe how you can go about making mods for the game. We’ll update you soon once the structure of those articles is coming together!

Character customization

We are aiming to make all characters more customizeable for modders, and want to set up the colors in a way that would require as little as possible special skills or programs to change them for an existing character, while still using a single mesh & single material on each character in game.

Normal textures would do the job, but editing the colors on them afterwards isn’t really for everyone. Vertex colors are easier, but editing has to be done with a 3D modelling tool and creating different color version of a character requires creating a different model (or messing with the vertex color data on the fly through code). Our solution was using a simple texture as color look-up table, so once the characters are set up correctly (by us) all one needs to do to change them is open the texture file in any image editor (even MS Paint would work great for this) and paint the tiles with new colors.
Since we don’t need any detailed textures or anything, we can just assign the UV’s of the models to those colored tiles. And in theory we only need one pixel per color so the textures can be really small. Although to make things easier to work with and avoid any issues with color bleeding when the images are compressed, we settled for a 64x64 texture (the resolution doesn’t actually matter) with 8x8 grid of colored tiles. That’s 64 colors per model (with 8x8 pixels each), which should be more than enough for our art style…

In addition to the color table, we also added support for an additional texture that works the same way, but is used to set the glossiness of the material, and to switch between metallic & non-metallic material.
…and both these textures are of course fully optional, all the characters have default textures built-in to the game so if the modders leave one or both textures out, we’ll just use the defaults instead.

2nd Floor of the Apostle level

If anyone reading this has had a chance to play Off Grid, and made it through our newspaper office level, you might remember the 2nd floor of the building being pretty barren, to put it nicely. Properly decorating that part of the level and adding some actual gameplay has always been our plan, but there’s just always been more important things to do.
Well, this sprint we finally decided it’s time to add some content there. No spoilers, but let’s just say that you can’t just run through that floor any more, and trying to sneak past the guards would be pretty difficult as well. Instead you need to figure out a way to get the guards out of your way. There’s a few possible solutions, like you’d expect in a stealth game, and we thinks there’s room for some more as well.

Other changes & fixes

• Added a bunch of new sound events for different parts of the UI


• Text-only lines in the ncurses-like remote connection window are now correctly displayed using the foreground text color (as defined in the device’s Lua script)


• File viewer UI can now be closed with the “back” button (B button on gamepad, backspace on keyboard), and also closes automatically when the pause menu is closed.

Biz Rumblings

There has been a bunch of production and business development going
on in parallel to all this, loads of interesting folk spoken to about
opportunities for the game, but as always, you’ll have to wait to hear
more about that!

New *MOAR* blog posts

We have decided to shift the format here on the
devblog slightly, we are planning on continuuing with these monthly
sprint updates but maybe trying to make them a little more concise while also expanding on one or two of the items in each
blog post in between each sprint update, so watch this space! We are
going to start with Harry writing up a follow on from his Develop talk,
the kinds of things he touched on, and his impressions of the conference
as a whole, so look out for that!

Speak to you even sooner!
Rich, Pontus, Harry, and Sarah.

Dynamic building generator/tool

We started prototyping a dynamic building generator/tool for our city recently, and you know what, for an afternoon’s work its starting to make things that look damn-near like buildings!

In fig.1 you can see the test building that we created the tiles from in Blender, then in fig.2 you can see the ‘WallBuilder.cs’ generating a wall with those tiles randomly, but in fig.3 … wait for it … we have it specifying a ground floor, with ground floor tiles, and intermediate floors with intermediate floor tiles.Still a long way to go, but this use of the Unity Editor scripting options and generally component based scripting, should prove a powerful way of letting our production and interior designers come up with clever architecture patterns and rules and play around with generating buildings in the game engine. Freeing us to focus on EVERYTHING ELSE THAT NEEDS DOING, AI, UI, interactions, gameplay programming, the story… yada yada yada!

Let us know if you have any tips for tile based building tools! Look forward to showing you all more as it comes together!