Game tools

About the actual programming of the game.

Re: Game tools

Postby khodzha » Tue Aug 27, 2013 8:13 pm

And what kind of documents should server handle?
khodzha
 
Posts: 1
Joined: Fri Aug 23, 2013 5:00 am

Re: Game tools

Postby Andrew » Tue Aug 27, 2013 11:03 pm

Only thing I see which needs mentioning is some kind of locking mechanism. Especially with binary files, the last thing needed is two people trying to edit a copy of the same document.

I'd love to help out with this but unfortunately the conflict-of-interest clause with my employer will prevent me from doing so. It's got a lot of similarities with my day job though so I'm happy to do things like code review as well as critique the architecture when it becomes more fleshed out.
IRC: |Andy|
Andrew
 
Posts: 110
Joined: Wed Aug 21, 2013 7:16 am
Location: Melbourne, Australia

Re: Game tools

Postby Pseudonym » Thu Aug 29, 2013 6:25 am

Andrew wrote:Only thing I see which needs mentioning is some kind of locking mechanism. Especially with binary files, the last thing needed is two people trying to edit a copy of the same document.

Let me repeat what is on the Insomniac blog:

The DB/server runs locally on the machine of everyone who is using a tool. It is the "model" in model/view/controller. That's all. It avoids the need for implementing load, save, undo, redo etc for any tool we write, because it just has to be written once. There is no need for locking because only one person will be using it.

Checking in an asset is a different story. For that, the DB/server or tool can publish to git or something, and the locking can be handled there.
Pseudonym
 
Posts: 129
Joined: Tue Aug 13, 2013 3:54 am

Re: Game tools

Postby mrout » Thu Aug 29, 2013 6:29 am

Pseudonym wrote:
Andrew wrote:Only thing I see which needs mentioning is some kind of locking mechanism. Especially with binary files, the last thing needed is two people trying to edit a copy of the same document.

Let me repeat what is on the Insomniac blog:

The DB/server runs locally on the machine of everyone who is using a tool. It is the "model" in model/view/controller. That's all. It avoids the need for implementing load, save, undo, redo etc for any tool we write, because it just has to be written once. There is no need for locking because only one person will be using it.

Checking in an asset is a different story. For that, the DB/server or tool can publish to git or something, and the locking can be handled there.


Just one thing: binary files should never, ever, ever, ever, ever touch git.
mrout
 
Posts: 731
Joined: Mon Aug 12, 2013 10:49 pm

Re: Game tools

Postby Pseudonym » Thu Aug 29, 2013 7:17 am

mrout wrote:Just one thing: binary files should never, ever, ever, ever, ever touch git.

Yes. I'm specifically referring to assets which are representable in the editable JSON format (which may be compiled to binary for the engine's benefit).
Pseudonym
 
Posts: 129
Joined: Tue Aug 13, 2013 3:54 am

Re: Game tools

Postby wrongu » Tue Sep 17, 2013 7:01 pm

This is a really interesting system - I may need to try implementing it. Let's see if I understand the basic workflow (my notation has "+ ..." as user operations):

Code: Select all
+ start the local server
+ open local editor(s) of choice
       editor requests binary files and JSON metadata from server
+ make changes
       editor keeps track of only what changed; delta-JSON sent to server
           server updates changed files on disk
           server saves undo/redo data in database
+ undo a change
       request is sent to server
       server replies with "reverse" delta for the relevant session
       server updates its undo/redo stack (in the database?)
(background) the client/editor is polling the server for changes
       server sends back updated documents (or delta)
(background or user controlled??) sync with remote/master files
       Perforce or some other git tool performs the sync with a remote repo.
(background) the tracker notifies the server that files were changed on disk
       the server updates the database


And a few questions:
  • Would this system require the dev team to build a custom editor for each "component" (for lack of a better word..)? Or, how could it be integrated with existing editors (let's say Blender just because people seem to be referencing it a lot)?
  • Why can't git touch binary files - is it a problem of large storage space or the fact that "diff" is meaningless for them?
  • Might there be a lag between making a change and receiving the updated document(s) from the server?
  • in what ways are the files on disk different from the DB documents (apart from literal data formats)? Are they meant to mirror each other at all times?
wrongu
 
Posts: 46
Joined: Sun Sep 08, 2013 6:15 pm

Re: Game tools

Postby Pseudonym » Thu Sep 19, 2013 2:55 am

wrongu wrote: editor keeps track of only what changed; delta-JSON sent to server

The way that Insomniac implements it is that the whole JSON is sent to the server, which computes the delta and stores that. That way, the JSON diff only has to be written once.

Basically, the server is both the undo/redo stack and save/load/push to central repository. Editors just edit.

wrongu wrote:Would this system require the dev team to build a custom editor for each "component" (for lack of a better word..)?

Yes. However, those editors could be written in anything that speaks JSON and REST, so many of the editors could be implemented in HTML5/JavaScript. Part of the benefit of using this approach is that writing a simple editor should be easy and quick.

wrongu wrote:Why can't git touch binary files - is it a problem of large storage space or the fact that "diff" is meaningless for them?

The latter.

wrongu wrote:Might there be a lag between making a change and receiving the updated document(s) from the server?

If you mean the local server, I think that would be negligible for the most part.

wrongu wrote:in what ways are the files on disk different from the DB documents (apart from literal data formats)? Are they meant to mirror each other at all times?

First, a bit of history.

The distinction goes back to the first days of animation. There were two types of paper that were passed around the studio, not including maps to peoples' houses and caricatures of the boss: artwork (i.e. actual drawings and paintings) and control documents (i.e. anything which explains what they have to do with the artwork). You can loosely of the artwork as data and the control documents as metadata, but actually, the control documents are data too.

Here is an animator at work. I don't know who this is, but I'm guessing this is Disney, because his pegbars are at the bottom. (EDIT On second thoughts, it could be Otto Messmer.)

Image

The paper that he's drawing on is the actual artwork, and the piece of paper to his right is an exposure sheet (sometimes called a "dope sheet"). Exposure sheets typically look like this:

Image

The main part of the sheet is a table. There is one row per frame of film. Reading the columns form left to right, the leftmost one is the action broken down by frame with the frame number printed next to it, then the sound (this can be musical beats or phonemes for lipsync), then several layers of drawing (which will eventually be inked and painted onto cels), from top to bottom, with the background image (which is more typically a painting) being the last one. Finally, the rightmost column is the instructions to the camera department about zooming, panning, fading etc.

As a historical note, the actual numbers for the zoom would typically be filled in by the camera department, not the animator. The exposure sheet gets defaced by many departments before the shot is done.

This is pretty much how things still are, except that assets are all digital. There is a sense in which they mirror each other, in that every piece of artwork is referred to by a control document and control documents say what to do with the artwork.

For us, consider a sound effect, for example. There is a sound file, say a PCM WAV file, which is the "artwork". But there is also a bunch of control information, like where it should emit from in 3D space, whether or not it should sound muffled (i.e. filtered) if there's something between you and the sound emitter, mix levels, and so on.

Or consider a material. The "artwork" is shaders and textures, which are best stored in a native format. However, there needs to also be some control about how they piece together, or if there needs to be animation of some kind (e.g. consider a flashing light), or all that stuff.

Did that help?
Pseudonym
 
Posts: 129
Joined: Tue Aug 13, 2013 3:54 am

Re: Game tools

Postby wrongu » Thu Sep 19, 2013 12:15 pm

Pseudonym wrote:Did that help?


Very much, yes!

The last piece of the puzzle for my understanding is the way that the JSON control-documents are eventually incorporated into the game. The 3D location of a sound source, for example, is usually not going to be a static property like it would in a movie. It is instead probably bound to the state of some other object, like a laser. So instead of a 3D coordinate, this sound object may instead have some attribute for "dependency" and point to the laser's coordinates?

^This is not a problem of server design, of course. The server just keeps track of JSON, it's up to us to use it properly.
wrongu
 
Posts: 46
Joined: Sun Sep 08, 2013 6:15 pm

Re: Game tools

Postby Pseudonym » Fri Sep 20, 2013 12:07 am

wrongu wrote:The last piece of the puzzle for my understanding is the way that the JSON control-documents are eventually incorporated into the game. The 3D location of a sound source, for example, is usually not going to be a static property like it would in a movie. It is instead probably bound to the state of some other object, like a laser.

The way that this will be handled is through the entity component system. Here's a very brief introduction if you need one.

Exactly what components we'll have is not yet decided, and the design of (say) the sound effect part of that will probably change, and change rapidly, as it's implemented. However, the short version is probably that "sound emitter" will be a component, and the location it comes from will be the position of the entity. The sound system will inspect both an entity's "sound emitter" component and "world position" component.
Pseudonym
 
Posts: 129
Joined: Tue Aug 13, 2013 3:54 am

Re: Game tools

Postby Pseudonym » Thu Oct 24, 2013 11:48 pm

I'm putting this comment on this thread because this is very likely to be the only place where we use REST.

Please consider using Swagger. It's basically a simple way to document REST APIs which puts the documentation next to the code, so avoids you having to write too much in the way of non-code.
Pseudonym
 
Posts: 129
Joined: Tue Aug 13, 2013 3:54 am

PreviousNext

Return to Code

Who is online

Users browsing this forum: No registered users and 1 guest

cron