battleMETAL - Why Quake / Darkplaces - Part 5 - Network Layer

Connecting for long distance mech mayhem
 
I will try to keep out of the deep end for this portion of the blog series. I forgot that these sections aren’t meant to be a detailed breakdown of the Quake engine or the Darkplaces source port. Rather, these posts are laying out the battleMETAL project’s discovery and assumptions before the work of battleMETAL truly began. In that spirit, these posts should be fairly short and mostly general about tech, code, architecture, etc.

At the time I was deciding whether or not to roll forward with Quake / Darkplaces for an engine, the concept of having multiplayer stood out. A mech game with some dedicated multiplayer would surely be a great game to try, and maybe even with friends. Quake’s network architecture, despite being a tad dated, is fairly robust. DooM’s network layer was peer-to-peer, where game-state was sent around to every player on the server; Quake’s networking is client-server. A main server of the game is started, and then all players connect to the server as clients. This is a server-authoritative design, where game-state is held on the server and then sent to clients. While this scales more nicely than DooM’s peer-to-peer system, the architecture wanders into sticky issues like client-side prediction and lag.

Deep diving into game networking aside, Quake had the feature set that overlapped with battleMETAL’s requirements nicely. Out-of-the-box, Quake allows any players to create their own servers, configure those servers (map, game mode, number of players, certain game rules, etc) and Quake has functionality for server-browsing. Although the menu system for Quake was rudimentary (even by late 90’s standards) it demonstrated that this functionality could be accessed by modders and programmers using custom code. The end result is customized server-browsing menus and features that have been added to battleMETAL.

In game code, Darkplaces also had mapped out functions for processing player input, data transmission between client and server, and functionality for fully customizing the player’s client-side view. Now that said, the exact knowledge for implementing these features was a bit more esoteric, but we will get into that later. Knowing that the network layer was securely placed in the engine code, vs the game code that battleMETAL would be using, Quake really had an attractive feature set for networked gameplay. One benefit of using a source port like Darkplaces is that there have been some key updates to the original Quakes network code. A little bit of client-side prediction which alleviates latency between server and client. Also increases the packet size, allowing more data to be transferred between client and server (after all, network speeds have increased since 1996.)

Further on in this series we will see that some of these initial assumptions might have been a tad generous or very much off the mark of what was actually useable. For now, let’s stick with the bright and rosy details that Quake was setup to be a multiplayer game and that therefore battleMETAL would be inheriting this functionality with little trouble.

battleMETAL - Why Quake / Darkplaces - Part 4 - The Game Loop

...and looping through it
The core of any game is the ‘main loop’. There’s a reason why we developers throw around the word ‘engine’ when talking about games. The best analogy I use is a car engine. The car burns fuel, which moves pistons that turn a camshaft. That camshaft then turns a belt called the ‘timing’ belt. The timing belt is the clock of the car engine. A game engine is very close to this. The base code is setup to run through a series of functions in a specific order, and there’s a ‘timing belt’ that keeps internal time. In Quake that ‘timing belt’ is a variable nicely called time.

Entities, remember those little guys from earlier? also form the only true ‘object’ for the Quake C language. Quake C is a cut-down version of the C language, which is powerful but does not do ‘objects’ very well. Quake C gets around this but ever so slightly. There is only 1 type of entity and all extensions to that entity are given to every entity. What this means is, if there is a custom value, such as ‘weapon type’ for an entity, every entity will now have this variable regardless if it used by that entity. This design paradigm is a blessing and a curse. The blessing is that it means creating customized entities is as simple as just assigning values to the programmer’s desired variables. The curse is that it means the code will bloat-up really fast, leaving hundreds of entities with unused values.

When a player launches a game of Quake, after going through the menu system, and launching into a desired map; the engine creates a list of entities. Almost every object on-screen is an entity. Quake organizes its entities into a list, or more specifically in-code, an array. The wonky aspects begin here. This array of entities has a fixed limit, ie how many total entities the engine can have ever. Original Quake can only handle about 640….Darkplaces however upped this cap to 32,768. Quite an improvement, eh? However there’s still a quirk remaining. The array of entities in Quake is more like a slot-system. Each of those 32,000 places in the array is a slot with a number. When the code wants to create a new entity, it finds the next available slot to place the entity...based on where the slot counter is. It is never a guarantee that the next entity the code creates is right after the current entity the code is on.

This also kinda creates a problem with removing entities. Quake never technically ‘removes’ an entity from the array. Instead, it ‘zeroes out’ the entity’s variables, removing things such as any model assigned to it, and other variables. This is was an intentional design by iD Software. Although there’s a difficulty in determining if any given entity is ‘empty’, it allows the system to ruthlessly clean up ‘unused’ entities to keep system cost down. Darkplaces keeps this paradigm in place, allowing for a fast run through of all the entities in the main array.

Entities can be invoked with the spawn() which returns an entity, usually the code will load this entity into a variable for modification: local entity foo = spawn(). From there, the programmer then must assign the variables they want to go into the entity. If the programmer wants the entity to do something in the future, they would assign a .think() function. A think() function is any code that is executed every time that entity is activated in the entity array. There are several other ‘template’ functions that can be used for various parts of entity behavior. Want to have custom code run when one entity touches another? Give that entity a .touch() function.

A programmercan also create their own child functions for the parent entity, such as entity.thisFunction(). And thanks to the code being C-like; this specific function can be whichever function the coder wants to point to. This allows for neat tricks where a set of entities can have the same function, .thisFunction() but for each entity, the actual function is completely different, like this example:

  Entity A

     .myFunction() = fireWeapon_Type_1()

  Entity B

     .myFunction() = fireWeapon_Type_2()


So when the code calls .myFunction(), if the entity is A, then fireWeapon_Type_1 is called. There’s a bunch of other C language quirks that are leveraged in Quake C, but that’s drilling down into programming language design.
Despite some of the limitations with the programming language, I liked the straightforward approach to the code design. 

I figured I could apply higher level code principles to the Quake C language to get around some of its limitations - a fine example is using the Factory Pattern for entity creation. I don’t need to rewrite the same entity customization when making, say, a missile entity. Instead, I can invoke a makeMissile function that does it all for me. Entity cleanup is a bit trickier, and checking if an entity is truly ‘empty’ or ‘null’ (in code parlance) remains a problem that has to be accounted for. Writing interfaces was sort of out of the question, but I think the entity object in Quake C qualifies as an abstract class which is super helpful.

battleMETAL - Why Quake / Darkplaces - Part 3 - 3D Rendering

Putting Stuff on-screen...

I understand that a lot of this content might be dry for readers who aren’t big into the nuts and bolts of game programming, and I’m sure that this probably doesn’t go deep enough for people who do. That said, I’m almost through this topic and I feel it needs to be typed out for context and for my own reference.

Keeping with the theme of why Quake/Darkplaces, this post will shift over to the rendering aspects of Darkplaces that merited looking into the engine. Darkplaces increases the types of 3D model file formats that can be used in-game, expanding from Quake 1’s thoroughly ancient (by 2016) .mdl format. Darkplaces can use .obj, .md2, .md3, .ase 3D model formats and their higher poly-counts. battleMETAL was never going to be a graphical powerhouse that could challenge modern games. I had the desire to make a game where graphics were clean enough that players could understand what they were looking at without being able to make super lush models.

The .md3 model format suited the project’s needs fairly well. MD3 is Quake III’s model format, which is from 1999. It's interesting to see what only 3 years of difference between Quake 1’s MDL and Quake III’s MD3 makes. MD3 can hold more detail, can something like 256 skins for a model file, and can have different textures for different models in the file. The MD3 format’s animation data is vertex based instead of bone based. MD3 was design by John Carmack for Quake III, so its footprint over a game network is incredibly low. Animations aren’t exactly a high priority (as we’ll see when I break down how I made the mech models).

Quake’s paradigm for adding models to objects is also fairly simple. Almost every object you see on the screen, in-code, is a thing called an entity. An entity is an abstract concept in game design, being a root-object of sorts from which all other objects inherit from. An entity in Quake can only ever have 1 model associated with it, which is given to the entity with the code command setmodel(). Once an entity has a model, there is functionality to affect the rendering of that model. Code can make the model partially visible, or invisible, glow with color, etc. So even if an entity has a mode, the game coder can dictate when that model should be visible. Another benefit is that an entity’s model can be changed quickly to something else entirely or removed if needed as well.

Now why does this matter? From where I sit, Quake lets the designer manage the game’s assets in their own way. Modern game design dictates that assets must be loaded into the entire development kit of the specific engine that the developer is using. There are benefits to this top-down approach, such as uniformity of design process when maintaining a project across a larger team. It also allows developers to move from project to project with familiarity of the tools. However, for battleMETAL and its one-man-army of me, I didn’t need such a heavy-handed approach. Quake allows me to manage assets as needed, adding or removing models at the code level, or even updated models in the file folders.

The last set of positives for Darkplaces is that the engine also provides tools for realtime lighting and shadows. Older game engines generally do not have this feature due to system limitations. ‘Realtime’ lighting is shorthand for saying lighting is dynamic as objects move around the game world as well as lights turning on and off. It also means lighting is more realistically represented when being projected from a source. Lighting in 3D computer graphics is its own Lovecraft-ian world that I won’t get into here, and you can find better explanations of it elsewhere. For Darkplaces, Levels can have light entities, and entities themselves can give off light. For battleMETAL, the Darkplaces lighting system is ‘good enough,’ providing enough detail to look good and allow for design aesthetics but also performant and manageable.

A neat tool that Darkplaces has is the ability to modify lights in a game level in real time using a series of built in commands. These changes are then saved to a specific file, ‘<map name>.rtlight’ which the game will load every time that map is opened. The advantage of this tool is no need to have to rebuild or recompile the map to test different lighting styles. The file is a plain text file with coordinate data, which also means the file can be copied and reapplied to other levels, or the same level. An example would be having a map transition from day to night, or a building’s lights turn on after being initially off.

The looser, bottom-up design of Darkplaces features put less emphasis on mastering a specific development environment and more on designer-based preference for their own tools. Engines like Unreal and Unity have lengthy tutorials for the various all-in-one tools they provide. For someone who doesn’t know these tools, the time required to learn them inhibits game design and the iterative testing of the project. Eventually I will give these a shot because they have their own pros and cons, but for now, I didn’t want to feel like I was fighting the tools too much….and yes we will see how I would eat this words in a buffet of hubris over this series of posts.

battleMETAL - Why Quake / Darkplaces - Part 2 - File management / File type loading

Files and their management; kind of important

This post was a bit late, things are getting a little busy around here.

One aspect about game coding that I like to pay attention to is file loading and management. Now this may seem like an obvious thing for any application, but one could be surprised at the variety of management strategies. Some games will hardcode data files to be loaded in a certain order. Other games may keep all their files compiled into the game itself. Quake uses what is called a Virtual File System, it sounds complicated but it boils down to this: When the game Quake runs, it treats its home folder like, say Windows, treats a folder. Quake knows how to dive into folders to get files, create new files. It has a set of file types that it can read and has loose preset folder structure.

Darkplaces really just extends the file types that Quake can load, mostly for modernization, but this also has an impact on other game aspects like graphics. Darkplaces allows for higher resolution images to be used, higher-poly models, and cleaner sound formats. Quake’s organizes these files pretty much for the programmer. Yes a programmer can extend the folder structure to their heart’s content, but that should only really be a last resort. An example of this automatic organization would be loading menu graphics.

/Quake/id1/gfx/

If we break this down, we see that id1 contains all of the game files that Quake needs to run, and that the menu graphics are then packed into the /gfx/ folder. This paradigm is also used for models with the /progs/ folder. An interesting note about the /id1/ folder, this is just for Quake’s game files. Modders wanting to make their own games and mods, can easily make their own separate folder such as /battleMETAL/ and then Quake / Darkplaces can switch between the two folders during runtime. This allows users to seamlessly switch between their favorite mods without too much interruption of gameplay. The caveat for my game, battleMETAL, however is that it's a total conversion which precludes compatibility between Quake’s settings and battleMETAL’s. So for my game, I stash everything in the id1 folder with the assertion that a user should not try to run both Quake and battleMETAL in the same install.

Why does this stuff matter? It makes management of dozens of game files easier and reduces to the amount of file loading that the programmer has to write themselves. Most game assets for Quake are loaded in the game code through the following functions precache_sound, precache_model, precache_pic. Even better, the textures for models are precached when their parent model object is, thus reducing the number of precache calls. This makes adding new content super easy as well. The built-in file loading code of Darkplaces knows how to process any of the files that are added through a precache call.

The final piece of this good feature is, we have all these files, how do we package them nicely for a release? Once again, Quake and Darkplaces have an easy answer. Within the game’s asset folder, Darkplaces can read and open .pk3 files. ‘Pak’ files are just a .zip file that has been renamed. This allows the developer to keep the release version of the asset folder clean. By naming pk3 files in successive numbers - PAK0.pk3, PAK1.pk3, the developer can actually split up game assets. Now, why would anyone want this? Think about it this way, if the developer splits game models into PAK1 and sound effects into PAK2, the developer can then patch the models by simply releasing a ‘new’ version of PAK1 with the new models. This siloing of resources is incredibly handy.

In conclusion, Quake / Darkplaces file-folder system implementation is clean and performant. It is useful at-scale, during development and is savvy in its organization for release. This is a strong reason to use Darkplaces as a game engine for simpler game making.