freedoom/BUILD-SYSTEM.adoc

= Build system

This is an explanation of the functioning of the Freedoom build
system.

== Overview

Freedoom is built using `deutex`, which is a command-line tool for
automated building of Doom WAD files.  However, multiple different WAD
files are built from the Freedoom material.  Therefore, the Freedoom
build system is more complicated than a “normal” deutex build would
otherwise be.

A top-level +Makefile+ controls the build system and executes the
appropriate commands to generate the output WAD files.  Various
intermediate files are generated in the process by scripts.  The
following diagram illustrates the process:

................................................................

        buildcfg.txt     lumps/textures/textures.cfg
             │                       │
             │                   ┌───┴────┐
             │                   │  cpp   │
             │                   └───┬────┘
             │                       │
             │             ┌─────────┴────────┐
             │             │  build-textures  │
             │             └──┬────────────┬──┘
             │                │            │
             │        pnames.txt    texture1.lmp, pnames.lmp
             │                │            │
         ┌───┴────┐           │            │
         │  cpp   ├───────────┘            │
         └───┬────┘                        │
             │                             │
        wadinfo.txt                        │
             │                             │
         ┌───┴────┐                        │
         │        ├────────────────────────┘
         │ deutex │
         │        ├───────── all other source files (graphics, etc)
         └───┬────┘
             │
      wads/(wadname).wad

................................................................

=== Output WAD files

The following are the resulting WAD files generated by the build
system:

 * +freedoom1.wad+: Phase 1, the _Ultimate Doom_-compatible IWAD file.
 * +freedoom2.wad+: Phase 2, the _Doom II_-compatible IWAD file.
 * +freedm.wad+: FreeDM IWAD file, containing deathmatch levels

== Source configuration files

=== Master configuration file (+buildcfg.txt+)

deutex is configured using a configuration file typically named
+wadinfo.txt+.  In the case of Freedoom, multiple different WADs are
built from the common material, with slightly different settings for
each.  Therefore, a _master configuration file_ named +buildcfg.txt+
is used to generate configuration files for each individual WAD to
build.

The +buildcfg.txt+ file is processed using a Python script named
+simplecpp+.  This processes files with a syntax similar to the +cpp+
tool (C Preprocessor).  Variables are defined on the command line,
based on the type of output target desired:

 * +PHASE1+: Build for the Phase 1 IWAD.
 * +PHASE2+: Build for the Phase 2 IWAD.

=== Texture configuration file (+lumps/textures/textures.cfg+)

The texture configuration file is used to generate the texture lumps.
These are +texture1.lmp+ (texture directory), +pnames.lmp+ (list of
patch names) and +texture2.lmp+ (Doom I only). They are generated
by the +build-textures+ script; deutex’s internal texture builder
is deliberately not used for compatibility reasons.

Unlike id’s Doom games, Freedoom includes nearly every texture
possible in all three of its IWADs.  There are a handful of textures
that must differ between Phase 1 and 2 for compatibility with Doom 1
and 2 mods, since their definitions changed in the Doom games.  We
also include logo textures for FreeDM levels.  The file is passed
through the +simplecpp+ script to account for the variant textures.
Command line variables are defined based on the desired build
settings:

 * +DOOM1_VERSIONS+: Include textures in Phase 1 that would conflict
   with Phase 2.
 * +FREEDM+: Include textures that are needed for FreeDM.

== Generated files

The following files are generated automatically by automated scripts
during the build process.

=== Texture files

+texture1.lmp+ is a binary file that contains the texture definitions
and is built by the +build-textures+ script from
+lumps/textures/textures.cfg+ as a template.

+pnames.txt+ contains a list of all of the patches used in the texture
definition file (+texture1.txt+). It is generated as an output by
the +build-textures+ script.

Multiple sets of these files are generated with different
configurations for each IWAD file.

[frame="topbot",grid="none",options="header"]
|===============================================================
| Subdirectory             | WAD File        | CPP Variables
| +lumps/textures/phase1/+ | +freedoom1.wad+ | PHASE1
| +lumps/textures/phase2/+ | +freedoom2.wad+ | PHASE2
| +lumps/textures/freedm/+ | +freedm.wad+    | FREEDM
|===============================================================

=== +wadinfo.txt+

This is the auto-generated deutex configuration file, built from the
+buildcfg.txt+ template.  It includes the PNAMES list from
+pnames.txt+.

Several different +wadinfo.txt+ files are generated for the different
WAD files that are built:

[frame="topbot",grid="none",options="header"]
|===============================================================
| Filename             | WAD File           | CPP Variables
| +wadinfo_phase1.txt+ | +freedoom1.wad+    | PHASE1
| +wadinfo_phase2.txt+ | +freedoom2.wad+    | PHASE2
| +wadinfo_freedm.txt+ | +freedm.wad+       | FREEDM
|===============================================================

== Auxiliary scripts

The build system uses a number of auxiliary scripts in order to
generate the necessary configuration files for the build.  These are
written in Python.

=== +simplecpp+

This script implements a subset of the syntax of the C preprocessor.
It accepts #ifdef ... #endif blocks to conditionally include sections
of the input file in the output file.  This is used when generating
the +wadinfo.txt+ and +texture1.txt+ files to conditionally include
certain resources depending on the type of WAD file being built.

The +simplecpp+ script is used in preference to the actual +cpp+
preprocessor, firstly so that it is not necessary to install +cpp+,
and secondly because processing with +cpp+ can alter the layout of the
output file.

=== +build-textures+

This script processes the texture file (+texture1.txt+) and outputs
the binary texture lumps +texture1.lmp+, +texture2.lmp+ and
+pnames.lmp+.

Also output from the +build-textures+ script is a file named
+pnames.txt+ which is simply a text file containing the patches
listed in +pnames.lmp+. This is included in the wadinfo.txt file
so that all patches listed by the texture directory are automatically
included in the IWAD with no need for manual configuration.

=== +textgen+

Found in +graphics/text+, this script generates various graphics
files that are used in the Doom menus and intermission screen.
It reads level names from the dehacked lumps.

=== +playpal.py+

Found in +lumps/playpal+, this script builds the +PLAYPAL+ lump which
contains the 256-color palettes used for special effects (injured
“red” flash, the green “radiation suit” effect, etc.)

=== +colormap.py+

Found in +lumps/colormap+, this script builds the +COLORMAP+ lump that
is used for the diminished lighting within the game.  This script is
also reused to generate additional colormaps that can be used for
special effects. It has a number of command line options that allow it
to do various different colorizing and “fog” effects.

=== +mkgenmidi+

Found in +lumps/genmidi+, this script builds the +GENMIDI+ lump used
for OPL MIDI synthesizer playback. The inputs for this script are a
collection of instrument files that are in the standard +SBI+ format
for OPL instruments. This lump is essential if you want to play using
a classic Adlib or Soundblaster card; these are emulated by several
source ports and some people still like to play using emulated OPL for
the authentic retro feel.

=== +gen-ultramid+

Found in +lumps/dmxgus+, this script builds the +DMXGUS+ lump used for
GUS MIDI playback. The GUS (Gravis UltraSound) card was a gaming sound
card popular in the ’90s. As with the +GENMIDI+ lump, few people are
still using a real GUS card nowadays, but several source ports emulate
them and require this file.

== deutex

`deutex` is the tool used to generate the WAD files.  It processes a
file typically named +wadinfo.txt+, reading files from the following
directories to generate the WAD:

 * +flats+: Floor and ceiling textures.
 * +graphics+: Graphics for the menu, heads up display and status bar, etc.
 * +levels+: The levels.  Files are named eg. map01.wad or e1m1.wad
   for Doom II and Doom I levels, with FreeDM levels named eg.
   dm01.wad.
 * +lumps+: Miscellaneous lumps.
 * +musics+: Music files, in MUS or MIDI format.
 * +patches+: Patch graphics that are used to compose wall textures.
 * +sounds+: Sound effects, in WAV format.
 * +sprites+: Graphics for the in-game sprites (monsters, power-ups,
   weapons, decorations, etc.)
 * +textures+: Texture definitions.

=== Idiosyncrasies

deutex is an old tool and has various quirks that must be worked
around.  Some of them are listed here.

 * deutex does not allow the exact path to the +texture1.txt+ file to
   be specified in the configuration file; this used to be a problem
   but the Freedoom’s texture lumps are now built using a custom
   script.

 * deutex requires an existing IWAD file in order to build WAD files,
   and includes the contents of the +TEXTURE1+ lump from the IWAD in
   any +TEXTURE1+ lumps it generates.  To work around this, a “dummy”
   IWAD file containing an empty +TEXTURE1+ lump is contained inside
   the +dummy+ directory.

== Node builder

Each Freedoom level found in the "levels" directory should have exactly
11 lumps in a https://zdoom.org/wiki/WAD[particular order] with no
padding between the lumps. Of those 11 lumps 6 (<map name>, THINGS,
LINEDEFS, SIDEDEFS, VERTEXES and SECTORS) are maintained by a level
editor such as GZDoom Builder, SLADE or Eureka. The remaining 5 lumps
(SEGS, SSECTORS, NODES, REJECT and BLOCKMAP) are added by a node
builder. The Freedoom project uses the
https://github.com/Doom-Utils/zennode[ZenNode] node builder. For
example, to rebuild the nodes for all 100 levels run the
"rebuild-nodes" make target in the top level makefile:
[source,bash]
-----------------
make rebuild-nodes
-----------------
To control how the node builder is invoked, or which levels are
rebuilt, override the NODE_BUILDER and NODE_BUILDER_LEVELS make
variables respectively. For example, to specify the full path to
ZenNode when it's not in the path and to rebuild only the first
episode of Freedoom 1 as well all of the deathmatch levels:
[source,bash]
-----------------
make NODE_BUILDER=/opt/zennode/ZenNode NODE_BUILDER_LEVELS="c1m? dm??" rebuild-nodes
-----------------