Using ZIPs as WAD replacement
NOTE: Only 2.1.0 Test3 and higher is able to read compressed archives.
- A .zip file used as the container for an 3DGE mod is usually given the .pk3 extension, instead of .zip. While there is no technical differences, this helps avoid potential confusion from users thinking the mod should be extracted from it, and also prevents Windows from presenting it as a directory. 3DGE supports several ZIP archiving methods: stored (no compression), shrunk, imploded, deflated (the most commonly used), bzip2 and LZMA algorithms. Note that many ZIP utility only partially implement the ZIP standard and as a result their support is limited to stored and deflated files. This partial implementation also affects some editing tools which normally support the PK3 format, such as 3DGE Builder, Doom Builder 2, and Slumped.
- EDGE 2.1.0 Test3 and higher uses a modified ZIP container named EPK (Edge PaK), which is the recommended extension for EDGE-specific mods. The main datafile for EDGE 2.1.0 Test3 now uses this container format.
- A .7z file used as the container for a 3DGE mod is usually given the .pk7 extension, for similar reasons. This format offers much better compression ratio, however its reduced support, especially by modding utilities, make it less interesting overall. It also increases the memory footprint on the engine, so caution must be taken when choosing between a 7z and a PK3 format. A possibility would be to develop a mod as a PK3, then convert it to a PK7 once finished, to decrease the size of the archive to be distributed. However, keep in mind that decompression is quite slower than for zips, so loading times are increased.
- A .PAK file is used as an alternative container for 3DGE mods; it follows Quake's PAK convention strictly. This also works for Quake II PAKs, as well as .SIN packfiles. It is nearly functionally identical to the WAD container format.
Compressed archives have many advantages over PWADs and PAKs, even for storing content for Doom:
- Smaller file size due to compression
- Greatly reduced memory usage during play
- Proper use of directories instead of WAD namespaces
- You no longer need dedicated WAD management tools like XWE for manipulating your data.
- More flexibility due to the use of full path names. Currently there is only limited support but it is planned to change everything that references lump names to support full paths.
You can embed WADs inside ZIPs. Any WAD file found in the root directory will be added to the lump directory as well. This allows loading many of the ZIPs distributed through /idgames without the need to extract them. If several wads are included in an archive, they will be loaded in alphabetical order.
- WARNING: Loading WAD files inside of a ZIP or a PK3 will cause memory issues to arise. 3DGE will unpack each WAD file and use more memory to access the contents. This effect quickly adds up, especially in larger mods (with the exception of map wads in the maps/ folder, provided they follow the format explicitly -- see below). The bigger the mod is, the more likely it may run out of memory during play and crashing 3DGE, or for memory corruption to occur. Thus, it is highly recommended to completely unpack the entire contents of any WADs inside, or part of the mod, into the sorted directories.
- Furthermore, 3DGE supports and promotes subdirectories under the main ones for sorting purposes (i.e. Sprites/Weapons/Cannon, Sprites/Monsters/Technodemon, etc). No extra effort is needed to indicate the sprites are present, as 3DGE automatically scans all subdirectories (though PNG and JPG images will still need to be defined in IMAGES.DDF.
Unlike WADs which do not have a true directory structure, the placement of data within ZIPs is much more strictly enforced in order to allow the engine to recognize it as a default lump. Any special data must be placed in the respective directory inside the ZIP or it will not be found. For the lump name, the first 8 characters of the file name are used, the extension is stripped. So for example, to place a COALHUD into a ZIP file, you have to name your file coalhud1.TXT, COALHUD1.LMP (or whatever extension you like) and place it in the ZIPs root directory. The following sub directories are used to assign data to the existing WAD namespaces in 3DGE:
|acs/||contains ACS libraries normally found between A_START and A_END Not yet implemented!|
|cameras/||contains CAMDAT files for camera-man scripts.|
|colormaps/||contains Boom and 3DGE colormaps normally found between C_START and C_END.|
|OPTIONAL! Can contain all DDF, RTS, and COAL scripts here. This is not explicitly needed however, since they can be added in the root directory of the ZIP all the same. These folders also exists in the root EDGE directory by default, so this is only needed if you do not want to reference your script files externally. It is recommended to use the /scripts namespace instead as EDGE will automatically use /scripts instead of the default external DDF directories.|
|filter/||(New from 2.1.0 Test3) contains lumps and directories that will only be loaded with specific games. Not yet implemented!|
|flats/||contains flats normally found between FF_START and FF_END|
|graphics/||All special graphics like title pictures or font characters must go here. This namespace does not exist in WADs.|
|hires/||contains high-resolution textures normally found between HI_START and HI_END|
|maps/||contains levels in form of WADs. These WADs must only contain the data for one single level (including GL nodes lumps if required). Any other data in such a WAD will be ignored. Note that the file name and not the map label inside the WAD determines how the map is named in the game, unless otherwise specified by Levels.DDF. 3DGE will automatically build the GL nodes via glBSP if a map is found.|
|models/||contains MD2, MD3, or MD5 models. This namespace does not exist in WADs.|
|music/||contains all data that is used as music. This namespace does not exist in WADs.|
|patches/||contains patches normally found between PP_START and PP_END|
|scripts/||contains DDF/RTS/COAL script files. They can be named by their 8 character name, or their full names as seen in 3DGE's root directory, under /doom_ddf.|
|skins/||contains all skins to be loaded for 3D models. This namespace does not exist in WADs.|
|sounds/||contains all sound files that are referenced by DDFSFX. This namespace does not exist in WADs.|
|sprites/||contains sprites normally found between S_START and S_END (also known as SS_START, SS_END)|
|textures/||contains textures normally found between TX_START and TX_END|
|voices/||contains Strife dialog sounds normally found between V_START and V_END|
|voxels/||contains voxel objects normally found between VX_START and VX_END|
(Note: Any lump found in the root directory is added to the global namespace.)
- SLADE or similar programs are recommended for creating the actual .zip file, to ensure programs like WinZIP or WinRAR do not create them with unsupported file compression. Otherwise, they may appear to be damaged, which may hinder modifications of the pk3 with SLADE, Slumped, etc.
- SLADE is also capable of creating the .pk3 file directly, where WinZIP and WinRAR cannot; WinRAR can still open pk3s though.
- The ZIP file's directory is sorted alphabetically before adding it to the lump directory; any WAD files in the .zip are then loaded afterwards. Keep that in mind when creating data that relies on file name ordering. For most things, there is no need to depend on ordering, with the exception of texture animations of type range that animate textures from the flats/ or textures/ subdirectories.
- Any file not in one of the reserved directories is not added to the WAD directory and can only be used by code that looks for full path names. Currently that is only the music code but it will be gradually extended so that everything that references a lump name can do it.
- Sprite lumps for the \ frame in a WAD (such as VILE\* for one of the Arch-Vile's healing frames) can be put in a ZIP file, the backslash character just has to be changed to a caret character (^). So, VILE^1 to VILE^8 in a ZIP file will be interpreted as the VILE\1 to VILE\8 lumps. This replacement only works for sprites, any other lump name should not contain backslashes anywhere. As it is possible to use multiple sprite names for an actor with DDF, frames beyond Z need not be used however.
- It is strongly recommended to use the .pk3 extension, not .zip, when creating a file meant to be loaded directly into 3DGE. The average user is used to opening a .zip file and extracting a .wad from it, so naming the actual game file a .zip will often cause unwanted confusion.
- The contents of the files inside the ZIP must be identical to the lumps in a WAD. This is especially important for graphics. 3DGE does not read .BMP files! Even though that is the most likely format you will encounter, importing them with a WAD manager will convert them to Doom's internal format. XWE converts graphics encoded in Doom's internal format to .BMP during export. SLumpEd can export these graphics as raw Doom lumps. In order for 3DGE to recognize graphics as such, they have to be either in Doom's internal format or a supported image format such as PNG.
- Beware of hidden files! If you add a full directory, or directory tree, to a PK3, it is possible that unwanted system files (such as Windows' thumbs.db file that tends to be created in any folder containing pictures) will be included as well. They will needlessly increase the file size and can possibly trigger warning messages when loading 3DGE. Slumped frequently crashes when browsing through a PK3's resources, as it expects thumbs.db to be a graphic and obtain incoherent values.
Currently the only editors which support this format are 3DGE Builder, Doom Builder 2, SLumpEd and Slade 3, but of course you can also use any Zip tool available if you do not need access to Doom specific features like texture definitions or graphics offsets.
ZDoom, Eternity, Vavoom and Doomsday also use PK3 files for their resources; however their subdirectory organization differ. Adapting a mod from one of these ports to work in 3DGE may therefore require repackaging, in addition to conversion of enhanced features to their 3DGE equivalent. The Eternity Engine adopted a model mostly similar to ZDoom's, but different enough for incompatibility to be possible in both ways. 3DGE is also similar to ZDoom's.