Skip to main content

ChatRoomMapManager

@description

Binary-encoded map data

This module implements the new way of encoding the map data.

At its core lies the concept of a BitString: a stream of tightly-packed numbers with arbitrary bit width. This allows us to store data way more efficiently than using plain JSONs, even if they are packed with LZString.

Compatibility

Binary encoding, while efficient, requires a very careful architectural approach to ensure maximum compatibility. Notable, we must ensure that:

  • Exported map strings from any older game version always remain compatible with the newer game versions. Players losing their old saved maps is an unacceptable outcome; we must ensure that we recover as much data as possible from those old saves.
  • Map data synced between the players in a map-enabled room must be readable by the clients one version older than the current one. This is to ensure that during the beta period the main branch players could join and play the rooms created by beta players. This is not as strict of a requirement as the previous point, but is still important.
  • Exported map strings from the newer version must be usable by the players using a game one version older. This ensures that the beta players can share map strings with non-beta ones, and is the least concern among others, since beta periods are quite short and sharing the map string doesn't happen too often. Still, it is good to at least make some effort to allow it.

Binary encoding makes achieving those requirements non-trivial, because to decode a given BitString the game must know exactly what were the bit widths of the integers encoded into it, and also their meaning. If we just change the code that encodes the map data, then we would no longer able to decode the old data.

To solve this issue, we introduce the concept of codec versions. A version is a number that we write into the bit stream before the actual data, which would allow the game to understand which codec was used to encode the data, and call it to decode the data.

Whenever we need to sufficiently change the encoding scheme, we copy the latest codec, increase its version and make the required changes. Copying and pasting the code, while usually not advised, would be a better approach in this specific case. This way, we ensure that the old codecs remain "frozen" in time, so no matter how old the map data is, we always have an appropriate codec for it.

One issue which may arise in the future is the change in the schemas of the objects we encode. In this case, we would need an additional "migrations" layer which would take the old decoded data and convert it to the one we currently require.

Solving the issue of letting the old clients to use the data from beta versions is not that straightforward, and on the most occasions we would require ad-hoc solutions. For example, during the beta period we may use two fields, Data and DataOld, with the former containing the data encoded with the most recent codec, and the latter having the data encoded with the previous codec. Of course, depending on the nature of the required changes, it may be possible to make a more space-efficient solution.

Future work

Currently, we only binary-encode the map effects, to remain in the scope of the original MR. We do this by storing the encoded map effects in the ChatRoomData.MapData.Effects global value, while ChatRoomData.MapData.Tiles and ChatRoomData.MapData.Objects remain unchanged. Thus, we don't need to change much of the existing code, which continues to use those latter fields.

In future MRs we hope to unify the encoding of tiles, objects and effects, writing them all into a single BitString. This would allow us to have much greater compression and save a lot of traffic.

Later, all map data would be stored in ChatRoomMapManager.Map global value instead of ChatRoomData.MapData. This is because we're no longer storing the map data as simple strings which we can trivially serialize and send to the server. Ideally, the outside code would use ChatRoomMapManager methods to obtain the encoded map data when needed (e.g. sending it to the server, or saving the map data for room recreation, or exporting the room via a room code). Failing that, we can continue the approach used in the initial version of this system: having the decoded map data in ChatRoomMapManager.Map and maintain the encoded representation of this map in ChatRoomData.MapData.

After that we would have an avenue for encoding additional arbitrary data within each tile while retaining the compact encoding. This, then, would allow us to have any sorts of "tile settings", which would be a great addition to the map rooms in the Club.

Mod compatibility

This module is a work in progress and would change significantly in the future. As such, only the minimum amount of public APIs is exposed as of now. Mod authors are advised to not rely on its current behavior if at all possible. We expect to expose more public APIs in the future as the module matures.

General design choices

While being public, ChatRoomMapManager.Map preferably should be only accessed inside this file as it is an implementation detail of this module. If the outside code requires to access something in this module, it's best to provide a separate function in the ChatRoomMapManager namespace, or a global one.

Codecs general overview

Version 0

The initial codecs version. Only encoding map effects. Only allows for a single map effect per tile (the groundwork for having multiple effects per tile is laid, but the rest of the code is not ready for it).

Effects are encoded by their IDs, similar to the original Tiles and Objects encoding. A simple RLE compression is applied to the "flat" effects array, with a small twist: we use larger bit width for storing run-lengths of the blank effect sequences. This allows us to more efficiently encode the typical maps where the most of the tiles would have blank effects.

Additionally, we modify the effect IDs in the following way:

  1. First, we subtract the lowest used effect ID (ChatRoomMapViewEffectStartID in the most cases) from them, getting what we call "shifted" IDs which begin from zero.
  2. Next, we create the list of all used "shifted" IDs and write them in the stream. The usage of "shifted" IDs ensures this array is very compact no matter what our ChatRoomMapViewEffectStartID is.
  3. Finally, when writing the effect IDs, we instead use the indexes in the list from the previous step, and call them the "remapped" IDs.

This allows us to write the least possible amount of data per effect ID: for example, if only one effect - besides the blank - is used in a map, then each mention of that ID would only require a single bit of data, no matter what the actual value of this effect is.

This scheme results in a sufficiently efficient compression rate in practice.

  • For maps without effects we will be sending 24 additional bytes (after base64 encoding).
  • Maps with a few patches of effects require around 0.5-1 bits per tile (after base64 encoding).
  • Moderately sophisticated maps with a lot of different effects require somewhere around 1.5-3 bits per tile.
  • In the worst case scenario (a map fully filled with all possible effects without repetitions), we would require slightly above 5.3 bits per tile after base64 encoding.

Index

Namespaces

Interfaces

Variables

Functions

Namespaces

MapEffectsCodecs_v0

MapEffectsCodecs_v0:

ConstSettings

ConstSettings:

arrayLenInBytesBits

arrayLenInBytesBits: number

bitLenBits

bitLenBits: number

rleBitsEmpty

rleBitsEmpty: number

rleBitsFilled

rleBitsFilled: number

rleMaxEmpty

rleMaxEmpty: number

rleMaxFilled

rleMaxFilled: number

DynamicSettings

DynamicSettings:

baseIdBits

baseIdBits: number

effectIdMin

effectIdMin: number

effectIdShiftedToRemapId

effectIdShiftedToRemapId: Map<number, number>

effectIdsShifted

effectIdsShifted: number[]

effectsLength

effectsLength: number

remapIdToEffectIdShifted

remapIdToEffectIdShifted: Map<number, number>

remappedIdBits

remappedIdBits: number

shiftedIdBits

shiftedIdBits: number

Interfaces

EffectsCodec

EffectsCodec:

read

write

MapCodec

MapCodec<T>:

Type parameters

  • T

read

write

Variables

Map

Map: MapManager = ...

Functions

OnMapDataUpdated

  • OnMapDataUpdated(): void

  • This function should be called each time the external code updates ChatRoomData.MapData.

    This function decodes the updated map data and replaces the data stored in $ChatRoomMapManager.Map with the decoded map.

    Returns void

OnViewActivate

  • OnViewActivate(): void
Page Options