====== Synchronet Message Base (SMB) Specification ======

The **Synchronet Message Base** (SMB) is a binary on-disk format for storing high-volume electronic mail and conference (sub-board / forum / "echo") messages. It supports text, attachments, embedded multimedia, FidoNet kludge headers, threading, voting/polls, file metadata, and a wide range of network and gateway encodings.

This page documents the SMB format itself (the on-disk layout and field semantics). For operational tools see [[util:smbutil]] (maintenance), [[util:chksmb]] (integrity check), and [[util:fixsmb]] (rebuild). For the C library that reads/writes SMB files see the [[#smblib|SMBLIB]] section.

  * **Current format version**: 3.10 (''0x0310''), defined as ''SMB_VERSION'' in [[https://gitlab.synchro.net/main/sbbs/-/blob/master/src/smblib/smblib.c|src/smblib/smblib.c]]
  * **Original spec**: v1.21 (1993), preserved at [[https://synchro.net/docs/smb.html]]

Since the original spec was published, the format has been extended significantly. This page reflects the current behavior, modernized from the original ''/sbbs/docs/smb.html''. Where the original normative wording remains accurate, it is preserved.

===== Introduction =====

==== What is SMB? ====
SMB is the on-disk format used to store messages — both private mail and public conference posts (sub-boards, forums, conferences, SIGs, echoes). A single message base is one logical "mailbag" or "conference"; a Synchronet system has one mail base plus one base per sub-board.

==== Why SMB? ====
SMB is designed for **high-volume** message bases under multi-user concurrent access. It supports fast incremental imports (at echomail-tosser rates), random retrieval by various keys, multiple storage tradeoffs (Self-packing / Fast / Hyper allocation), flexible content (text, attachments, multimedia), and extensibility for new field types and network encodings.

==== Implementations ====
The format is open. Third-party programs (message readers, mail tossers, gateways, archive tools) may read and write SMB bases using either the published spec (this page) or the reference C library [[#smblib|SMBLIB]] (LGPL).

===== Definitions =====

==== Notation ====

  * **Control characters** — Written as ''^X'' (caret + letter) or ''ctrl-X''. Case-insensitive (''^z'' = ''^Z''). ''^@'' (ASCII 0) is written ''NULL'' or ''0''.
  * **Hexadecimal** — Numbers are prefixed ''0x'' or ''\x'' or suffixed ''h''. Case-insensitive (''0xff'' = ''0xFF'').
  * **Bit values** — In C notation as ''(1<<x)'', e.g. ''(1<<7)'' = ''0x80''.
  * **Byte order** — All 16-bit and 32-bit integers are stored **little-endian** (Intel x86). A 16-bit value ''0x1234'' is stored as ''0x34 0x12''.
  * **File dump format** — Hex offset, then 16 hex bytes, then ASCII rendering. All numbers in hex.

==== Translation Strings ====
A //translation string// (''xlat'') is an array of 16-bit words listing the encoding/compression transforms applied to a piece of data, in **storage order**. The array is terminated by a single 16-bit zero (''XLAT_NONE''). If no translations apply, the array contains only ''XLAT_NONE''.

When **retrieving** data with multiple translations, apply the inverse of each transform in **reverse** order.

==== Local E-mail ====
The system **mail** message base (''[[dir:data]]/mail.s*'') is special:
  * The ''SMB_EMAIL'' status header attribute is set
  * Hyper Allocation storage method is **not** supported
  * The ''to'' and ''from'' fields of [[#index_file_sid|index records]] do **not** contain CRCs (they store user numbers instead)

==== Acronyms ====
^ Acronym ^ Meaning ^
| ANSI    | American National Standards Institute |
| ASCII   | American Standard Code for Information Interchange |
| BBS     | Bulletin Board System |
| C       | The C programming language as defined by ANSI X3.159-1989 |
| CR      | Carriage Return character (ASCII 13) |
| CRC     | Cyclic Redundancy Check |
| CRC-16  | Standard 16-bit CRC using ''0x1021'' polynomial (seed 0) |
| CRC-32  | Standard 32-bit CRC using ''0xEDB88320'' polynomial (seed -1) |
| CRLF    | Carriage Return followed by Line Feed |
| FSC     | FidoNet Standards Committee (FTS proposal) |
| FTN     | FidoNet Technology Network |
| FTS     | FidoNet Technical Standard |
| LF      | Line Feed character (ASCII 10) |
| QWK     | Compressed message packet format for message reading/networking |
| RFC     | Request for Comments (IETF) |
| SMB     | Synchronet Message Base |
| UT      | Universal Time (formerly "Greenwich Mean Time") |

==== Data Types ====

^ Type      ^ Definition ^
| ''uchar'' / ''uint8_t'' | Unsigned 8-bit (0-255) |
| ''int16_t''  | Signed 16-bit (-32768 to 32767) |
| ''uint16_t'' | Unsigned 16-bit (0-65535) |
| ''uint32_t'' | Unsigned 32-bit (0-4294967295) |
| ''time_t''   | **Stored as 32-bit ''uint32_t''** in SMB on-disk structures (unsigned, seconds since 1970-01-01 00:00 UT — the Unix epoch). The 32-bit unsigned value supports dates through 2106. (Note: on the host system ''time_t'' may be 64-bit or signed; SMB persists it explicitly as ''uint32_t''.) |
| ''ASCII''    | Array of 8-bit characters. Bytes ''0x80''-''0xFF'' are CP437 (IBM PC extended ASCII) historically; modern usage may be UTF-8 (see attribute flags). May or may not be NULL-terminated when stored in a header field. |
| ''ASCIIZ''   | ASCII string with **mandatory** NULL terminator. |
| ''nulstr''   | ASCII string immediately terminated by NULL (empty string). |
| ''undef''    | Data buffer with undefined contents. |

==== Composite types ====

''when_t'' — Time stamp with timezone information (matches current ''src/smblib/smbdefs.h''):
<code c>
typedef struct {       /* Time with time-zone */
    uint16_t  year;
    uint32_t  time;    /* Month/Day/Hour/Minute/Second OR (legacy) time_t */
    int16_t   zone;    /* Time zone */
} when_t;
</code>

The ''time'' field is **dual-format**, distinguished by its **upper 6 bits**:

  * **Upper 6 bits non-zero** → //legacy// 32-bit ''time_t'' (Unix epoch seconds in **local time** at the writer's site). Used by all SMB messages written before SMBLIB v3.10, and still used today for ''when_imported''.
  * **Upper 6 bits zero** → //encoded wallclock//: Month/Day/Hour/Minute/Second packed into the low 26 bits, with the year stored separately in the ''year'' field. Used by ''when_written'' since SMBLIB v3.10.

The wallclock-encoded layout is:

^ Field ^ Width ^ Bit position ^ Macros ^
| Month  | 4 bits | bits 22-25 | ''SMB_DATE_MON_BITWIDTH'' / ''SMB_DATE_MON_BITPOS'' |
| Day    | 5 bits | bits 17-21 | ''SMB_DATE_DAY_BITWIDTH'' / ''SMB_DATE_DAY_BITPOS'' |
| Hour   | 5 bits | bits 12-16 | ''SMB_DATE_HR_BITWIDTH'' / ''SMB_DATE_HR_BITPOS'' |
| Minute | 6 bits | bits 6-11  | ''SMB_DATE_MIN_BITWIDTH'' / ''SMB_DATE_MIN_BITPOS'' |
| Second | 6 bits | bits 0-5   | ''SMB_DATE_SEC_BITWIDTH'' / ''SMB_DATE_SEC_BITPOS'' |

This change was introduced in commit ''[[https://gitlab.synchro.net/main/sbbs/-/commit/445394f9fc31d3d40261737c101ff39fe6ba01d0|445394f9]]'' (2024-12-20), which incremented ''SMB_VERSION'' to ''0x0310''. The motivation: with the old time_t encoding, **changing the system or OS time zone changed the displayed dates of all stored messages** (issue #845). The new wallclock encoding stores what the message author saw on their clock when writing, so the displayed date never shifts with subsequent timezone changes. As a side benefit it sidesteps the 32-bit ''time_t'' Y2038 / Y2106 problems — at least for ''when_written''.

The ''year'' field repurposes 16 bits previously taken by the upper half of ''netattr'' (which was reduced from ''uint32_t'' to ''uint16_t'' in the same commit). File-base entries still use ''time_t'' for ''when_written'' — they were not converted, since the field is not used for much in file metadata.

''when_t.zone'' encoding:

  * In the range ''-720..+720'' — minutes east (positive) or west (negative) of UT.
  * Outside ''±720'' — zone code with bit fields:
    * ''0x1000'' — Non-US zone, east of UT
    * ''0x2000'' — Non-US zone, west of UT
    * ''0x4000'' — U.S. zone (Uniform Time Act)
    * ''0x8000'' — Daylight savings active
  * Lower 12 bits — minutes east/west of UT (without DST adjustment).
  * A value of ''0'' for the entire timestamp is invalid (un-initialized).

Common U.S. zone codes:

^ Code ^ Hex (Std) ^ Hex (DST) ^ Offset ^
| AST  | ''0x40F0'' | ''0xC0F0'' (ADT) | -04:00 / -03:00 |
| EST  | ''0x412C'' | ''0xC12C'' (EDT) | -05:00 / -04:00 |
| CST  | ''0x4168'' | ''0xC168'' (CDT) | -06:00 / -05:00 |
| MST  | ''0x41A4'' | ''0xC1A4'' (MDT) | -07:00 / -06:00 |
| PST  | ''0x41E0'' | ''0xC1E0'' (PDT) | -08:00 / -07:00 |
| YST  | ''0x421C'' | ''0xC21C'' (YDT) | -09:00 / -08:00 |
| HST  | ''0x4258'' | ''0xC258'' (HDT) | -10:00 / -09:00 |
| BST  | ''0x4294'' | ''0xC294'' (BDT) | -11:00 / -10:00 |

A range of non-standard zone codes for international cities is also defined; see [[https://gitlab.synchro.net/main/sbbs/-/blob/master/src/smblib/smbdefs.h|smbdefs.h]] for the complete table.

''fidoaddr_t'' — FidoNet address (zone:net/node.point):
<code c>
typedef struct {
    uint16_t zone;
    uint16_t net;
    uint16_t node;
    uint16_t point;
} fidoaddr_t;
</code>

All on-disk SMB structures are written with ''#pragma pack(push,1)'' / ''pack(pop)'' bracketing them — they are byte-packed without alignment padding.

===== File Formats =====

Each message base is stored as a set of binary files sharing a common base name (max 8 chars under DOS, longer permitted on modern systems) but differing by file extension. The first character of the extension is always ''S'' (for SMB).

^ Extension ^ Purpose ^ Re-creatable? ^
| ''.sid'' | Index — fixed-length per-message records for fast lookup | Yes (from ''.shd'') |
| ''.shd'' | Header — variable-length per-message records (256-byte blocks) | **No** — primary data |
| ''.sha'' | Header block allocation table | Yes (from ''.shd'') |
| ''.sdt'' | Data — message bodies and attachments (256-byte blocks) | **No** — primary data |
| ''.sda'' | Data block allocation table | Yes (from ''.shd'') |
| ''.sch'' | Optional CRC history for duplicate-message detection | Yes |
| ''.hash'' | Hash file with multiple digests for duplicate detection | Yes |
| ''.ini''  | Auxiliary configuration (created at base creation; minimum: ''Created'' timestamp) | — |
| ''.lock'' | Whole-base exclusive lock (transient — created by ''[[#whole-base_lock|smb_lock()]]'', removed by ''smb_unlock()''; presence means another process is performing maintenance/repair) | — |

Allocation files (''.sda'', ''.sha'') are not used when the base uses **Hyper Allocation** storage method.

==== Index File (.sid) ====

Fixed-length records, **20 bytes** each (''SIZEOF_SMB_IDXREC_T''), one per message. The first 6 bytes are a union discriminated by the message ''type'' in the corresponding header:

<code c>
typedef struct {            /* Index record */
    union {
        struct {            /* msg.type != BALLOT/FILE (normal message) */
            uint16_t to;       /* 16-bit CRC of recipient name (lowercase) or user # */
            uint16_t from;     /* 16-bit CRC of sender name (lowercase) or user # */
            uint16_t subj;     /* 16-bit CRC of subject (lowercase, w/o "RE:") */
        };
        struct {            /* msg.type == BALLOT */
            uint16_t votes;    /* votes value */
            uint32_t remsg;    /* number of message this vote responds to */
        };
        struct {            /* msg.type == FILE */
            uint32_t size;
            uint16_t size_ext; /* additional 16 bits of file size */
        };
    };
    uint16_t attr;          /* attributes (MSG_PRIVATE, MSG_READ, etc.; mirrored in msghdr) */
    uint32_t offset;        /* byte-offset of msghdr in header file */
    uint32_t number;        /* message number (1-based) */
    uint32_t time;          /* time/date message was imported/posted (32-bit time_t) */
} idxrec_t;
</code>

For the **mail** base, ''to'' and ''from'' contain user numbers (not CRCs). For sub-boards, they contain 16-bit CRCs of the lowercased names, which speeds searches but allows aliasing.

The message ''type'' is held in the corresponding header record (''msghdr_t.type''); ''enum smb_msg_type'' values: ''SMB_MSG_TYPE_NORMAL'' (0), ''SMB_MSG_TYPE_POLL'', ''SMB_MSG_TYPE_BALLOT'', ''SMB_MSG_TYPE_POLL_CLOSURE'', ''SMB_MSG_TYPE_FILE''.

The index file is fully re-creatable from the header file by walking the headers and emitting one record per message — this is what [[util:fixsmb]] does.

==== Header File (.shd) ====

The .shd file begins with an SMB header followed by a status header, then variable-length message header records aligned on 256-byte boundaries.

=== SMB header (file root) ===
<code c>
typedef struct {
    uchar     smbhdr_id[4]; /* "SMB\x1A" (SMB_HEADER_ID) */
    uint16_t  version;      /* SMB format version (e.g. 0x0310) */
    uint16_t  length;       /* Length of this header in bytes */
} smbhdr_t;
</code>

=== Status header (follows SMB header) ===

The status header doubles as the file-base status header — fields with parallel meanings between message and file bases use a C union:

<code c>
typedef struct {        /* Message/File base status header */
    union {
        uint32_t last_msg;     /* last message number */
        uint32_t last_file;    /* last file number */
    };
    union {
        uint32_t total_msgs;   /* total messages */
        uint32_t total_files;  /* total files */
    };
    uint32_t header_offset;    /* byte offset to first header record */
    uint32_t max_crcs;         /* maximum number of CRCs to keep in history */
    union {
        uint32_t max_msgs;     /* maximum messages to keep */
        uint32_t max_files;    /* maximum files to keep */
    };
    uint16_t max_age;          /* maximum age (days), 0 = unlimited */
    uint16_t attr;             /* base-wide SMB_* attribute flags */
} smbstatus_t;
</code>

Status header ''attr'' (base-wide flags):

^ Flag ^ Hex ^ Meaning ^
| ''SMB_EMAIL''         | ''0x01'' | Mail base — ''to''/''from'' indexes hold user numbers, not CRCs |
| ''SMB_HYPERALLOC''    | ''0x02'' | Hyper Allocation method (no .sda/.sha files) |
| ''SMB_NOHASH''        | ''0x04'' | Don't calculate or store hashes |
| ''SMB_FILE_DIRECTORY''| ''0x08'' | Storage for a file area (not message base) |

Storage method values used elsewhere (passed to functions, not stored as status flags) are ''SMB_SELFPACK'' = 0, ''SMB_FASTALLOC'' = 1, ''SMB_HYPERALLOC'' = 2.

=== Message header record ===

Each message header begins with a fixed-length ''msghdr_t'' structure, followed by an array of //data field records// (''dfield_t'') for the body, then a sequence of variable-length //header field records// (each a ''hfield_t'' followed by its data bytes). The whole record is padded to a multiple of 256 bytes (''SHD_BLOCK_LEN'').

<code c>
typedef struct {                              /* Message/File header */
    /* 00 */ uchar     msghdr_id[4];          /* "SHD\x1A" (SHD_HEADER_ID) */
    /* 04 */ uint16_t  type;                  /* enum smb_msg_type */
    /* 06 */ uint16_t  version;               /* version of this header format */
    /* 08 */ uint16_t  length;                /* total length of fixed record + all fields */
    /* 0a */ uint16_t  attr;                  /* primary attributes (duped in idx) */
    /* 0c */ uint32_t  auxattr;               /* auxiliary attributes */
    /* 10 */ uint16_t  netattr;               /* network attributes (was uint32_t before v3.10) */
    /* 12 */ when_t    when_written;          /* date/time/zone written (8 bytes; v3.10 wallclock or legacy time_t) */
    /* 1a */ struct {
                 uint32_t time;               /* legacy 32-bit time_t (always) */
                 int16_t  zone;
             } when_imported;                 /* date/time/zone imported (6 bytes; not a when_t) */
    /* 20 */ uint32_t  number;                /* message number */
    /* 24 */ uint32_t  thread_back;           /* aka thread_orig — original message in thread */
    /* 28 */ uint32_t  thread_next;           /* next reply in thread */
    /* 2c */ uint32_t  thread_first;          /* first reply to this message */
    /* 30 */ uint16_t  delivery_attempts;     /* SMTP delivery attempts */
    /* 32 */ int16_t   votes;                 /* poll: max votes per ballot; ballot: response value */
    /* 34 */ uint32_t  thread_id;             /* number of original in thread (or 0 if unknown) */
    /* 38 */ union {
                 struct {                     /* for messages */
                     uint8_t priority;        /* enum smb_priority */
                 };
                 struct {                     /* for files */
                     uint32_t times_downloaded;
                     uint32_t last_downloaded;
                 };
             };
    /* 40 */ uint32_t  offset;                /* offset for buffer into data file (0 or mod 256) */
    /* 44 */ uint16_t  total_dfields;         /* total number of dfield records following */
} msghdr_t;
</code>

Notes:

  * ''when_imported'' is **not** a ''when_t''. It uses the pre-v3.10 inline layout (''uint32_t time'' + ''int16_t zone''), with ''time'' always interpreted as a 32-bit Unix ''time_t''. This was retained in the v3.10 commit (''[[https://gitlab.synchro.net/main/sbbs/-/commit/445394f9fc31d3d40261737c101ff39fe6ba01d0|445394f9]]'') for backward compatibility — the same commit converted ''when_written'' to the new ''when_t'' (with ''year'') and reduced ''netattr'' from 32-bit to 16-bit to make room.
  * ''thread_back'' is also exposed under the legacy name ''thread_orig'' via ''#define''.
  * Messages with ''type == SMB_MSG_TYPE_FILE'' use the file-base variant of the union (download counters); other types use the ''priority'' variant.

==== Header Block Allocation (.sha) ====

For each 256-byte block of the .shd file, the .sha file holds **one byte** of allocation state:

  * ''0x00'' — block is free (deleted/unused)
  * non-zero — block is in use (the library writes ''0x01'')

The .sha file enables blocks freed by message deletion to be reused for new messages without "holes" in the file. Re-creatable from .shd. Not used in **Hyper Allocation** mode.

==== Message Data (.sdt) ====

The .sdt file holds message body text, attachments, and embedded media. Like .shd, it uses 256-byte blocks (''SDT_BLOCK_LEN'').

Each //data field record// in a message header points to a contiguous run of blocks here, with type, offset, and length. Message text is stored as one or more text records (each prefixed by a translation-string and followed by the body bytes), each in its own data field.

==== Data Block Allocation (.sda) ====

For each 256-byte block of the .sdt file, the .sda file holds a **16-bit** ''uint16_t'' reference count:

  * ''0x0000'' — block is free (deleted/unused)
  * non-zero — number of messages currently referencing this block (data blocks may be shared between message-copies, e.g. via forwarding, hence the refcount rather than a simple in-use flag)

When a message is freed, the refcount is decremented and the block is freed only when it reaches zero. ''SMB_ALL_REFS'' (= 0) passed to ''smb_freemsgdat()'' forces all references to be cleared regardless of count. Re-creatable from .shd. Not used in Hyper Allocation mode.

==== CRC History (.sch) ====

Optional file holding 32-bit CRC-32 values of recently fingerprinted content, capped to ''max_crcs'' entries (set in the status header). Maintained by ''smb_addcrc()''; new entries push old ones out FIFO when the cap is reached.

In modern Synchronet, the ''.sch'' file is **not** the primary duplicate-detection mechanism. ''smb_addmsg()'' (the canonical entry point in [[https://gitlab.synchro.net/main/sbbs/-/blob/master/src/smblib/smbadd.c|smbadd.c]]) checks the ''.hash'' file via ''smb_findhash()'' and skips dupe-checking entirely for bases with ''SMB_EMAIL'', ''SMB_NOHASH'', or ''SMB_FILE_DIRECTORY'' set. The ''.sch'' file is still actively maintained by ''email.cpp'' for the mail base and by some legacy import paths, but ''.hash'' is the mechanism for sub-boards and file bases.

==== Whole-base Lock ====

A ''<basecode>.lock'' file is used by ''smb_lock()'' as a //cooperative// exclusive lock on the entire base. It is created with ''O_CREAT | O_EXCL'' (atomic create-or-fail), so concurrent attempts to create it fail until the holder deletes it via ''smb_unlock()''. ''smb_islocked()'' tests for its presence.

This is **separate from** the in-process file locking used during normal reads/writes (''smb_locksmbhdr()'' / ''smb_lockmsghdr()'' apply ''fcntl''-style record locks on the ''.shd'' file itself). The ''.lock'' file is intended for offline maintenance/repair operations — typically by the sysop running ''[[util:smbutil]]'', ''[[util:chksmb]]'', or ''[[util:fixsmb]]'' — that need to exclude the BBS and other tools for the duration.

If a process holding the lock crashes or is killed without removing the lock file, the lock will persist indefinitely. Manually removing a stale ''.lock'' file is safe **only if** no other process is actually operating on the base.

==== Hash File (.hash) ====

Modern duplicate-detection file containing one ''hash_t'' record (64 bytes, ''SIZEOF_SMB_HASH_T'') per fingerprinted source. Each record carries up to four digests (CRC-16, CRC-32, MD5, SHA-1; see [[#hash_flags|hash flags]]) and a ''source'' field identifying what was hashed (''SMB_HASH_SOURCE_BODY'', ''SMB_HASH_SOURCE_MSG_ID'', ''SMB_HASH_SOURCE_FTN_ID'', ''SMB_HASH_SOURCE_SUBJECT'').

For sub-boards, ''smb_addmsg()'' computes the dupe-source set ''SMB_HASH_SOURCE_DUPE'' (''BODY'' | ''MSG_ID'' | ''FTN_ID'') and rejects a new message if any of those hashes match an existing entry. For file bases the relevant hashes (configured per directory) gate uploads. Re-creatable via ''smbutil''.

==== Storage Methods ====

^ Method ^ Constant ^ .sda/.sha used? ^ Speed ^ Disk usage ^ Notes ^
| Self-packing     | ''SMB_SELFPACK'' (0)  | Yes | Slowest imports | Smallest | Reuses freed blocks; no periodic packing needed for typical usage |
| Fast Allocation  | ''SMB_FASTALLOC'' (1) | Yes | Faster imports  | Grows over time; needs ''[[util:smbutil]] p'' periodically | |
| Hyper Allocation | ''SMB_HYPERALLOC'' ((1<<1)) | No  | Fastest imports | Grows; needs ''[[util:smbutil]] p'' periodically; no allocation files | Selectable for sub-boards via the ''SUB_HYPER'' [[config:message_areas#sub-board_advanced_options|toggle]]. The library accepts it for any base, but the BBS's ''smb_storage_mode()'' (in ''src/sbbs3/load_cfg.c'') never selects Hyper for the mail base — mail uses Self-packing or Fast (''SM_FASTMAIL'' system flag). |

===== Message Bases vs. File Bases =====

The same SMB format and library serve two distinct purposes in modern Synchronet:

  * **Message bases** — store messages: e-mail (the system mail base) and conference posts (one base per sub-board).
  * **File bases** — store //file records// (filename, description, uploader, hashes) for the file transfer section. The actual file contents live elsewhere on disk; the SMB base is the metadata catalog. Each "message" in a file base represents one file.

File bases were converted to use the SMB format in **Synchronet v3.19** (see [[history:newfilebase]] for the migration details). Prior to v3.19, file bases used a separate format (''.ixb'' / ''.dat'' / ''.exb'' / ''.dab''). The conversion was automatic via the one-time ''upgrade_to_v319'' utility.

==== Storage Locations ====

^ Type ^ Location ^ Notes ^
| Mail base       | ''[[dir:data]]/mail.s*''       | One per system; ''SMB_EMAIL'' status flag set |
| Sub-board (message) bases | ''[[dir:data]]/subs/<code>.s*''  | One per sub-board, named by the sub-board's [[config:message_areas|internal code]] |
| File bases      | ''[[dir:data]]/dirs/<code>.s*'' | One per file directory, named by the directory's [[config:file_areas|internal code]]; ''SMB_FILE_DIRECTORY'' status flag set |

==== Discriminator ====

The status header ''attr'' field tells client code which kind of base it is reading:

^ Flag ^ Hex ^ Meaning ^
| ''SMB_EMAIL''         | ''0x01'' | This is the mail base (''to''/''from'' indexes hold user numbers, not name CRCs) |
| ''SMB_FILE_DIRECTORY''| ''0x08'' | This is a **file base**, not a message base |

A normal sub-board base has neither flag set.

==== What Differs ====

Although both share the same on-disk file set (''.shd'' / ''.sid'' / ''.sdt'' / ''.sda'' / ''.sha'' / ''.hash'' / ''.ini''), the //meaning// of records differs:

^ Aspect ^ Message base ^ File base ^
| Index record | ''idxrec_t'' (20 bytes; ''SIZEOF_SMB_IDXREC_T'') — ''to''/''from''/''subj'' CRCs (or user numbers for mail; or ballot/file variants via union), ''attr'', ''offset'', ''number'', ''time'' | ''fileidxrec_t'' (128 bytes; ''SIZEOF_SMB_FILEIDXREC_T'') — embedded ''idxrec_t'' (with the //file// union variant: ''size''/''size_ext''), plus ''name[SMB_FILEIDX_NAMELEN+1]'' (64+1 chars indexed) and a ''struct hash_info'' (CRC16/CRC32/MD5/SHA1) |
| Each "record" represents | One message | One file (filename, description, uploader, hashes, etc.) |
| Header fields (typical) | ''SENDER'', ''RECIPIENT'', ''SUBJECT'', ''SMB_SUMMARY'', ''SMB_AUTHOR'', RFC822/FidoNet kludges, … | ''SMB_FILEUPLOADER'' (= ''SENDER''), ''SMB_FILENAME'' (= ''SUBJECT''), ''SMB_FILEDESC'' (= ''SMB_SUMMARY''), ''SMB_TAGS''; optionally ''RECIPIENTLIST'' for [[#user-to-user_file_transfers|user-to-user file transfers]] |
| Body data fields | ''TEXT_BODY'' (and optional ''TEXT_TAIL'') containing the message text | Extended (multi-line) file description text |
| Hash file (''.hash'') | Used (when enabled) for duplicate **message** detection (typically by message body) | Used (when enabled) for duplicate **file** detection — multiple algorithms (CRC16/CRC32/MD5/SHA1) computed at upload time |
| Recipient-based lookup | Via ''to'' index (CRC of name) | Files have no recipient //in the index//, but a file may have a ''RECIPIENTLIST'' header field naming destination users (see [[#user-to-user_file_transfers|user-to-user transfers]] below) |

==== Hash Flags ====

The per-record hash structure (''hash_t'' / ''struct hash_data'') uses these flags to indicate which algorithms have been computed and stored:

^ Flag ^ Hex ^ Hash ^
| ''SMB_HASH_CRC16'' | ''0x01'' | CRC-16 |
| ''SMB_HASH_CRC32'' | ''0x02'' | CRC-32 |
| ''SMB_HASH_MD5''   | ''0x04'' | MD5 |
| ''SMB_HASH_SHA1''  | ''0x08'' | SHA-1 |

By default Synchronet computes all four during upload. Hashing can be disabled per-directory in [[util:scfg|SCFG]] → File Areas → //Toggle Options// (e.g. for very large files or for performance).

==== Field Aliases ====

Modern source uses the following compile-time aliases when writing to or reading from a file base, mapping file-base concepts onto the same header field type codes used by message bases:

^ File-base alias ^ Equals ^ Hex ^ Meaning ^
| ''SMB_FILEUPLOADER'' | ''SENDER''      | ''0x00'' | The user who uploaded the file |
| ''SMB_FILENAME''     | ''SUBJECT''     | ''0x60'' | The filename |
| ''SMB_FILEDESC''     | ''SMB_SUMMARY'' | ''0x61'' | The single-line description ("file summary") |

Long extended descriptions are stored as ''TEXT_BODY'' data fields, the same way long message bodies are stored in message bases.

==== Tools and APIs ====

All SMB-aware tools work on either kind of base:

  * [[util:smbutil]] — operates on any ''.shd'' file (message or file base)
  * [[util:chksmb]] — integrity check
  * [[util:fixsmb]] — index rebuild

JavaScript APIs:
  * ''MsgBase'' class — for message bases
  * ''FileBase'' class — for file bases (added in v3.19; see [[module:fileman|fileman.js]] / ''addfiles.js'' / ''postfile.js'' / ''filelist.js'' / ''delfiles.js'' / ''dupefind.js'')

==== User-to-User File Transfers ====

A file uploaded into a sysop-designated //user-to-user// directory may be addressed to one or more specific destination users (rather than the public file base). This is implemented in the file base by attaching a ''RECIPIENTLIST'' header field to the file's record — a comma-separated list of destination **user numbers**.

  * The in-BBS upload path stores the recipients via ''smb_hfield_str(&f, RECIPIENTLIST, ...)'' (see ''src/sbbs3/upload.cpp'').
  * The ''FileBase'' JavaScript class exposes this as the ''to_list'' property on a file object: a comma-separated list of recipient user numbers (see ''src/sbbs3/js_filebase.cpp'').
  * Recipients are addressed by **user number**, not name — analogous to how the **mail** message base stores user numbers (rather than name CRCs) in its index.
  * The destination users retrieve the file with the ''/D'' command at the transfer menu; the uploader sends with ''/U''.

A file base directory is configured as user-to-user by enabling the appropriate //Toggle Option// in [[util:scfg|SCFG]] → File Areas. The legacy ''xfer.ixt'' file (pre-v3.19) tracked pending user-to-user transfers; existing entries in that file are not migrated by ''upgrade_to_v319'' — see [[history:newfilebase#user-to-user_file_transfers]] for migration notes.

===== Header Field Types =====

A message header consists of a fixed-length prefix followed by zero or more //header field records//. Each field has the form:

<code c>
typedef struct {        /* Header field */
    uint16_t  type;     /* field type code */
    uint16_t  length;   /* length of the field data following this struct */
} hfield_t;
/* the struct is immediately followed in the file by 'length' bytes of data */
</code>

Header fields hold sender/recipient information, subject, network addresses, kludge lines, and many other attributes. Field type codes are 16-bit and grouped by range. See [[https://gitlab.synchro.net/main/sbbs/-/blob/master/src/smblib/smbdefs.h|smbdefs.h]] for the canonical list of type codes and their on-disk numeric values.

^ Range ^ Category ^ Examples ^
| ''0x00''-''0x0e'' | Sender    | ''SENDER'', ''SENDERAGENT'', ''SENDERNETTYPE'', ''SENDERNETADDR'', ''SENDEREXT'', ''SENDERPOS'', ''SENDERORG'', ''SENDERIPADDR'', ''SENDERHOSTNAME'', ''SENDERPROTOCOL'', ''SENDERPORT'', ''SENDERUSERID'', ''SENDERTIME'', ''SENDERSERVER'' |
| ''0x10''-''0x16'' | Author    | ''SMB_AUTHOR'', ''SMB_AUTHOR_ORG'' |
| ''0x30''-''0x37'' | Recipient | ''RECIPIENT'', ''RECIPIENTAGENT'', ''RECIPIENTNETTYPE'', ''RECIPIENTNETADDR'', ''RECIPIENTEXT'', ''RECIPIENTPOS'', ''RECIPIENTORG'', ''RECIPIENTLIST'' |
| ''0x48''           | Forwarding | ''FORWARDED'' |
| ''0x60''-''0x6a'' | Subject / metadata | ''SUBJECT'', ''SMB_SUMMARY'', ''SMB_COMMENT'', ''SMB_CARBONCOPY'', ''SMB_GROUP'', ''SMB_EXPIRATION'', ''SMB_COST'', ''SMB_EDITOR'', ''SMB_TAGS'', ''SMB_COLUMNS'' |
| ''0x70''-''0x72'' | File attach | ''FILEATTACH'', ''DESTFILE'', ''FILEATTACHLIST'' |
| ''0xa0''-''0xaa'' | FidoNet kludges | ''FIDOCTRL'' (generic), ''FIDOAREA'', ''FIDOSEENBY'', ''FIDOPATH'', ''FIDOMSGID'', ''FIDOREPLYID'', ''FIDOPID'', ''FIDOFLAGS'', ''FIDOTID'', ''FIDOCHARSET'', ''FIDOBBSID'' |
| ''0xb0''-''0xb8'' | RFC822 / Internet | ''RFC822HEADER'', ''RFC822MSGID'', ''RFC822REPLYID'', ''RFC822TO'', ''RFC822FROM'', ''RFC822REPLYTO'', ''RFC822CC'', ''RFC822ORG'', ''RFC822SUBJECT'' |
| ''0xe0''           | Polls    | ''SMB_POLL_ANSWER'' (poll answer; the subject is the question) |
| ''0xf1''-''0xff'' | Other / unused | ''UNKNOWN'', ''UNKNOWNASCII'', ''UNUSED'' |

The file-base entry types reuse some of the above with aliases: ''SMB_FILENAME'' = ''SUBJECT'', ''SMB_FILEDESC'' = ''SMB_SUMMARY'', ''SMB_FILEUPLOADER'' = ''SENDER''.

==== RECIPIENT and related ====

  * ''RECIPIENT'' — single name or full network address of the message recipient (US-ASCII or UTF-8). **Every message must have a single ''RECIPIENT'' header field.**
  * ''RECIPIENTLIST'' — original comma-separated list of all recipients (excluding CCs). Optional.
  * ''RFC822TO'' — MIME-encoded version of ''RECIPIENTLIST''. Should not exist without ''RECIPIENTLIST''.

===== Data Field Types =====

A //data field record// (''dfield_t'') in a message header points to a region of the .sdt file:

<code c>
typedef struct {        /* Data field */
    uint16_t  type;     /* data field type code */
    uint32_t  offset;   /* byte offset within .sdt */
    uint32_t  length;   /* length of the data region */
} dfield_t;
</code>

Currently defined data field types (''dfield_t.type''):

^ Type ^ Hex ^ Holds ^
| ''TEXT_BODY''   | ''0x00'' | The plain message body text (usually one per message; multiples are concatenated when displayed) |
| ''TEXT_TAIL''   | ''0x02'' | Tear-line / origin / SEEN-BY block (FidoNet) |

File attachments are referenced via the [[#header_field_types|header field]] ''FILEATTACH'', not via a dedicated data field type.

===== Message Attributes =====

The 16-bit ''attr'' field of the ''idxrec_t'' / ''msghdr_t'' carries the primary message-level flags. The ''auxattr'' field adds extended (auxiliary) flags, and ''netattr'' adds network-related flags.

Primary attribute flags (''attr''):

^ Flag ^ Hex ^ Meaning ^
| ''MSG_PRIVATE''      | ''0x0001'' | Private message |
| ''MSG_READ''         | ''0x0002'' | Recipient has read |
| ''MSG_PERMANENT''    | ''0x0004'' | Don't auto-delete by age/count |
| ''MSG_LOCKED''       | ''0x0008'' | //(deprecated, never used)// |
| ''MSG_DELETE''       | ''0x0010'' | Marked for deletion |
| ''MSG_ANONYMOUS''    | ''0x0020'' | Author is anonymous |
| ''MSG_KILLREAD''     | ''0x0040'' | Delete after recipient reads |
| ''MSG_MODERATED''    | ''0x0080'' | Awaiting moderator validation |
| ''MSG_VALIDATED''    | ''0x0100'' | Sysop-validated |
| ''MSG_REPLIED''      | ''0x0200'' | Recipient has replied |
| ''MSG_NOREPLY''      | ''0x0400'' | No replies (or bounces) should be sent to the sender |
| ''MSG_UPVOTE''       | ''0x0800'' | Upvote (poll vote) |
| ''MSG_DOWNVOTE''     | ''0x1000'' | Downvote (poll vote) |
| ''MSG_POLL''         | ''0x2000'' | Message is a poll |
| ''MSG_SPAM''         | ''0x4000'' | Flagged as SPAM |
| ''MSG_FILE''         | ''0x8000'' | This is a file (file-base entry, not a message) |
| ''MSG_VOTE''         | ''0x1800'' | Composite: ''MSG_UPVOTE \| MSG_DOWNVOTE'' |
| ''MSG_POLL_CLOSURE'' | ''0x3800'' | Composite: ''MSG_POLL \| MSG_VOTE'' |

Auxiliary attribute flags (''auxattr''):

^ Flag ^ Hex ^ Meaning ^
| ''MSG_FILEREQUEST''  | ''0x00000001'' | FidoNet file request |
| ''MSG_FILEATTACH''   | ''0x00000002'' | Carries attached file(s) (paths/names in subject) |
| ''MSG_MIMEATTACH''   | ''0x00000004'' | One or more MIME-embedded attachments |
| ''MSG_KILLFILE''     | ''0x00000008'' | Delete file(s) when sent |
| ''MSG_RECEIPTREQ''   | ''0x00000010'' | Return receipt requested |
| ''MSG_CONFIRMREQ''   | ''0x00000020'' | Confirmation receipt requested |
| ''MSG_NODISP''       | ''0x00000040'' | Message may not be displayed to user |
| ''MSG_FIXED_FORMAT'' | ''0x00000080'' | Preformatted message body text |
| ''MSG_HFIELDS_UTF8'' | ''0x00002000'' | Message header fields are UTF-8 encoded |
| ''POLL_CLOSED''      | ''0x01000000'' | Closed to voting |

Network attribute flags (''netattr''):

^ Flag ^ Hex ^ Meaning ^
| ''NETMSG_LOCAL''     | ''0x0001'' | Locally created |
| ''NETMSG_INTRANSIT'' | ''0x0002'' | In transit (for batch processing) |
| ''NETMSG_SENT''      | ''0x0004'' | Sent to remote |
| ''NETMSG_KILLSENT''  | ''0x0008'' | Kill when sent |
| ''NETMSG_HOLD''      | ''0x0020'' | Hold for pick-up |
| ''NETMSG_CRASH''     | ''0x0040'' | High priority (crash mail) |
| ''NETMSG_IMMEDIATE'' | ''0x0080'' | Send immediately, ignore restrictions |
| ''NETMSG_DIRECT''    | ''0x0100'' | Send directly to destination |

For the complete and current set, see [[https://gitlab.synchro.net/main/sbbs/-/blob/master/src/smblib/smbdefs.h|smbdefs.h]].

===== Translation Types =====

A 16-bit ''xlat'' code identifies an encoding/compression transform applied to data. Multiple translations may be chained.

Currently defined translation types (''enum smb_xlat_type''):

^ Code ^ Name ^ Meaning ^
| ''0'' | ''XLAT_NONE'' | No translation; also the terminator of an xlat array |
| ''9'' | ''XLAT_LZH''  | LHarc (LHA) Dynamic Huffman coding |

Apply translations in **storage order** to encode; reverse the order to decode.

===== Agent Types =====

The ''SENDERAGENT'' and ''RECIPIENTAGENT'' header fields identify the //software agent// type at each end (''enum smb_agent_type''):

^ Code ^ Name ^ Meaning ^
| ''0'' | ''AGENT_PERSON''      | A human user |
| ''1'' | ''AGENT_PROCESS''     | Unknown automated process |
| ''2'' | ''AGENT_SMBUTIL''     | Imported via Synchronet [[util:smbutil|SMBUTIL]] |
| ''3'' | ''AGENT_SMTPSYSMSG''  | Synchronet [[server:mail|SMTP server]] system message |

===== Network Types =====

''SENDERNETTYPE'' / ''RECIPIENTNETTYPE'' identify which network's address format is in the corresponding ''*NETADDR'' field (''enum smb_net_type''):

^ Code ^ Name ^ Network ^
| ''0'' | ''NET_NONE''     | Local message (no network) |
| ''1'' | ''NET_UNKNOWN''  | Unknown network type |
| ''2'' | ''NET_FIDO''     | FidoNet address, ''faddr_t'' format (4D) |
| ''3'' | ''NET_POSTLINK'' | Imported with UTI driver |
| ''4'' | ''NET_QWK''      | QWK networked message |
| ''5'' | ''NET_INTERNET'' | Internet e-mail, netnews, etc. |

===== Message Storage Pseudo-Code =====

To **add** a new message to a base (the ''smb_addmsg()'' flow in [[https://gitlab.synchro.net/main/sbbs/-/blob/master/src/smblib/smbadd.c|smbadd.c]]):

  - Auto-create the base if the ''.shd'' file is empty (''smb_create()'').
  - Lock the SMB header (exclusive).
  - Read the ''smbstatus_t''.
  - Compute the new message's serial number as ''status.last_msg + 1''.
  - **Unless** the base has ''SMB_EMAIL'', ''SMB_NOHASH'', or ''SMB_FILE_DIRECTORY'' set, compute the dupe-source hashes (''smb_msghashes()'' for ''SMB_HASH_SOURCE_DUPE'' = ''BODY'' | ''MSG_ID'' | ''FTN_ID'') and check against the ''.hash'' file (''smb_findhash()''). If a match is found, abort with ''SMB_DUPE_MSG''.
  - Apply translations to the body in **storage order** (the ''xlat[]'' chain — only ''XLAT_LZH'' is currently implemented).
  - Allocate space in the ''.sdt'' file for each data field (text body, tail) according to the storage method:
    * **Hyper Allocation** (''smb_hallocdat()'') — append at end of file; no .sda update.
    * **Fast Allocation** (''smb_fallocdat()'') — append at end; record refcount in ''.sda''.
    * **Self-packing** (''smb_allocdat()'') — search ''.sda'' for a contiguous free run of the required size; allocate there. Append if no fit.
  - Build the ''msghdr_t'' with the ''dfield[]'' entries pointing into the .sdt regions and the variable-length header field records.
  - Allocate a header-aligned region in ''.shd'' (''smb_hallochdr()'' / ''smb_fallochdr()'' / ''smb_allochdr()'').
  - Write the header (''smb_putmsg()'') — this writes the ''msghdr_t'' + dfields + header fields, then writes the corresponding ''idxrec_t'' to ''.sid''.
  - Increment ''status.last_msg'' and ''status.total_msgs''; write status (''smb_putstatus()'').
  - Add the new message's hashes to ''.hash'' (when applicable).
  - Unlock the SMB header.

===== Message Retrieval Pseudo-Code =====

To **read** a message by index entry:

  - Open the ''.shd'' file (shared lock).
  - From the ''idxrec_t'', take the ''offset'' and seek into ''.shd''.
  - Read the ''msghdr_t'' header; iterate the trailing ''hfield_t'' records.
  - For each ''dfield_t'' entry of type ''TEXT_BODY'' (etc.), seek into ''.sdt'' and read the data, including its ''xlat[]'' prefix.
  - Apply translations in **reverse** order to recover the original data.
  - Concatenate body fields (if multiple).

===== SMBLIB =====

**SMBLIB** is the C library Synchronet uses to read and write SMB bases. Third-party programs may link against it (LGPL) rather than re-implement the format from spec. The library lives in [[https://gitlab.synchro.net/main/sbbs/-/tree/master/src/smblib|src/smblib/]] and exposes ~140 public functions covering opening/closing bases, allocating / freeing blocks, adding messages, reading messages, dump/utility helpers, and CRC/hash maintenance.

Key files (see GitLab for current source):

^ File ^ Purpose ^
| [[https://gitlab.synchro.net/main/sbbs/-/blob/master/src/smblib/smbdefs.h|smbdefs.h]]    | All format constants — field type codes, attribute flags, agent/network/media types, error codes, structure definitions. **The canonical reference for current values.** |
| [[https://gitlab.synchro.net/main/sbbs/-/blob/master/src/smblib/smblib.h|smblib.h]]      | Public function prototypes |
| [[https://gitlab.synchro.net/main/sbbs/-/blob/master/src/smblib/smblib.c|smblib.c]]      | Library implementation (open/close, headers, indexing) |
| [[https://gitlab.synchro.net/main/sbbs/-/blob/master/src/smblib/smballoc.c|smballoc.c]]  | Block allocation (Self-pack, Fast, Hyper) |
| [[https://gitlab.synchro.net/main/sbbs/-/blob/master/src/smblib/smbadd.c|smbadd.c]]      | Adding messages |
| [[https://gitlab.synchro.net/main/sbbs/-/blob/master/src/smblib/smbfile.c|smbfile.c]]    | File-base entries |
| [[https://gitlab.synchro.net/main/sbbs/-/blob/master/src/smblib/smbhash.c|smbhash.c]]    | Hash file maintenance |
| [[https://gitlab.synchro.net/main/sbbs/-/blob/master/src/smblib/smbstr.c|smbstr.c]]      | String / header field helpers |
| [[https://gitlab.synchro.net/main/sbbs/-/blob/master/src/smblib/smbdump.c|smbdump.c]]    | Dump utility helpers (used by ''[[util:smbutil]]'') |

For working examples of message storage and retrieval, see ''[[util:smbutil]]'' (''[[https://gitlab.synchro.net/main/sbbs/-/blob/master/src/sbbs3/smbutil.c|src/sbbs3/smbutil.c]]'') and the relevant message-handling code in ''src/sbbs3/'' (''postmsg.cpp'', ''readmsgs.cpp'', ''writemsg.cpp'').

===== Performance Notes =====

  * **Self-packing** is the default for typical sub-boards. It minimizes disk usage; import speed is bounded by allocation searches.
  * **Fast Allocation** trades disk usage for speed by appending; needs ''[[util:smbutil]] p'' periodically.
  * **Hyper Allocation** is fastest for large echomail imports but never reuses freed space — packing is mandatory.
  * **Mail base** must use Self-packing or Fast Allocation (Hyper is not supported).
  * **Duplicate checking** (''max_crcs'' > 0 or ''.hash'' enabled) adds a small per-message cost; disable on bases that don't need it (e.g. transient outbound bases).
  * **CRC index** keys (''to'', ''from'', ''subj'') let readers find messages by name/subject without opening every header — keep ''.sid'' in sync after operations.

===== Bibliography =====

  * Original SMB v1.21 spec: [[https://synchro.net/docs/smb.html]]
  * SMBLIB source: [[https://gitlab.synchro.net/main/sbbs/-/tree/master/src/smblib]]
  * FidoNet Technical Standards: [[http://ftsc.org/]]
  * RFC 5322 (Internet Message Format): [[https://datatracker.ietf.org/doc/html/rfc5322]]
  * RFC 2045 / 2046 (MIME): [[https://datatracker.ietf.org/doc/html/rfc2045]]

===== Implementations =====

  * Synchronet (BBS, mail server, NNTP server, gateways) — full implementation, all features
  * [[util:smbutil]], [[util:chksmb]], [[util:fixsmb]] — bundled utilities
  * Synchronet's [[server:mail|SMTP/POP3/IMAP server]] — SMB-backed
  * Synchronet's [[service:nntp|NNTP server]] — SMB-backed
  * Third-party readers and gateways may use SMBLIB or implement the spec independently

===== See Also =====
  * [[ref:|Reference Library]]
  * [[util:smbutil]] — message base maintenance utility
  * [[util:chksmb]] — integrity checker
  * [[util:fixsmb]] — index rebuilder
  * [[ref:fidonet]] — FidoNet specifications
  * [[ref:qwk]] — QWK packet format
  * [[https://gitlab.synchro.net/main/sbbs/-/tree/master/src/smblib|SMBLIB source on GitLab]]

{{tag>smb messagebase reference spec}}
