# NNTPSwitch v16.9.0-rc5 

> This document is intended as the repository README for the current `v16.9.0-rc5` tree. 

## Status

`v16.9.0-rc5` is a release candidate.

It is a heavily modernized fork of **NNTPSwitch 0.12+**, the last public production source tree published after News-Service.com was shut down.  The upstream repository is still useful as the historical reference point, but this tree has diverged substantially in architecture, security model, packaging, operational tooling, clustering, overview handling, licensing, testing and deployment workflow.

Important release notes:

- `v16.9.0-rc5` keeps the NNTP command surface broadly compatible with the original daemon, but many commands now have new internal paths.
- The production package is split from lab/benchmark tooling.
- Commercial builds intentionally fail `nntpswitch-pre-rc-check --commercial-release` while the development public license key is still embedded.
- Full 20000+ concurrent-user validation must still be performed in the operator's real master/slave lab or a dedicated load-test environment.

## Executive summary

| Area | Upstream 0.12+ | v16.9.0-rc5 |
|---|---:|---:|
| Manifest entries | 58 | 186 entries / 185 unique |
| Runtime `nntpswitch.conf` directives | 56 observed | 138 |
| Runtime directives retained | — | 45 |
| New runtime directives | — | 93 |
| Retired/moved runtime directives | — | 11 |
| NNTP command names | Same family | Same command table, extended behaviour |
| Overview strategy | proxied backend overview | local Overview Union, generation workflow, LMDB/sqlite storage |
| Backend connections | direct/proxied per client flow | optional per-client persistent backend pool |
| Article cache | no read-through article cache | optional disk article cache |
| Auth password file | plaintext/case-insensitive comparison | PBKDF2 hash verification + constant-time comparison |
| SQL auth | string-built SQL in original MySQL auth | parameterized MySQL/MariaDB and PostgreSQL auth |
| Accounting | syslog/radius/none legacy modules | MySQL/MariaDB, PostgreSQL and none |
| Post filtering | none in core | external Perl postfilter bridge |
| Cluster | none | JSON-RPC clusterd, generations, push/commit/rollback, TLS/mTLS, HMAC |
| Licensing | none in public core | signed commercial license token + online check + offline cache |
| Packaging | old Debian stub/init.d style | modern Debian package, systemd, logrotate, sysctl, package split |
| Observability | legacy stats scripts | Prometheus exporter, doctor, ops inspector, backend health |
| Test/lab tooling | small legacy testsuite | smoke tests, crosshost tests, heavy simulators, stress/bench tools |

## Compatibility at a glance

The fork is source-related to upstream NNTPSwitch, but it is **not** a drop-in replacement for every old configuration.

Mostly compatible or intentionally preserved:

- Core daemon concept: NNTP reader/content router in C.
- Main NNTP command table: `MODE`, `AUTHINFO`, `LIST`, `GROUP`, `LISTGROUP`, `OVER`, `XOVER`, `XHDR`, `XPAT`, `STAT`, `HEAD`, `BODY`, `ARTICLE`, `POST`, `NEXT`, `LAST`, `DATE`, `HELP`, `QUIT`, `STATISTICS`, `IHAVE`, plus the legacy HTTP probe `GET`.
- Main config files: `nntpswitch.conf`, `access.conf`, `servers.conf`, `overview.fmt`, `passwd`, `active`, `newsgroups`.
- Many legacy runtime options remain present: listener, timeout, log, posting, buffer and retry settings.

Changed or not drop-in compatible:

- `servers.conf` no longer uses the old `level` style in the current parser; routing is driven by backend blocks, group patterns, policy and newer backend controls.
- Several legacy auth/accounting modules were removed or replaced.
- Global SSL key/password/CA options from the old config are moved into vhost/profile-style access configuration.
- Remote HTTP/JSON ACL/rate-limit endpoints from the old commercial configuration are retired.
- Production deployment now assumes systemd paths, `/srv/nntpswitch`, `/etc/nntpswitch`, `/var/lib/nntpswitch`, `/var/cache/nntpswitch` and `/var/spool/nntpswitch`.
- Large-scale overview operation now expects generated overview data rather than repeatedly fanning every heavy XOVER request to upstream backends.

## NNTP protocol and command behaviour

The command names are intentionally conservative.  The daemon still presents itself as an NNTP server and retains the legacy command table.  The important change is not the command list, but the implementation behind several commands.

### Commands whose behaviour changed significantly

| Command family | Original behaviour | v16.9.0-rc5 behaviour |
|---|---|---|
| `OVER` / `XOVER` / `XZVER` | proxied to a selected backend | can be served from local Overview Union; range caps protect the frontend |
| `LISTGROUP` | proxied/listed from backend data | can be served from local generation data with line caps |
| `ARTICLE` / `HEAD` / `BODY` | backend fetch path | quota/accounting checks plus optional read-through article cache |
| `POST` | backend post path with legacy checks | posting size/header/group limits plus optional external postfilter |
| `AUTHINFO` | legacy module-based auth | updated auth modules, PBKDF2 password hashes, SQL prepared statements, remote auth hardening |
| `LIST MOTD` | not a modern structured MOTD path | returns one random MOTD section, ignores comments/separators |
| `STATISTICS` | legacy stats | retained command with broader operational ecosystem around stats/metrics |

## Major new subsystems

### 1. Commercial licensing core

New files:

- `licensing.c`
- `fingerprint.c`
- `fingerprint.h`
- `genkey.c`
- `license_public_key.h`

Capabilities:

- Machine fingerprint generation (`NNSW-C1-*`).
- Signed license token verification (`NNSW-LIC1-*`).
- Mandatory signed online check in commercial builds (`NNSW-CHECK1-*`).
- Offline signed cache with bounded validity.
- TLS peer/host verification for license server calls.
- Enforcement of license properties such as maximum connections and feature flags.
- Release gate: `nntpswitch-pre-rc-check --commercial-release` intentionally fails while the public key marker is present but no valid license issued.

New runtime directives:

- `LicenseServer`
- `LicenseFingerprint`
- `LicenseKey`
- `LicenseCacheFile`
- `LicenseConnectTimeout`

### 2. Overview Union and generation workflow

New core file:

- `overview_union.c`

New tooling includes:

- `nntpswitch-generation.py`
- `nntpswitch-overview-admin.py`
- `nntpswitch-overview-refresh.py`
- `nntpswitch-overview-kv-admin.py`
- `nntpswitch-overview-kv-export.py`
- `nntpswitch_mdbx.py`
- `nntpswitch-rebuild-generation-safe.sh` in the lab-tools package

Capabilities:

- Local overview service for multi-backend union groups.
- Storage backends: sqlite range-sharded and LMDB.
- Generation-style deployment using `current` symlinks/manifests.
- Frontend mode for read-only generated overview data.
- Startup modes and generation-required mode.
- Caps for XOVER ranges and LISTGROUP output.
- Fallback behaviour for missing overview/article data.
- Retention controls for generated data.

Important runtime directives include:

- `OverviewUnionEnable`
- `OverviewUnionMode`
- `OverviewUnionStorage`
- `OverviewUnionRoot`
- `OverviewUnionCurrent`
- `OverviewUnionPublishRoot`
- `OverviewUnionGenerationFile`
- `OverviewUnionReadOnly`
- `OverviewUnionRequireGeneration`
- `OverviewUnionMaxOverviewRange`
- `OverviewUnionMaxListGroupLines`
- `OverviewUnionKVMapSizeMB`
- `OverviewUnionStartupMode`
- `OverviewUnionRefreshCommand`
- `OverviewUnionRefreshInterval`
- `OverviewUnionRetentionDays`

### 3. Backend connection pool

New core file:

- `backend_pool.c`

Capabilities:

- Optional per-client backend socket reuse.
- Idle TTL.
- Max idle sockets per client.
- Optional validation on reuse.
- Strict server matching to avoid cross-server state leakage.
- Reserve-connection behaviour.
- Limits by article count and bytes per socket.
- Behaviour controls when every eligible backend is full: fallback, queue or reject.

Runtime directives:

- `BackendPoolEnable`
- `BackendPoolMaxIdlePerClient`
- `BackendPoolIdleTTL`
- `BackendPoolValidateOnReuse`
- `BackendPoolStrictServerMatch`
- `BackendPoolReserveConnections`
- `BackendPoolMaxArticlesPerSocket`
- `BackendPoolMaxBytesPerSocketMB`
- `BackendFullAction`
- `BackendWaitTimeout`
- `BackendQueueSleepMs`

### 4. Article cache

New core file:

- `article_cache.c`

Capabilities:

- Optional read-through disk cache for `ARTICLE`, `HEAD` and `BODY`.
- TTL-based cache retention.
- Max object size.
- Max total cache size.
- Optional logging for hits, stores and misses.
- Admin/prune support through tooling and systemd timer.

Runtime directives:

- `ArticleCacheEnable`
- `ArticleCacheRoot`
- `ArticleCacheMaxObjectMB`
- `ArticleCacheTTL`
- `ArticleCacheMaxBytesMB`
- `ArticleCacheLogHits`
- `ArticleCacheLogStores`
- `ArticleCacheLogMisses`

### 5. Quota and rate-limit enforcement

New core files:

- `quota.c`
- `quota.h`

Capabilities:

- Daily and monthly quota fields.
- Quota exceeded actions.
- Throttle/deny/disconnect style policy.
- Per-user and per-profile bandwidth caps.
- Accounting integration.

Profile/config keys include:

- `DailyQuota` / `DailyQuotaBytes`
- `MonthlyQuota` / `MonthlyQuotaBytes`
- `QuotaExceededAction`
- `QuotaExceededSpeed` / `QuotaExceededBPS` / `QuotaExceededKbit` / `QuotaExceededMbit`
- `MaxUserBPS` / `MaxUserKbit` / `MaxUserMbit`
- `MaxProfileBPS` / `MaxProfileKbit` / `MaxProfileMbit`
- `Limits`

### 6. Postfilter bridge

New files:

- `postfilter.c`
- `perl/nntpswitch/PostFilter.pm`
- `perl/postfilter_policy.pl`
- `perl/postfilter_accept_all.pl`
- `tests/postfilter/*`

Capabilities:

- External policy filter for posted articles.
- Accept/reject/tempfail style outcomes.
- Optional article modification.
- Timeout and output-size controls.
- Separate temp and spool paths.
- Test articles and Perl test runner.

Runtime directives:

- `PostFilterEnable`
- `PostFilterCommand`
- `PostFilterTimeout`
- `PostFilterFailAction`
- `PostFilterAllowModify`
- `PostFilterTempDir`
- `PostFilterSpoolDir`
- `PostFilterMaxOutputBytes`

### 7. Posting and IHAVE safety limits

New protocol-abuse controls:

- `MaxPostBytes`
- `MaxIHAVEBytes`
- `MaxPostLines`
- `MaxHeaderBytes`
- `MaxPostGroups`

These protect the frontend from oversized articles, oversized IHAVE input, oversized headers and excessive crossposting.

### 8. Cluster control plane

New daemon/tooling:

- `nntpswitch-clusterd.py`
- `nntpswitch-cluster-admin.py`
- `nntpswitch-cluster-cert-admin.py`
- `nntpswitch-cluster-secret-admin.py`
- `nntpswitch-cluster-lab.py` in lab-tools

Capabilities:

- JSON-RPC control plane for frontend nodes.
- Native upload of generated overview data.
- Prepare/verify/commit/rollback style generation deployment.
- Commit journals and recovery support.
- Retention/prune workflow.
- Reload-on-commit support through frontend SIGHUP.
- Shared-secret mode for simple labs.
- HMAC-SHA256 mode with key-id, timestamp, nonce and body hash.
- Anti-replay cache.
- TLS/mTLS support with local CA tooling.
- Automated secret rotation and certificate renewal scaffolding.

Cluster runtime directives include:

- `ClusterListenAddress`
- `ClusterListenPort`
- `ClusterSharedSecretFile`
- `ClusterAuthMode`
- `ClusterSecretKeyringFile`
- `ClusterAuthMaxClockSkewSeconds`
- `ClusterNonceRoot`
- `ClusterNativeUploadEnable`
- `ClusterNativeUploadRoot`
- `ClusterNativeUploadMaxFiles`
- `ClusterNativeUploadMaxBytes`
- `ClusterNativeUploadMaxChunkBytes`
- `ClusterReloadOnCommit`
- `ClusterReloadWaitSeconds`
- `ClusterMaxJSONBytes`
- `ClusterTLSEnable`
- `ClusterTLSCertFile`
- `ClusterTLSKeyFile`
- `ClusterTLSCAFile`
- `ClusterTLSRequireClientCert`
- `ClusterStateRoot`
- `ClusterRetentionKeepLast`

### 9. Operational health and metrics

New operational components:

- `nntpswitch-backend-health.py`
- `nntpswitch-metrics-exporter.py`
- `nntpswitch-doctor.py`
- `nntpswitch-ops-inspector.py`
- `nntpswitch-preflight.py`
- `nntpswitch-pre-rc-check.py`

Capabilities:

- Backend health state file.
- Prometheus metrics exporter.
- Dependency validation.
- TLS material validation.
- Pre-release sanity checks.
- Operational inspection and stale state detection.

New runtime directives:

- `BackendHealthFile`
- `OpsMaxStatsAge`
- `OpsMaxBackendHealthAge`

## Authentication and accounting changes

### Shipped authentication modules

Current tree:

- `auth_passwd.c`
- `auth_mysql.c`
- `auth_postgres.c`
- `auth_remote.c`
- `auth_ignore.c`

Removed/retired from the old tree:

- `auth_ldap.c`
- `auth_pop3.c`
- `auth_radius.c`
- old HTTP/JSON/remote config support around `curl.c` / `rconfig.c`

### Password file authentication

Original behaviour:

- Password file authentication compared username/password fields directly.
- Password comparison was plaintext and case-insensitive.

Current behaviour:

- Passwords are generated/verified through `password_hash.c`.
- PBKDF2 hashes are supported.
- Secret comparison is constant-time where appropriate.
- `nntpswitch-mkpasswd.pl` is provided to generate password hashes.

### MySQL/MariaDB authentication

Original behaviour:

- The original `auth_mysql.c` built a SQL query with string formatting using the supplied username and password.
- The original users table model used plaintext password matching.

Current behaviour:

- SQL identifiers are validated.
- User-supplied values are passed through prepared statements/bound parameters.
- Password hash verification is used instead of plaintext matching.
- Optional policy table integration is available.

### PostgreSQL authentication

Current behaviour:

- PostgreSQL auth uses parameterized query execution.
- Policy table loading is supported when configured.

### Remote auth

Current rc5 hardening:

- Remote auth server replies are read with one byte reserved for NUL termination.
- Raw `read_socket()` buffers are explicitly terminated before parsing.
- The remote authentication socket is closed on accept, reject and error paths.

### Accounting modules

Current tree:

- `acct_none.c`
- `acct_mysql.c`
- `acct_postgres.c`

Removed/retired:

- `acct_radius.c`
- `acct_syslog.c`

Current SQL accounting supports session/event style data suitable for portal/billing integration.

## `access.conf` changes

The current access configuration keeps the legacy idea of wildmats, aliases, profiles, auth masks and ACL rules, but extends it substantially.

### Top-level sections

Retained from original style:

- `wildmat`
- `alias`
- `args`
- `profile ... end`
- `include`
- `auth`
- `acl`

Added/current:

- `pfxmap`
- `vhost ... end`

### Current profile keys parsed by `access.c`

- `ReadPat`
- `PostPat`
- `Banner`
- `ReplaceBanner`
- `MaxConnections`
- `MaxUserConcurrent`
- `MaxHostConcurrent`
- `Slave`
- `MaxUsers`
- `MaxSessionTime`
- `MaxSessionBytes`
- `DailyQuota` / `DailyQuotaBytes`
- `MonthlyQuota` / `MonthlyQuotaBytes`
- `QuotaExceededAction`
- `QuotaExceededSpeed` / `QuotaExceededBPS` / `QuotaExceededKbit` / `QuotaExceededMbit`
- `MaxUserBPS` / `MaxUserKbit` / `MaxUserMbit`
- `MaxProfileBPS` / `MaxProfileKbit` / `MaxProfileMbit`
- `XComplaintsTo`
- `Organization`
- `ForceOrganization`
- `AddNNTPPostingHost`
- `MaxCross`
- `NoControl`
- `ArticleSearch`
- `Hostname`
- `PostingPath` / `PathIdentity`
- `PostingIdentity`
- `ResellerId`
- `ResellerName`
- `PublicAbuse`
- `MainAbuse`
- `ComplianceHeaderN_Name` / `ComplianceHeaderN_Value`
- `CustomHeaderN_Name` / `CustomHeaderN_Text`
- `FooterFile`
- `UserRLFactorU`
- `UserRLFactorD`
- `ProfileRLFactorD`
- `ProfileRLFactorU`
- `ClientReadTimeout`
- `Limits` / `Limit`
- `Accounting`
- `AccountingPrefix`
- `Disabled`
- `Retention`
- `XHeaderSSLUpload`
- `AllowSSL`
- `PrefixMap`
- `FailProfile`

### Current vhost keys parsed by `access.c`

- `Banner`
- `ReplaceBanner`
- `AddSuffix`
- `SSLKeyfile`
- `SSLCertFile`
- `SSLPrivateKeyFile`
- `SSLMinVersion`
- `SSLCAList`
- `SSLPassword`

### Notable old profile/config behaviour changed

- `SingleHostUser` appears in the upstream sample profile, but is not parsed by current `access.c`. Use `MaxHostConcurrent`, `MaxUserConcurrent`, ACLs and authentication policy instead.
- SSL certificate/key settings are now naturally represented in `vhost` configuration instead of only as global legacy config options.
- Reseller, abuse/contact, compliance header and retention metadata are first-class profile fields.

## `servers.conf` changes

The current backend configuration keeps server blocks but extends backend policy and removes older assumptions.

### Current server keys parsed by `server.c`

- `hostname`
- `groups`
- `Username`
- `Password`
- `timeout`
- `port`
- `maxconnections`
- `weight`
- `backendwaittimeout`
- `backendqueuesleepms`
- `backendfullaction`
- `mirrormode`
- `failoveron430`
- `failoveron423`
- `type`
- `policy`
- `activetimes`
- `descriptions`
- `splitlist`

### Backend policy changes

Original policy values:

- `single`
- `backup`
- `balance`

Current policy values:

- `single`
- `backup`
- `balance`
- `union`

New or changed backend behaviour:

- `Username` / `Password` for upstream AUTHINFO.
- `maxconnections` cap per backend.
- `weight` for weighted balancing.
- `backendfullaction` with `fallback`, `queue` or `reject` behaviour.
- `backendwaittimeout` and `backendqueuesleepms` for queue mode.
- `mirrormode` with strict/loose semantics.
- `failoveron430` and `failoveron423` to retry another backend for missing article responses.
- `union` policy for multi-backend group views.

Old `level`-based examples from the historical `servers.conf` are not part of the current parser.

## Runtime configuration directive delta

### Retained runtime directives from upstream

- `ListenPorts`
- `BindAddress`
- `RunAsUser`
- `PidFile`
- `StatsFile`
- `StatsFilePeriod`
- `ConnDumpFile`
- `BasePath`
- `AccessFile`
- `ServerFile`
- `OverviewFmtFile`
- `ActiveFile`
- `NewsgroupsFile`
- `ServerReadTimeout`
- `ServerConnectTimeout`
- `ClientReadTimeout`
- `MaxConnections`
- `LogWriteclient`
- `LogReadserver`
- `LogWriteserver`
- `LogCommands`
- `LogMissing`
- `LogAuthInfo`
- `PostServer`
- `PostServerPort`
- `PostXTraceHeader`
- `ModeratorDomain`
- `MailerPath`
- `BufSize`
- `SockBufSize`
- `ListenBackLog`
- `DenyNoIndexField`
- `DownDelay`
- `RetryDelay`
- `AccountingDelay`
- `RatePeriod`
- `DNSLookups`
- `DropUnknown`
- `DisableFile`
- `DisableText`
- `NoSuchArticleRetries`
- `LocalDreader`
- `CoreBind`
- `EnableSSL`
- `ListenSSLPorts`

### New runtime directives in v16.9.0-rc5

- `ClusterListenAddress`
- `ClusterListenPort`
- `ClusterSharedSecretFile`
- `ClusterAuthMode`
- `ClusterSecretKeyringFile`
- `ClusterAuthMaxClockSkewSeconds`
- `ClusterNonceRoot`
- `ClusterNativeUploadEnable`
- `ClusterNativeUploadRoot`
- `ClusterNativeUploadMaxFiles`
- `ClusterNativeUploadMaxBytes`
- `ClusterNativeUploadMaxChunkBytes`
- `ClusterReloadOnCommit`
- `ClusterReloadWaitSeconds`
- `ClusterMaxJSONBytes`
- `ClusterTLSEnable`
- `ClusterTLSCertFile`
- `ClusterTLSKeyFile`
- `ClusterTLSCAFile`
- `ClusterTLSRequireClientCert`
- `ClusterStateRoot`
- `ClusterRetentionKeepLast`
- `BackendHealthFile`
- `OpsMaxStatsAge`
- `OpsMaxBackendHealthAge`
- `MotdFile`
- `OverviewUnionEnable`
- `OverviewUnionMode`
- `OverviewUnionStorage`
- `OverviewUnionDB`
- `OverviewUnionRoot`
- `OverviewUnionCurrent`
- `OverviewUnionPublishRoot`
- `OverviewUnionGenerationFile`
- `OverviewUnionReadOnly`
- `OverviewUnionRequireGeneration`
- `OverviewUnionAllowLocalRefresh`
- `OverviewUnionShardSize`
- `OverviewUnionKVMapSizeMB`
- `OverviewUnionMaxListGroupLines`
- `OverviewUnionMaxOverviewRange`
- `OverviewUnionStartupMode`
- `OverviewUnionUnavailableIfEmpty`
- `OverviewUnionBlockOnNewBackend`
- `OverviewUnionStartupTimeout`
- `OverviewUnionRefreshCommand`
- `OverviewUnionRefreshInterval`
- `OverviewUnionBackendTimeout`
- `OverviewUnionPartialResults`
- `OverviewUnionArticleFallback`
- `OverviewUnionFallbackByMessageID`
- `OverviewUnionInitialHeaders`
- `OverviewUnionMaxFetchPerRefresh`
- `OverviewUnionRetentionDays`
- `OverviewUnionCompleteOnly`
- `ArticleCacheEnable`
- `ArticleCacheRoot`
- `ArticleCacheMaxObjectMB`
- `ArticleCacheTTL`
- `ArticleCacheMaxBytesMB`
- `ArticleCacheLogHits`
- `ArticleCacheLogStores`
- `ArticleCacheLogMisses`
- `BackendPoolEnable`
- `BackendPoolMaxIdlePerClient`
- `BackendPoolIdleTTL`
- `BackendPoolValidateOnReuse`
- `BackendPoolStrictServerMatch`
- `BackendPoolReserveConnections`
- `BackendPoolMaxArticlesPerSocket`
- `BackendPoolMaxBytesPerSocketMB`
- `BackendFullAction`
- `BackendWaitTimeout`
- `BackendQueueSleepMs`
- `MaxPostBytes`
- `MaxIHAVEBytes`
- `MaxPostLines`
- `MaxHeaderBytes`
- `MaxPostGroups`
- `PostFilterEnable`
- `PostFilterCommand`
- `PostFilterTimeout`
- `PostFilterFailAction`
- `PostFilterAllowModify`
- `PostFilterTempDir`
- `PostFilterSpoolDir`
- `PostFilterMaxOutputBytes`
- `AccountingInterimInterval`
- `LicenseConnectTimeout`
- `LicenseServer`
- `LicenseFingerprint`
- `LicenseKey`
- `LicenseCacheFile`

### Retired or moved runtime directives

These were config baseline but are not current runtime directives in rc5:

- `SSLKeyfile`
- `SSLPassword`
- `SSLCAList`
- `AsLoginURL`
- `AsLogoutURL`
- `RemoteConfig`
- `WildmatURL`
- `JSONACL`
- `ACLURL`
- `JSONRL`
- `RLURL`

Notes:

- `SSLKeyfile`, `SSLPassword` and `SSLCAList` are represented under vhost/profile style access configuration.
- `AsLoginURL`, `AsLogoutURL`, `RemoteConfig`, `WildmatURL`, `JSONACL`, `ACLURL`, `JSONRL` and `RLURL` belong to older remote HTTP/JSON integration paths that are not part of the rc5 core model.

## Packaging and filesystem layout

The original repository included old packaging material, but rc5 adds a modern deployable layout.

### Current production paths

- `/srv/nntpswitch/bin`
- `/srv/nntpswitch/libexec`
- `/srv/nntpswitch/docs`
- `/srv/nntpswitch/share/examples`
- `/etc/nntpswitch`
- `/var/lib/nntpswitch`
- `/var/cache/nntpswitch`
- `/var/spool/nntpswitch`
- `/run/nntpswitch`

### Debian packages

rc5 generates separate packages:

- `nntpswitch_16.9.0-rc5-1_amd64.deb`
- `nntpswitch-lab-tools_16.9.0-rc5-1_all.deb`

The production core package contains daemon/runtime/admin components.  The lab-tools package contains simulator, benchmark, stress and cross-host test helpers.

### rc5 core/lab split

Moved out of production core and into `nntpswitch-lab-tools`:

- `nntpswitch-sim-lab-init`
- `nntpswitch-rebuild-generation-safe`
- `nntpswitch-cluster-lab`
- `nntpswitch-crosshost-test`
- `nntpswitch-nntp-bench`
- `nntpswitch-overview-bench`
- `nntpswitch-stress-runner`
- `examples/simulator-lab/*`

## systemd, timers, logrotate and sysctl

New systemd units/timers:

- `nntpswitch.service`
- `nntpswitch-clusterd.service`
- `nntpswitch-metrics.service`
- `nntpswitch-backend-health.service`
- `nntpswitch-backend-health.timer`
- `nntpswitch-article-cache-prune.service`
- `nntpswitch-article-cache-prune.timer`
- `nntpswitch-cluster-cert-renew.service`
- `nntpswitch-cluster-cert-renew.timer`
- `nntpswitch-cluster-secret-rotate.service`
- `nntpswitch-cluster-secret-rotate.timer`

rc5 packaging hardening:

- `prerm` stops/disables all packaged units, not only `nntpswitch.service`.
- `postrm purge` removes all packaged unit files and resets failed systemd state.
- Production package installs logrotate policy.
- Production package installs sysctl tuning file.

## Build and dependency changes

Added or modernized:

- `CMakeLists.txt`
- extended `Makefile`
- dependency manifest: `packaging/dependencies/dependencies.yaml`
- portable installer: `packaging/scripts/install.sh`
- uninstaller: `packaging/scripts/uninstall.sh`
- `nntpswitch-init.sh`
- `nntpswitch-setup.sh`
- `nntpswitch-config-render.py`
- `nntpswitch-doctor.py`

The install flow now supports production roles, cluster roles, dependency installation profiles, TLS/mTLS bootstrap, HMAC keyring initialization and non-interactive package setup.

## Security hardening since the original tree

Important security changes in the current core:

- MySQL/MariaDB authentication no longer builds a login query by concatenating user input.
- PostgreSQL authentication uses parameterized execution.
- Password-file authentication no longer relies on plaintext case-insensitive password comparison.
- Password hashing uses PBKDF2 helpers.
- Remote auth reads are bounded and NUL-terminated.
- Remote auth sockets are closed on all rc5 paths.
- Backend socket read helpers were hardened against one-byte NUL overflows.
- `BufSize` is rejected outside a controlled range.
- Legacy stack VLA usage in socket/auth paths was reduced.
- Config parser string conversions in access/server parsing were width-limited.
- MySQL escaping helper clamps source length to output capacity.
- Cluster shared-secret comparison uses constant-time comparison.
- HMAC cluster mode adds timestamp/nonce/body-hash signing.
- TLS/mTLS is supported for cluster traffic.
- Commercial release gate prevents accidental release with development public key.

## Documentation added

New documentation under `docs/`:

- `BUILD_FROM_SOURCE.md`
- `CLEAN_SLATE_AND_SIM_LAB.md`
- `CLUSTER_OPERATIONS.md`
- `CONFIG_REFERENCE.md`
- `DEPENDENCIES.md`
- `EXAMPLES.md`
- `INSTALL.md`
- `PRODUCTION_SIZING.md`
- `SECURITY.md`
- `TROUBLESHOOTING.md`
- `UPGRADE.md`
- `VERSIONING.txt`

Additional release/change notes are kept in `NNTPSwitch_v16.9.0-*_CHANGELOG_TEST_COMMANDS.txt` files.

## Testing, lab and benchmark additions

New test/lab capabilities:

- build smoke test: `tests/integration/smoke-build.sh`
- local cluster self-test: `tests/integration/cluster-local-selftest.py`
- postfilter tests: `tests/postfilter/*`
- legacy testsuite kept and extended under `testsuite/`
- cross-host cluster/frontend tests via `nntpswitch-crosshost-test`
- NNTP benchmark tool via `nntpswitch-nntp-bench`
- overview benchmark via `nntpswitch-overview-bench`
- stress runner via `nntpswitch-stress-runner`
- simulator lab init via `nntpswitch-sim-lab-init`

Validation performed for rc5 during packaging:

- `make -j2 all`: OK
- `make sanitize`: OK
- `tests/integration/smoke-build.sh`: OK
- `python3 -m py_compile tools/*.py`: OK
- shell/debian script syntax checks: OK
- postfilter Perl syntax checks: OK
- `nntpswitch-pre-rc-check`: PASS
- `nntpswitch-pre-rc-check --commercial-release`: expected FAIL while development public key is embedded

## Operational goal: 10000+ concurrent users

The current rc5 architecture is designed to make 10000+ concurrent users realistic.

Features added specifically to make large deployments viable:

- high `MaxConnections` support with modern systemd/sysctl packaging;
- backend `maxconnections` controls;
- weighted backend balancing;
- per-client backend socket pooling;
- local generated overview for heavy XOVER/LISTGROUP workloads;
- LMDB overview storage option;
- frontend generation mode with atomic publish/current switching;
- Prometheus metrics exporter;
- backend health checks;
- stress, bench and cross-host test tools;
- lab package split so production nodes do not ship test-only helpers.

Recommended production posture for large services:

- run Overview Union in `frontend` mode;
- use `OverviewUnionRequireGeneration 1`;
- use LMDB-backed generated overview for large groups;
- cap `OverviewUnionMaxOverviewRange` to a public-service-safe value;
- keep log verbosity low during load;
- keep simulators/benchmark tools out of production core installs;
- validate p95/p99 latency under realistic XOVER-heavy workloads before tagging stable.

## Companion projects

The broader NNTPSwitch commercial system also has companion PHP projects for customer self-service, billing/reseller management and license issuance.  They are not part of this rc5 core source tree unless published separately in the same GitHub organization/repository layout.

When included, their main differences from upstream are complete additions, because upstream NNTPSwitch 0.12+ did not ship a customer portal or licensing portal.

Companion webportal capabilities include:

- registration, login, reset and session management;
- TOTP 2FA;
- reseller/multi-tenant support;
- PayPal/Stripe orders/subscriptions and webhook processing;
- usage/active-session APIs;
- abuse/DMCA/admin tooling;
- quota reset and usage snapshot CLI jobs.

Companion licensing portal capabilities include:

- RSA-signed license issuance;
- license check API;
- customer/license CRUD;
- roles and TOTP;
- check logs and rate limits;
- expiration reminders;
- tamper-evident event ledger.

## File tree delta

### Original manifest entries removed, renamed or moved

- `Changes`
- `README`
- `acct_radius.c`
- `acct_syslog.c`
- `auth_ldap.c`
- `auth_pop3.c`
- `auth_radius.c`
- `etc/access.conf`
- `etc/init.d`
- `etc/overview.fmt`
- `etc/passwd`
- `etc/servers.conf`
- `nsstats.pl`
- `nsstats.sh`

Additional original tree items observed in the file listing but not present in the current rc5 source tree include:

- `auth/`
- `bin/`
- `debian/`
- `curl.c`
- `curl.h`
- `rconfig.c`

Notes:

- `nsstats.pl` and `nsstats.sh` are preserved as legacy tools under `tools/stats/legacy/`.
- Example files moved from `etc/` to `examples/` and are installed into `/etc/nntpswitch` by packaging.
- Some legacy commercial/auth/config modules were intentionally removed rather than carried forward.

## Release candidate notes

rc5 is intentionally narrow compared with rc4:

- no NNTP feature expansion;
- remote auth buffer/socket hardening;
- production package split tightened;
- service removal/purge handling fixed;

Before a final commercial stable release, at minimum:

1. Replace the development licensing public key with the real commercial public key.
2. Re-run `nntpswitch-pre-rc-check --commercial-release` and require PASS.
3. Complete cross-host master/slave validation with real key and not develop key.
4. Complete 20000+ concurrent-user test.
5. Complete XOVER-heavy benchmark with realistic large groups.
6. Complete soak testing for file descriptor stability, RSS stability and backend pool behaviour.
7. Tag the final release only after the above gates pass.