Upgrading veilid-server
Veilid nodes are stateful, and veilid-server nodes are no different. They have identities (i.e. keypairs), they have DHT value storage (for their own data as well as data requested to be cached by others), they have data stored by local Python-style clients (which includes keypairs used for DHT entry ownership and writing). As a result their data directories should not be destroyed on upgrade. (This author would say 'MUST NOT be destroyed on upgrade', but there are always situations which cannot be foreseen. Unless you have a truly good reason, though, just don't do it.)
Generally, on a version upgrade only the veilid-server binary within the container or environment should be updated. The data directory should never be deleted in a "nuke-and-pave" type upgrade process. This loses all the state, including cached DHT entries.
Data directory contents
The data directories contain files and databases which are related to the operation of the node and the Veilid network itself. The data directory contains the following directories:
protected_storetable_storesslblock_store
protected_store is where device-specific secrets (such as keypairs) are kept. If the configuration variable core.protected_store.always_use_insecure_storage is true (which is the default), these aren't protected using platform-provided APIs. This allows these secrets to be used without additional processes elsewhere, including via restoring these files to a different device. However, it is dangerous to user privacy if these backups are not protected against disclosure. If you want to protect your data directory against restoration attacks, you will need to change the core.protected_store.always_use_insecure_storage parameter to false. To be able to migrate, you will need to use the --new-password option to change the key used to encrypt the device key.
table_store is where table-oriented data is stored. (This includes local and remote DHT data.) For more information, see the forthcoming Internals Book.
ssl contains two directories, certs/ and keys/, which are used to store certified public keys (server.crt) and their corresponding private keys (server.key) for the https and wss protocols.
block_store is where data related to block storage (aka file storage) is planned to go. As of 0.2.6, this directory is empty, as the block storage system is not implemented.
Potential update process changes
The development team takes great care to ensure that version upgrades can read and use the files and directories created by prior versions of the veilid-core library. Since veilid-server implicitly runs veilid-core, there should generally be no need to worry about it.
Exceptions may exist, however. For example, Veilid crptography suite upgrades (such as from VLD0 to VLD1) may need additional processing. The development team endeavors to minimize these extra requirements, because they increase the load on the people who run veilid-server nodes, and any increased load can drive people away from contributing to the project and network. This is generally seen as a bad thing, but is sometimes unavoidable.
If there are any exceptions, they will be explained as part of the documentation, as part of the README, and in as many other places as we can feasibly fit them.
DHT Entries and the effect of nuke-and-pave upgrades
As of version 0.2.6, veilid-core nodes do not verify the ongoing availability of DHT entries. They do not search for them in the background, and they do not re-write them to the network if their availabilities are low.
This is a high-priority item, and the develpment team is working on it (it's called "DHT rehydration"). The practical upshot right now, though, is that if a node that stores a copy of a DHT entry goes down, that stored copy is lost to the network forever.