The JaiaBots and Hubs require SSH access for several purposes:
SSH is primarily allowed through FIDO hardware authenticators (the Yubikey product) via OpenSSH's "ed25519-sk" public key type. This provides convenient two-factor authentication ([1] physical key + [2] private key file).
All permissions flow from a set of "root" Yubikeys that are flashed to the image at generation time (jaiabot/config/ssh/root_authorized_keys).
The root yubikeys are controlled by trusted parties at the JaiaBot headquarters. This always allow access, but only when the Bots/Hubs are accessible via the service VPN (which can be enabled/disabled using the Upgrade GUI).
These keys are stored in /etc/jaiabot/ssh/root_authorized_keys.
This must currently be done manually, but future upgrades will provide a mechanism by which Root Yubikeys can be revoked during software upgrades (During apt install jaiabot-embedded).
Each hub ships with its own Yubikey permanently installed into the USB port of the hub. The public key is distributed to all the bots during fleet configuration and is stored in /etc/jaiabot/ssh/hub_authorized_keys.
Re-running fleet configuration with a new key will the key for that hub on all bots.
For a test fleet (for example a VirtualBox fleet), jaia admin fleet create --test-keys (also accepted by edit) gives each new hub an ordinary ed25519 key instead of asking for its Yubikey. The private key is then only a file in the fleet config, so anyone with a copy of the config can log in as that hub. Never use it for a real deployment. Such keys are marked by their comment, hub<N>_fleet<M>_test_key.
This section written by Claude
The CloudHub has no Yubikey. It generates its own key on first boot, the private half never leaves it, and the fleet config records only the public half. Every node accepts that key only from the CloudHub's address on the CloudHub VPN. See CloudHub SSH key.
For various reasons (repairs, debugging, etc.), keys may need to be temporarily authorized that belong to JaiaBot employees, repair facilities, etc. These keys are stored in /etc/jaiabot/ssh/tmp_authorized_keys and are set with a timeout corresponding to the required time to perform the necessary activities.
These must be provisioned by a system with existing SSH access, typically via the root Yubikeys.
The jaia command line tool can be used to add temporary Yubikeys to a given system (via the jaia admin ssh set of actions).
For convenience, known trusted public keys are compiled into the source code at jaiabot/src/bin/tool/actions/admin/ssh/pubkeys.cpp (in the pubkeys vector). Revoked (lost, stolen, damaged) keys are also listed within this file.
Each authorized public key in OpenSSH is formatted as such (see https://man.openbsd.org/sshd#AUTHORIZED_KEYS_FILE_FORMAT):
This format is what we also use here with the jaia tool.
To add a key to a Bot or Hub, run:
where the parameters are:
This entry is added to /etc/jaiabot/ssh/tmp_authorized_keys, replacing the same key if it already exists.
For example,
adds the key corresponding to the comment "jaia@repair_test1" in pubkeys.cpp to Bot 1 on Fleet 0 (over the service VPN) for a period of 2 weeks. This key is only allowed to run /usr/share/jaiabot/config/fleet/fleet-config.sh as defined in pubkeys.cpp.
For another example,
adds Toby's contractor/employee key 16719472 (also defined in pubkeys.cpp) for 3 days to the Hub 1 on Fleet 3 so he can perform debugging requested by the customer.
To remove a given key run:
where the parameters are the same as for adding a key.
Alternatively,
removes all known (compiled-in) revoked public keys on that host.
shows all the authorized keys on the host and which file they reside in.
removes all temporary keys from the host.
shows the comment for all the known valid keys.
shows the full authorized keys line for all known valid keys.
shows the comment for all the revoked keys.
Keys can be revoked by adding them to the revoked_pubkeys vector in pubkeys.cpp. These keys are no longer allowed to be added (with jaia admin ssh add) and can be removed from existing systems using jaia admin ssh rm <host> --revoked.
Customers are free to add SSH keys of any type to /home/jaia/.ssh/authorized_keys, which is otherwise empty. When specifying "forever" as the "valid_for" parameter, the keys are defaulted to this file, rather than /etc/jaiabot/ssh/tmp_authorized_keys.
The jaia tool can add these keys using, for example:
or by manually editing the /home/jaia/.ssh/authorized_keys file.
vpn.jaia.tech and packages.jaia.tech can be managed in the same way, e.g.:
The keys above decide who a node will accept. Reaching a customer's CloudHub turns on something else first: whether anyone can knock at all.
Port 22 is shut at the CloudHub's own AWS security group, in the customer's own account. A grant opens it, for the window the customer approved and only to the address the request was signed from; the expiry timer shuts it again. The keys that then work are the root Yubikeys the image already carries, so nothing is pushed anywhere and nothing has to be taken back.
That is worth stating plainly, because it is the whole design: reachability is the control, not key trust. The customer cannot stop a Yubikey from being a valid key, and does not need to — they decide whether there is a route to use it on, in a place Jaia cannot reach.
It also means recovery never depends on our software. If the CloudHub's own directory will not start, its support page is unreachable too, since that page sits behind Authelia. The customer opens port 22 from the AWS console and a Yubikey gets in.
From that shell, bots and hubs are reached as they always are — jaia admin ssh add run on the CloudHub, with the CloudHub's own key already authorized on every node. There is no second grant.
The customer's administrator should never have to judge whether a phone call claiming to be Jaia really is. So the right to ask is tied to the root Yubikeys rather than to convention. jaia admin fleet support_request signs the fleet, the window, the reason and the address to admit with one of them:
It prints a block to send to the customer, who pastes it at https://support.<their base uri> — a page beside the directory, behind the same login, open to lldap_admin and super_admin at two factors. The CloudHub verifies the signature against the root keys already on its own image (/etc/jaiabot/ssh/root_authorized_keys) and draws nothing at all for a request it cannot verify, so a request cannot be forged by anyone who reaches that page. Approving it is one click, and opens the port. The page shows the customer the address it will admit before they do.
Granting runs from the approval, not from when the request was made, and no grant lasts more than two weeks.
The grant is one record, and the firewall is derived from it by a timer that reconciles every few minutes — so an expiry takes effect with nobody acting, and a rule with no grant behind it is closed rather than left. The rule is written both to the security group and to ufw: the first is the one that matters, because AWS enforces it off the instance, and the second so the gate also exists on a CloudHub that is not in EC2. The support page shows what is open, until when, and the log of every grant and every ending.
A hub can disable and re-enable its own tunnel to the CloudHub from its Upgrade GUI. That decides whether the fleet reaches the cloud, and has nothing to do with support — but the two meet in one place: a hub with its tunnel off is out of reach even during a live grant.
That control is deliberately local. It acts on the hub it is run from and no other node, so it can never stop the CloudHub's own end and take the whole fleet off, and because the GUI serving it runs on that hub over the fleet WLAN, the link can always be restored on-site. Note that it also carries HUB2HUB, so cutting it costs inter-hub comms as well.