On 2022-08-10 we have planned a system session to set up the same ssh key across the 4 complexes using the current ac-login node as a master key. This change addresses the issue of ssh errors when connecting regarding host key changes for a given host after an update, such as:
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: POSSIBLE DNS SPOOFING DETECTED! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ The ECDSA host key for hpc-login has changed, and the key for the corresponding IP address 10.100.192.100 is unknown. This could either mean that DNS SPOOFING is happening or the IP address for the host and its host key have changed at the same time. @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that a host key has just been changed. The fingerprint for the ECDSA key sent by the remote host is SHA256:QdNPyN2jAR5m7ngLbtIUjc2JgzknvFP2flMOGbd1i5k. Please contact your system administrator. Add correct host key in /home/user/.ssh/known_hosts to get rid of this message. Offending ECDSA key in /home/user/.ssh/known_hosts:4 ECDSA host key for hpc-login has changed and you have requested strict checking. Host key verification failed.
You may need to delete old entries on your ~/.ssh/known_hosts after the session to make sure the new host keys are added, but after that you should not need to do this again.
We will also take the opportunity to make available the cron service on the hpc-cron node. Any tasks that need to be added into a crontab should be placed on this node. In case of node failure or system session, those crontabs would automatically be moved to another node.