To ensure that computing resources are distributed equitably and to discourage irresponsible use users' jobs on the Atos HPCF are charged for the resources they have used against a project account. Each project account is allocated a number of System Billing Units(SBU) at the beginning of each accounting year (1 January to 31 December).
ECS usage
Resources used on the ECGATE Class Service (ECS) cluster will not be charged
How much is an SBU?
The accounting on the HPCF is currently based on the total elapsed time for which cores are used. With that, the HPCF unit of account, the System Billing Unit, or SBU, for both serial and parallel jobs is defined as follows.
SBU = P * N * T
where:
| P = 117602602 / (2000 * 128 * 86400) | is a fixed proportionality factor, adjusted so that one cluster will deliver 117,602,602 units per day on a "typical" |
| N | Is the number of (physical) cores allocated to a job. |
| T | is the elapsed time (real time in seconds) recorded for a job. |
Examples:
- A serial job using 1 hour (3600 seconds) of elapsed time will be charged about 19.14 SBU.
- A parallel job requesting 2 nodes, running for 3 hours elapsed time will be charged 14700.32 SBU.
End of job information
At the end of your jobs a short report is appended to the corresponding job output file, which amongst other useful information, includes how many SBUs where used and the account they were charged on:
[ECMWF-INFO -ecepilog] ---------------------------------------------------------------------------------------------------- [ECMWF-INFO -ecepilog] This is the ECMWF job Epilogue [ECMWF-INFO -ecepilog] +++ Please report issues using the Support portal +++ [ECMWF-INFO -ecepilog] +++ https://support.ecmwf.int +++ [ECMWF-INFO -ecepilog] ---------------------------------------------------------------------------------------------------- [ECMWF-INFO -ecepilog] Run at 2022-08-24T09:09:01 on ac [ECMWF-INFO -ecepilog] JobName : myjob [ECMWF-INFO -ecepilog] JobID : 37015044 [ECMWF-INFO -ecepilog] Submit : 2022-08-24T09:08:55 [ECMWF-INFO -ecepilog] Start : 2022-08-24T09:08:55 [ECMWF-INFO -ecepilog] End : 2022-08-24T09:09:01 [ECMWF-INFO -ecepilog] QueuedTime : 0.0 [ECMWF-INFO -ecepilog] ElapsedRaw : 6 [ECMWF-INFO -ecepilog] ExitCode : 0:0 [ECMWF-INFO -ecepilog] DerivedExitCode : 0:0 [ECMWF-INFO -ecepilog] State : COMPLETED [ECMWF-INFO -ecepilog] Account : myaccount [ECMWF-INFO -ecepilog] QOS : np [ECMWF-INFO -ecepilog] User : user [ECMWF-INFO -ecepilog] StdOut : /home/user/slurm-37015044.out [ECMWF-INFO -ecepilog] StdErr : /home/user/slurm-37015044.out [ECMWF-INFO -ecepilog] NNodes : 1 [ECMWF-INFO -ecepilog] NCPUS : 256 [ECMWF-INFO -ecepilog] SBU : 4.083 [ECMWF-INFO -ecepilog] ----------------------------------------------------------------------------------------------------
What are my project accounts?
All users on the HPCF have at least one project account where their usage will be charged. You can check what accounts you have access to with the command:
account [-l uid]
If more than one account is listed, then by default, job usage will be debited to the first account, which is then considered the primary account.
If you wish to debit the usage of a job to an alternative account (out of your list of assigned accounts) you may do so by using the following Slurm directive in your job header:
#SBATCH --account=your_preferred_account
How do I monitor my usage?
Interim Atos HPCF usage monitoring
SBU usage on the Atos, which is currently not charged against SBU allocations, can be monitored with the Atos Preproduction HPC SBU Accounting portal
Until 1 November 2022, the main HPC SBU accounting portal will only shows Cray HPCF usage.
You may use the Atos Preproduction HPC SBU Accounting portal to keep track of your SBU utilisation on the Atos HPCF. You can login with your usual ECMWF credentials.
You will find the accumulated SBU usage for your different accounts from the beginning of the accounting year as well as the allocations granted for each of them.
Please note the data shown is updated daily, so resources used by your latest jobs will not be shown immediately.
This web application is still under development and further enhancements and bug fixes will be implemented. Please send any questions or feedback via our Support Portal.
Accounting and account management
Computing resource allocation rules
Member States are allocated computing resources for their own purposes according to the Council guidelines on distribution of computer resources. Member State Computing Representatives may distribute those resources among projects registered at ECMWF as they feel appropriate. Should a redistribution of resources between the projects of a Member State be required during the course of a year, this may be carried out by the Member State Computing Representative or another designated "privileged user" (see section Administration of project accounts in the Member States below).
Special Projects
Special Projects are experiments or investigations of a scientific or technical nature undertaken by one or more Member States, likely to be of interest to the general scientific or technical community. In the spring of each year, Directors of national meteorological services are invited to submit requests for Special Project resources to be available for use in the following year. Requests are considered by the Council at its winter session, in conjunction with recommendations from the Scientific Advisory Committee and Technical Advisory Committee. Since Special Project accounts are separate from Member State resources, their account administration is performed by ECMWF.
For more details see http://www.ecmwf.int/en/research/special-projects.
Daily Accounting Controls
Information is extracted daily from HPCF report files to produce a cumulative total of resources used per user and per account. Use of each account is checked daily and once usage reaches 90% of its resources available, a warning message is sent out, to enable users to arrange for the transfer of additional resources, if necessary. Once usage reaches 100% no further use of that account should be made.
Registering a new account
New accounts are registered by ECMWF only on the authority either of an ECMWF Department Head (for ECMWF staff accounts) or of a Member State Computing Representative (for Member State accounts) or the Principal Investigator (for Special Projects).
For further information get in touch via our Support Portal.
Administration of project accounts in the Member States
Member States may nominate "privileged users" to perform limited administration of Member State accounts using HPC SBU accounting portal. By default, the Centre will grant this status to all Member State Computing Representatives with a registered user identifier, but Member State Computing Representatives may propose an alternative person and/or nominate a backup if they wish.
A "privileged user" may be authorised to modify all or a reduced set of Member States' accounts. For instance, Member States are free to allocate control to several "privileged" users, each having control of a particular sub-node. It should be noted, however, that the nomination of privileged users should be kept to a minimum.
For security reasons, the establishment of new project accounts and the addition of new users remains under the control of ECMWF.