...

Introduction

Many problems in Earth Observation and modelling communities require a common processing algorithm independently applied to thousands (or millions!) of bits of input data.  A system to do this with many processing nodes is "High Throughput Computing" (vs "High Performance Computing", which concentrates on running large jobs that will not fit on a single machine on a pool of processing nodes, typically using MPI).

EWC provides a common solution for HTC batch processing, using HTCondor.  The major advantage of this approach is that it provides a centrally-managed system where users can take advantage of a much larger pool of resources than they have themselves.  The resources come from tenants contributing their spare resources for the common good, and additional spare resources from EWC that are also made available for anyone to use. 

The common EWC solution for HTC batch processing service is HTCondor.  HTCondor is a specialized batch system for managing compute-intensive jobs. HTCondor provides a queuing mechanism, scheduling policy, priority scheme, and resource classifications.

Users can submit their compute jobs to HTCondor, HTCondor puts the jobs in a queue, runs them, and then informs the user as to the result.

Of course, any tenant can install their own batch processing systems for their own purposes with their own resources, but will not be able to take advantage of other shared resources in a centrally organised way.

General

EWC HTcondor is a managed service. The central manager node is deployed in a tenancy on the EWC. Users can join the existing pool adding compute and submit nodes.

...

...