Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

Overview

Statistics for entire archiving period (2006-01-01 - present)

background:#B8D1F3,background:#DAE5F4,background:#B8D1F3
Section
bordertrue


Column


Panel


Archive size [PiB]


SQL
border0
SQL
rowStyles
noDataErrortrue
enableSortingfalse
columnStylesfont-size: 1.0em
rowOrientationhorizontal
expandArrayfalse
stylefumble:0
convertNullfalse
idmydata3
class@default testing
dataSourcemars_apistats
enableHighlightingfalse
tablefalse
retainRowStyleOrderfalse
SELECT  to_char(
sum(fields),
Grand_total/1024^5, 'FM999,999,999,990.999
,999
') FROM mars_dataset_details
WHERE name 
as "Total fields",
= 'tigge' ORDER BY Date DESC LIMIT 1


Number of  fields


SQL
border0
noDataErrortrue
enableSortingfalse
columnStylesfont-size: 1.0em
rowOrientationhorizontal
expandArrayfalse
stylefumble:0
convertNullfalse
idmydata3
dataSourcemars_apistats
enableHighlightingfalse
tablefalse
retainRowStyleOrderfalse
SELECT  to_char(
sum(requests)
Number_of_fields, 'FM999,999,999,990.999
,999
') FROM mars_dataset_details
WHERE name = 'tigge' ORDER BY Date DESC LIMIT 1





Column


Panel


Number of active users


SQL
dataSourcemars_apistats
tablefalse
SELECT count(DISTINCT auth_user.username) as "Total 
requests", rtrim(
number of users" FROM  auth_user
   RIGHT JOIN users_stats_monthly ON users_stats_monthly.username = auth_user.username
     WHERE users_stats_monthly.dataset='tigge-global' and users_stats_monthly.month>=date'2006-01-01'
AND  auth_user.username IN (
      SELECT username                        
      FROM  users_stats_monthly
      WHERE dataset='tigge-global'
      GROUP BY username
      HAVING sum(requests) >= 1)


Delivered data volume [TiB]


SQL
dataSourcemars_apistats
tablefalse
SELECT to_char(sum(delivered/1024^4), '
FM999999999999990.999'), '.') as "Delivered volume (TB)" FROM
FM999,999,999,990')FROM datasets_stats_monthly
 WHERE datasets_stats_monthly.dataset='tigge-global' and datasets_stats_monthly.month>=date'
20060101
2006-01-01';

Monthly sums

Data volume [TB]

  • Delivered: data volume delivered to user after post-processing(e.g. interpolation) applied if asked for
  • Retrieved: data volume retrieved from MARS before post-processing
    • retrieved data volume is normally equal or bigger then the delivered one with exception when user asked for higher then archived grid resolution resulting in artificially bigger interpolated files
true


Number of retrieved fields


SQL
border0
noDataErrortrue
enableSortingfalse
columnStylesfont-size: 1.0em
rowOrientationhorizontal
expandArrayfalse
stylefumble:0
convertNullfalse
idmydata3
dataSourcemars_apistats
enableHighlightingfalse
tablefalse
retainRowStyleOrderfalse
SELECT to_char(sum(fields),'FM999,999,999,999,999')
FROM  datasets_stats_monthly
WHERE datasets_stats_monthly.
Chart
showShapesfalse
timePeriodQuarter
width1200
categoryLabelPositionup45
dataOrientationvertical
typeline
yLabelTB
height400
SQL
idmydata
dataSourcemars_apistats
select to_char(month, 'MM/YYYY') as monthyear, CAST(delivered AS FLOAT)/1024^4 as "Delivered" from datasets_stats_monthly where
dataset='tigge-global' and datasets_stats_monthly.month>=date'
20061001' order by month ; select to_char(month, 'MM/YYYY') as monthyear, CAST(retrieved AS FLOAT)/1024^4 as "Retrieved" from datasets_stats_monthly where
2006-01-01';


Number of requests


SQL
border0
noDataErrortrue
enableSortingfalse
columnStylesfont-size: 1.0em
rowOrientationhorizontal
expandArrayfalse
stylefumble:0
convertNullfalse
idmydata3
dataSourcemars_apistats
enableHighlightingfalse
tablefalse
retainRowStyleOrderfalse
SELECT to_char(sum(requests), 'FM999,999,999,999,999')
FROM  datasets_stats_monthly
WHERE datasets_stats_monthly.dataset='tigge-global' and datasets_stats_monthly.month>=date'
20061001' order by month
Chart
3D
2006-01-01';





Monthly sums

Data volumes

Volume of data retrieved from MARS before post-processing vs volume of data delivered to users after post-processing.

Chart
showShapesfalse
timePeriodQuarter
width1200
categoryLabelPositionup45
dataOrientationvertical
typebar
yLabelTBTiB
height400


SQL
idmydata
dataSourcemars_apistats
select  to_char(month, 'MM/YYYY') as monthyear, CAST(delivered AS FLOAT)/1024^4 as "Delivereddelivered" from datasets_stats_monthly where dataset='tigge-global' and month>=date'200610012006-01-01' order by month ;
select  to_char(month, 'MM/YYYY') as monthyear, CAST(retrieved AS FLOAT)/1024^4 as "Retrievedretrieved" from datasets_stats_monthly where dataset='tigge-global' and month>=date'200610012006-01-01' order by month

User requests

(1 original user request could be split to more resulting  MARS database retrieval requests  shown in the graph below. It depends on MARS data design for given datasets. On average number of the original users requests is about 1/4 of the resulting "user requests" hitting MARS shown here)


Number of requests

Chart
Chart
3Dtrue
showShapesfalse
width1200
categoryLabelPositionup45
dataOrientationvertical
typebar
yLabelNr of requests
colors#FFAB00
height400300


SQL
idmydata
dataSourcemars_apistats
select  to_char(month, 'MM/YYYY') as monthyear, requests from datasets_stats_monthly where dataset='tigge-global' and month>=date'20061001' order by month;

Fields retrieved

(1 field means the retrieved field for given step, level, etc)

Chart
3Dtrue
showShapesfalse
width1200
categoryLabelPositionup45
dataOrientationvertical
typebar
yLabelNr of fields
height400
SQL
idmydata
dataSourcemars_apistats
select  to_char(month, 'MM/YYYY') as monthyear, fields"user requests" from datasets_stats_monthly where dataset='tigge-global' and month>=date'200610012006-01-01' order by month;

Users

(Users at least with 3 requests ever since)


Number of active users

Chart
Chart
3Dtrue
showShapesfalse
width1200
categoryLabelPositionup45
dataOrientationvertical
typebar
yLabelNr of users
colors#36B37E
height400300


SQL
idmydata
dataSourcemars_apistats
SELECT to_char(month, 'MM/YYYY') as monthyear, count(DISTINCT username) as "active users"
FROM  users_stats_monthly 
WHERE dataset='tigge-global' AND month>=date'200610012006-01-01' 
AND username IN (
      SELECT username                        
      FROM  users_stats_monthly
      WHERE dataset='tigge-global'
      GROUP BY username
      HAVING sum(requests) >= 31
      )
GROUP BY month
ORDER BY month;


Users per

...

country

  • as per  2018-06-01
  • only countries with at least 5 active users are shown
Chart

(Users at least with 3 requests ever since)

Chart
3Dtrue
stackedtrue
width1200900
dataOrientationvertical
opacity75
height1200700


SQL
idmydata
dataSourcemars_apistats
SELECT profile_country.name, count(DISTINCT auth_user.username) as total_users
    FROM  profile_country
         RIGHT JOIN profile_profile ON profile_profile.country_id = profile_country.id
         RIGHT JOIN auth_user ON auth_user.id = profile_profile.user_id
         RIGHT JOIN users_stats_monthly ON users_stats_monthly.username = auth_user.username
    WHERE users_stats_monthly.dataset='tigge-global' and users_stats_monthly.month>=date'200610012006-01-01'
GROUP BY profile_country.name
HAVING count(DISTINCT auth_user.username)>=5
ORDER BY total_users DESC;

...