Page History

...

1. Create the Data Tailor standalone instance by clicking on + ADD button on your Morpheus Dashboard.
2. Search for Data Tailor Standalone, select it and click next.
   
3. Choose a a NAME for the instance type and move to next step.
   Image Modified
4.  Insert at least the following inputs and click the next button:
   • layout: make sure it is rocky-datatailor
   • plan: slect select one with at least 16GB of RAM (e.g. eo2.xlarge)
   • network: private
   • security group: ssh
     
5. Once done, click Complete and the provisioning will start.
6. Look for the instance list (Provisioning → Instances) to check the status of it.
7. Once ready you will see the green Running sign below the instance as below:
   

...

1. SSH into your instance following the Remote Access to the VM documentation. Alternatively, you can also use the console on Morpheus from your instance (Provisioning → Instances → select your instance and go to the Console section)
2. Running the following commands you can verify if the installation was successful.

   Code Block
   language bash title Check conda installation and dt_3_3 environment
   $ conda env list # conda environments: # base                  *  /usr/local/miniconda dt_3_3                   /usr/local/miniconda/envs/dt_3_3

3. After confirming the installation, add conda environment to your shell and then restart it.
   

   Code Block
   language bash title Integrate conda in your shell and restart your shell
   $ conda init no change     /usr/local/miniconda/condabin/conda no change     /usr/local/miniconda/bin/conda no change     /usr/local/miniconda/bin/conda-env no change     /usr/local/miniconda/bin/activate no change     /usr/local/miniconda/bin/deactivate no change     /usr/local/miniconda/etc/profile.d/conda.sh no change     /usr/local/miniconda/etc/fish/conf.d/conda.fish no change     /usr/local/miniconda/shell/condabin/Conda.psm1 no change     /usr/local/miniconda/shell/condabin/conda-hook.ps1 no change     /usr/local/miniconda/lib/python3.11/site-packages/xontrib/conda.xsh no change     /usr/local/miniconda/etc/profile.d/conda.csh modified      /home/USERNAME/.bashrc ==> For changes to take effect, close and re-open your current shell. <==

4. After restarting your shell, you should be able to see that you're in base environment of conda.
   

   Code Block
   language bash title Activate the conda environment and test the application
   (base) $ conda activate dt_3_3 (dt_3_3) $ epct info epct_version: 3.3.0 etc_dir: /usr/local/miniconda/envs/dt_3_3/etc/epct workspace_dir: /usr/local/miniconda/envs/dt_3_3/var/cache/epct customisations_log_dir: /usr/local/miniconda/envs/dt_3_3/var/log/epct output_dir: / installed_plugins:   - epct-plugin-ncarrays, 1.1.0   - epct-plugin-gis, 3.2.0 registered_backends:   - epct_1d_array   - epct_2d_array   - epct_ascatl1szf   - epct_eps_gome2l1b   - epct_eps_grasl1   - epct_eps_iasil1c   - epct_eps_iasisnd02   - epct_eps_native   - epct_eps_native_gome2l1_iasisnd02   - epct_gis   - epct_gis_eps   - epct_gis_eps_grib2   - epct_gis_glbsst   - epct_gis_glbsst_grib2   - epct_gis_grib2   - epct_gis_hrit   - epct_gis_hrit_grib2   - epct_gis_msg   - epct_gis_msg_meteo_bufr   - epct_or1sww025

The Data Tailor Standalone is ready to use. Whenever you want to use the application, just activate your conda environment with $ conda activate dt_3_3 and run your customisations.

...

The Data Tailor standalone has multiple ways of use including:

•  webGUI Web-app GUI App (graphical user-interface)
• CLI (command line interface), a
• Python Library
• and within EUMDAC.

For more information, please refer to:  Data Tailor Standalone Guide.

...

1. Activate the conda environment.
   

   Code Block
   language bash title Activate the conda environment
   (base) $ conda activate dt_3_3

...

1. Create the chain.yaml
   

   Code Block
   language yml title chain.yaml
   product: HRSEVIRI format: netcdf4 projection: geographic roi: western_europe filter: bands: - channel_9 - channel_10 - channel_11

...

...

1. Run your customization using the command below:
   

   Code Block
   language bash title Run chain
   (dt_3_3) $ epct run-chain -f chain.yaml MSG3-SEVI-MSG15-0100-NA-20240307112742.143000000Z-NA.zip 2024-03-07 11:39:46 - PROCESSING.chain_runner[286] - INFO - Start process "39923ab4" 2024-03-07 11:39:46 - PROCESSING.chain_runner[236] - INFO - WORKER: localhost 2024-03-07 11:39:46 - PROCESSING.chain_runner[237] - INFO - PID: 187464 2024-03-07 11:39:46 - PROCESSING.chain_runner[238] - INFO - backend: epct_gis_msg 2024-03-07 11:39:46 - PROCESSING.chain_runner[239] - INFO - user: None 2024-03-07 11:39:49 - PROCESSING.preprocessing[612] - INFO - Processing details - product: HRSEVIRI chain-name: None chain-details: -product: HRSEVIRI -format: NetCDF4 (simplified) -projection: Geographic / Plate-Carree -roi: Western Europe -filter: Custom 2024-03-07 11:39:49 - PROCESSING.preprocessing[618] - INFO - ROI name: Western Europe - NSWE: [67, 35, -23, 16] - process: 39923ab4 2024-03-07 11:39:49 - PROCESSING.preprocessing[619] - INFO - Input products: MSG3-SEVI-MSG15-0100-NA-20240307112742.143000000Z-NA.nat 2024-03-07 11:39:49 - PROCESSING.epct_gis[1452] - INFO - Expected steps: 6 2024-03-07 11:39:49 - PROCESSING.epct_gis[1454] - INFO - Starting step "IMPORT" 1/6 ... 2024-03-07 11:39:49 - PROCESSING.vrt[89] - INFO - Command line and its output ... gdal_translate -of VRT -a_srs "+proj=geos +h=35785831 +lon_0=0.0 +ellps=GRS80 +no_defs" -a_ullr -5565747.7 5565747.7 5565747.7 -5565747.7 -b 9 -b 10 -b 11 RAD:/usr/local/miniconda/envs/dt_3_3/var/cache/epct/EPCT_HRSEVIRI_FPC_39923ab4/decompressed_data/MSG3-SEVI-MSG15-0100-NA-20240307112742.143000000Z-NA.nat /usr/local/miniconda/envs/dt_3_3/var/cache/epct/EPCT_HRSEVIRI_FPC_39923ab4/MSG3-SEVI-MSG15-0100-NA-20240307112742.143000000Z-NA.vrt 2024-03-07 11:39:49 - PROCESSING.epct_gis[1091] - INFO - a preliminary ROI is applied to the intermediate VRT 2024-03-07 11:39:49 - PROCESSING.vrt[89] - INFO - Command line and its output ... gdalwarp -cutline /usr/local/miniconda/envs/dt_3_3/var/cache/epct/EPCT_HRSEVIRI_FPC_39923ab4/roi.shp -crop_to_cutline -cblend 10 -r bilinear --config OGR_ENABLE_PARTIAL_REPROJECTION YES /usr/local/miniconda/envs/dt_3_3/var/cache/epct/EPCT_HRSEVIRI_FPC_39923ab4/MSG3-SEVI-MSG15-0100-NA-20240307112742.143000000Z-NA.vrt /usr/local/miniconda/envs/dt_3_3/var/cache/epct/EPCT_HRSEVIRI_FPC_39923ab4/MSG3-SEVI-MSG15-0100-NA-20240307112742.143000000Z-NA_roi.vrt 2024-03-07 11:39:50 - PROCESSING.epct_gis[1457] - INFO - ... step "IMPORT" finished! 2024-03-07 11:39:50 - PROCESSING.epct_gis[1464] - INFO - Starting step "FILTER" 2/6 ... 2024-03-07 11:39:50 - PROCESSING.epct_gis[1468] - INFO - ... step "FILTER" finished! 2024-03-07 11:39:50 - PROCESSING.epct_gis[1464] - INFO - Starting step "INTERMEDIATE_REPROJECTION" 3/6 ... 2024-03-07 11:39:50 - PROCESSING.vrt[89] - INFO - Command line and its output ... gdalwarp -overwrite --config CPL_MAX_ERROR_REPORTS 1 -of GTiff -co BIGTIFF=IF_SAFER -t_srs EPSG:4326 --config THRESHOLD 0.005 -r bilinear -co COMPRESS=DEFLATE -srcnodata -1000.0 "/usr/local/miniconda/envs/dt_3_3/var/cache/epct/EPCT_HRSEVIRI_FPC_39923ab4/MSG3-SEVI-MSG15-0100-NA-20240307112742.143000000Z-NA_roi.vrt" "/usr/local/miniconda/envs/dt_3_3/var/cache/epct/EPCT_HRSEVIRI_FPC_39923ab4/intermediate_warped.tif" 2024-03-07 11:39:55 - PROCESSING.epct_gis[1134] - INFO - ... step "INTERMEDIATE_REPROJECTION" finished! 2024-03-07 11:39:55 - PROCESSING.epct_gis[1464] - INFO - Starting step "ROI" 4/6 ... 2024-03-07 11:39:55 - PROCESSING.vrt[89] - INFO - Command line and its output ... gdalwarp -overwrite --config CPL_MAX_ERROR_REPORTS 1 -te -23.0 35.0 16.0 67.0 --config THRESHOLD 0.005 -r bilinear "/usr/local/miniconda/envs/dt_3_3/var/cache/epct/EPCT_HRSEVIRI_FPC_39923ab4/intermediate_warped.tif" "/usr/local/miniconda/envs/dt_3_3/var/cache/epct/EPCT_HRSEVIRI_FPC_39923ab4/intermediate_warped_cut.tif" 2024-03-07 11:39:55 - PROCESSING.epct_gis[1134] - INFO - ... step "ROI" finished! 2024-03-07 11:39:55 - PROCESSING.epct_gis[1464] - INFO - Starting step "FORMAT" 5/6 ... 2024-03-07 11:39:55 - PROCESSING.vrt[89] - INFO - Command line and its output ... gdal_translate -ot Float32 -of netCDF -co FORMAT=NC4 "/usr/local/miniconda/envs/dt_3_3/var/cache/epct/EPCT_HRSEVIRI_FPC_39923ab4/intermediate_warped_cut.tif" "/usr/local/miniconda/envs/dt_3_3/var/cache/epct/EPCT_HRSEVIRI_FPC_39923ab4/OUTPUTS/EPCT_HRSEVIRI_FPC_39923ab4.nc" 2024-03-07 11:39:55 - PROCESSING.epct_gis[1134] - INFO - ... step "FORMAT" finished! 2024-03-07 11:39:55 - PROCESSING.postprocessing[468] - INFO - Starting step "POST-PROCESSING" 6/6 ... 2024-03-07 11:39:55 - PROCESSING.postprocessing[493] - INFO - ... step "POST-PROCESSING" finished! 2024-03-07 11:39:55 - PROCESSING.postprocessing[495] - INFO - output-product: ./HRSEVIRI_20240307T111509Z_20240307T112742Z_epct_39923ab4_FPC.nc 2024-03-07 11:39:55 - PROCESSING.postprocessing[503] - INFO - customisation time: 9 - process: 39923ab4 2024-03-07 11:39:55 - PROCESSING.postprocessing[504] - INFO - *** STOP PROCESSING - Status DONE *** processing-time: 9.4 [sec]

...