Develop version

  • FORCE HLPS

    • Analogous to Python UDFs, R UDFs can now be used through FORCE. This applies both to the TSA and UDF submodules. Three new parameters were added: FILE_RSTATS, RSTATS_TYPE, OUTPUT_RSP. At least two functions must be present in the UDF: force_rstats_init, as well as force_rstats_pixel or force_rstats_block (depending whether RSTATS_TYPE is set to PIXEL or BLOCK). To run PIXEL-type functions, you need to install the snow and snowfall packages. To run the FORCE components that are now compiled against the R API (mostly force-higher-level, but also force-qai-inflate), you need to provide two environment variables, i.e. R_HOME and LD_LIBRARY_PATH. You can do it like this:

      • export R_HOME=$(R RHOME)

      • export LD_LIBRARY_PATH=$R_HOME/lib

      • force-higher-level parameters.prm

      In the pre-built Docker container, everything is pre-configured already.

    • Added a new parameter: STREAMING = FALSE can disable the streaming strategy of FORCE HLPS. When TRUE (previous behaviour, and still the default), FORCE will perform reading, computing and writing at the same time. If FALSE, these operations are done after one another in sequential mode. Disabling streaming might be necessary for some UDFs that otherwise produce threading conflicts with the internally used OpenMP functionality. When using R UDFs, FORCE will issue a warning and gracefully disable streaming.

    • Added a new parameter: PRETTY_PROGRESS. HLPS will display progress information on screen. When TRUE (previous behaviour, and still the default), the progress information overwrites itself to produce a pretty displayal. However, this can cause error messages (or printing in UDFs) to be overwritten. If disabled (FALSE), the progress information will be simply be appended on screen (stdout).

    • Removed warning message that no output is produced when using the sampling submodule. It is now checked whether the files are actually written and will only warn if no input was detected or if no files were written. The behaviour for all other submodules stays the same.

    • It is now possible to specify the product types that should be used. This is to give the user more freedom with respect to data sources to be used. There are two new parameters: PRODUCT_TYPE_MAIN and PRODUCT_TYPE_QUALITY. The main product is usually a reflectance product like BOA. When using composites, you may use BAP. This can be anything, but make sure that the string can uniquely identify your product. As an example, do not use LEVEL2 as this will not filter products apropriately. Note that the product should contain the bands that are to be expected with the sensor used, e.g. 10 bands when sensor is SEN2A. The quality product type should be a bit flag product like QAI. When using composites, you may use INF. This can be anything, but make sure that the product contains quality bit flags as outputted by FORCE L2PS. As an exception, it is also possible to give NULL if you don’t have any quality masks. In this case, FORCE will only be able to filter nodata values, but no other quality flags as defined with SCREEN_QAI. Feature requested by Marcel Schwieder and Felix Lobert.

    • In the TSA submodule, it is now possible to disable fitting a monotonic trend within the harmonic model. The new parameter HARMONIC_TREND = TRUE/FALSE was added. Default behaviour (as before) is TRUE.

    • The Level 3 submodule now allows to use the internal scores for determining the best available observation as as a weight, i.e., not performing a selection, but a weighted average. This idea is taken from the MAJA WASP approach of Olivier Hagolle. A new parameter SELECT = TRUE/FALSE was introduced. with FALSE enabling the weighted average. Note that the INF and SCR products will not be produced when using this option.

    • The Level 3 submodule received the new parameters USE_CLOUDY and USE_HAZY, which can be used to allow observations with a cloud or haze score of less than 1% in the composite. On one hand, this reduces nodata gaps in the composite. On the other hand, it will include clouds or haze if there is no better observation.

    • The Level 3 submodule received a new parameter COMBINE_SCORES, which allows you to choose between additive or multiplicative combination of the individual scores into the total score. Additive combination is the previously used method and remains the default. Multiplicative combination can be used, which is much stricter in case one observation is characterized by a very poor suitability in a single score.

    • a new parameter OUTPUT_SUBFOLDERS was implemented that allows to output the data into subfolders under the tile folder instead of just dumping everything into the tile folder itself. It is planned that this feature will be expanded in the future. OUTPUT_SUBFOLDERS = FALSE remains the default.

  • FORCE L2PS

    • force-level2 was refactored. Little has changed for users. The logfiles are more verbose, and log messages generated by force-level2 and by the core process force-l2ps are more cleary separated. The queue file has got an additional status, wherein unsuccessfully processed images get FAIL status instead of DONE. The wrapper script forcel-l2ps_ was discontinued; force-level2 is now directly calling force-l2ps.

    • A new parameter STRICT_WATER_VAPOR was introduced, which will result in process abortion when the water vapor database does not hold a day-specific value, i.e. the usage of the climatology is forbidden. Applies to Landsat only.

  • FORCE AUX

    • new auxilliary program force-info, which replaces force effective immediately. The new program has the same functionality, but does not hardcode the available tools. Tools that are still missing the -i option are currently not listed though. This still needs to be done.

    • new auxilliary program force-init. This program will create a new project with reasonably named folders that can be used as a starting point for a new project with some suggestions on organizing things. Unused folders can be deleted and the naming and structure is a mere suggestion and by no means prescriptive or mandatory. This tool is especially meant for beginners.

    • new auxilliary program force-datacube-size. This program prints the size of your datacube, per sensor and in total.

    • new auxilliary program force-hist. This program computes the histogram of image values (can be vrt) and writes a csv table. This is intended to be used in a validation workflow.

    • new auxilliary program force-sample-size. This program computes the required sample size following Olofsson et al. 2013/2014, which should be used to properly validate a classification map. This is intended to be used in a validation workflow.

    • new auxilliary program force-stratified-sample. This program draws a stratified random sample based on a classification map and sample size computations. This is intended to be used in a validation workflow.

    • new auxilliary program force-map-accuracy. This program computes area-adjusted accuracies following Olofsson et al. 2013/2014. This is intended to be used in a validation workflow.

    • force-tabulate-grid has been updated to produce properly named output files. The default output file name is grid.kml, created in the current directory, using the KML format. -o can override the output file name (including path and extension). -f can override the file format, use the GDAL vector drive short name - thus grid can now be generated in any GDAL vector format. Both need to be given if not using defaults.

    • force-tile-extent has been updated to common Linux usage with options. Small changes for users are necessary, though all the functionality remained the same

    • The CLI help of force-tile-finder has been corrected concerning the separator for the coordinates.

  • **FORCE L1AS*

    • force-level1-sentinel2 was finally removed after being deprecated for a while now. Use force-level1-csd. Contributions to substitute this program using the Copernicus Data Space Ecosystem endpoint are welcome and appreciated!