Develop version
FORCE HLPS
Analogous to Python UDFs, R UDFs can now be used through FORCE. This applies both to the TSA and UDF submodules. Three new parameters were added:
FILE_RSTATS
,RSTATS_TYPE
,OUTPUT_RSP
. At least two functions must be present in the UDF:force_rstats_init
, as well asforce_rstats_pixel
orforce_rstats_block
(depending whetherRSTATS_TYPE
is set to PIXEL or BLOCK). To run PIXEL-type functions, you need to install thesnow
andsnowfall
packages. To run the FORCE components that are now compiled against the R API (mostly force-higher-level, but also force-qai-inflate), you need to provide two environment variables, i.e.R_HOME
andLD_LIBRARY_PATH
. You can do it like this:export R_HOME=$(R RHOME)
export LD_LIBRARY_PATH=$R_HOME/lib
force-higher-level parameters.prm
In the pre-built Docker container, everything is pre-configured already.
Added a new parameter:
STREAMING = FALSE
can disable the streaming strategy of FORCE HLPS. When TRUE (previous behaviour, and still the default), FORCE will perform reading, computing and writing at the same time. If FALSE, these operations are done after one another in sequential mode. Disabling streaming might be necessary for some UDFs that otherwise produce threading conflicts with the internally used OpenMP functionality. When using R UDFs, FORCE will issue a warning and gracefully disable streaming.Added a new parameter:
PRETTY_PROGRESS
. HLPS will display progress information on screen. When TRUE (previous behaviour, and still the default), the progress information overwrites itself to produce a pretty displayal. However, this can cause error messages (or printing in UDFs) to be overwritten. If disabled (FALSE), the progress information will be simply be appended on screen (stdout).Removed warning message that no output is produced when using the sampling submodule. It is now checked whether the files are actually written and will only warn if no input was detected or if no files were written. The behaviour for all other submodules stays the same.
It is now possible to specify the product types that should be used. This is to give the user more freedom with respect to data sources to be used. There are two new parameters:
PRODUCT_TYPE_MAIN
andPRODUCT_TYPE_QUALITY
. The main product is usually a reflectance product likeBOA
. When using composites, you may use BAP. This can be anything, but make sure that the string can uniquely identify your product. As an example, do not useLEVEL2
as this will not filter products apropriately. Note that the product should contain the bands that are to be expected with the sensor used, e.g. 10 bands when sensor is SEN2A. The quality product type should be a bit flag product likeQAI
. When using composites, you may use INF. This can be anything, but make sure that the product contains quality bit flags as outputted by FORCE L2PS. As an exception, it is also possible to giveNULL
if you don’t have any quality masks. In this case, FORCE will only be able to filter nodata values, but no other quality flags as defined withSCREEN_QAI
. Feature requested by Marcel Schwieder and Felix Lobert.In the TSA submodule, it is now possible to disable fitting a monotonic trend within the harmonic model. The new parameter
HARMONIC_TREND = TRUE/FALSE
was added. Default behaviour (as before) isTRUE
.The Level 3 submodule now allows to use the internal scores for determining the best available observation as as a weight, i.e., not performing a selection, but a weighted average. This idea is taken from the MAJA WASP approach of Olivier Hagolle. A new parameter
SELECT = TRUE/FALSE
was introduced. withFALSE
enabling the weighted average. Note that theINF
andSCR
products will not be produced when using this option.The Level 3 submodule received the new parameters
USE_CLOUDY
andUSE_HAZY
, which can be used to allow observations with a cloud or haze score of less than 1% in the composite. On one hand, this reduces nodata gaps in the composite. On the other hand, it will include clouds or haze if there is no better observation.The Level 3 submodule received a new parameter
COMBINE_SCORES
, which allows you to choose between additive or multiplicative combination of the individual scores into the total score. Additive combination is the previously used method and remains the default. Multiplicative combination can be used, which is much stricter in case one observation is characterized by a very poor suitability in a single score.a new parameter
OUTPUT_SUBFOLDERS
was implemented that allows to output the data into subfolders under the tile folder instead of just dumping everything into the tile folder itself. It is planned that this feature will be expanded in the future.OUTPUT_SUBFOLDERS = FALSE
remains the default.
FORCE L2PS
force-level2 was refactored. Little has changed for users. The logfiles are more verbose, and log messages generated by
force-level2
and by the core processforce-l2ps
are more cleary separated. The queue file has got an additional status, wherein unsuccessfully processed images getFAIL
status instead ofDONE
. The wrapper scriptforcel-l2ps_
was discontinued;force-level2
is now directly callingforce-l2ps
.A new parameter
STRICT_WATER_VAPOR
was introduced, which will result in process abortion when the water vapor database does not hold a day-specific value, i.e. the usage of the climatology is forbidden. Applies to Landsat only.
FORCE AUX
new auxilliary program force-info, which replaces force effective immediately. The new program has the same functionality, but does not hardcode the available tools. Tools that are still missing the
-i
option are currently not listed though. This still needs to be done.new auxilliary program force-init. This program will create a new project with reasonably named folders that can be used as a starting point for a new project with some suggestions on organizing things. Unused folders can be deleted and the naming and structure is a mere suggestion and by no means prescriptive or mandatory. This tool is especially meant for beginners.
new auxilliary program force-datacube-size. This program prints the size of your datacube, per sensor and in total.
new auxilliary program force-hist. This program computes the histogram of image values (can be vrt) and writes a csv table. This is intended to be used in a validation workflow.
new auxilliary program force-sample-size. This program computes the required sample size following Olofsson et al. 2013/2014, which should be used to properly validate a classification map. This is intended to be used in a validation workflow.
new auxilliary program force-stratified-sample. This program draws a stratified random sample based on a classification map and sample size computations. This is intended to be used in a validation workflow.
new auxilliary program force-map-accuracy. This program computes area-adjusted accuracies following Olofsson et al. 2013/2014. This is intended to be used in a validation workflow.
force-tabulate-grid
has been updated to produce properly named output files. The default output file name isgrid.kml
, created in the current directory, using theKML
format.-o
can override the output file name (including path and extension).-f
can override the file format, use the GDAL vector drive short name - thus grid can now be generated in any GDAL vector format. Both need to be given if not using defaults.force-tile-extent
has been updated to common Linux usage with options. Small changes for users are necessary, though all the functionality remained the sameThe CLI help of force-tile-finder has been corrected concerning the separator for the coordinates.
**FORCE L1AS*
force-level1-sentinel2 was finally removed after being deprecated for a while now. Use force-level1-csd. Contributions to substitute this program using the Copernicus Data Space Ecosystem endpoint are welcome and appreciated!