Apart from a lot fixes, changes and additions, a lot of work has gone into replacing deprecated code. Since Java has been changing more rapidly since Java 9, quite a lot of code needed tweaking. Subtle changes between Java versions (e.g., ordering in Properties) were breaking the regression unit tests. In some cases, I had to resort to Java-version specific test output to compare against. But most importantly, ADAMS now successfully compiles, executes its tests and runs under the following Java versions: 11 (LTS), 17 (LTS), 21 (LTS), 22 and 23. Code base and compiled output are still targeting Java 11.

Fixes

• When enabled in the Flow editor, garbage collection is now run after the flow finished and when the flow is being cleaned up.

• Fixed handling of mnemonics and accelerators for properties-based menu items (e.g., Spreadsheet file viewer).

• Editing enums in the GenericArrayEditor now allows adding multiple enums via their text-representation (fixed out of bounds exception).

• Spreadsheet tables now apply numeric filters to numeric values correctly.

• Debug view in the Flow editor no longer hides the notification area when a flow finishes.

• The Storage debugging panel in the Flow editor had its Edit operation fixed.

• The GOE editors for annotations, text and markdown now respect the line wrap state when reopening the editor.

• The GenericObjectEditor no longer updates the current value of a chooser panel while editing an object, only when accepting the changes.

• The FlowRunner now uses BackgroundScriptingEngineRegistry for stopping scripting engines.

• adams-imaging: The PolygonPointAnnotator now plots a small circle or a line when there are only one or two points selected before plotting a polygon.

• adams-compress: The GUNZIP, UnBzip2 and UnTar transformers now decompresses the data till they reach the end-of-file, handling concatenated streams now correctly.

• adams-spectral-2dim-core: improved performance of the BulkSampleDataDbWriter by fixing the underlying

  SampleDataT code (auto-commit was always on regardless of parameter).

• adams-weka and adams-weka-lts: Instances tables now apply numeric filters to numeric values correctly.

Changes

• Variables and storage in the Flow editor are now available for inspecting after flow execution (even when stopped) till the flow is being cleaned up.

• The FlatLaf* look'n'feel themes now show scrollbars again.

• The ViewDataClickAction of the SequencePlotter and SimplePlot sink now allows the user to select the outline of a polygon and display the data points that fall within that polygon. The polygon is defined (invisible unless the PolygonSelectionPaintlet overlay paintlet is specified) by SHIFT+left-click on vertices and SHIFT+right-click to finalize the polygon (CTRL+right-click discards any points).

• The WaitForFile transformer now uses the file use check class hierarchy.

• The Storage debugging panel in the Flow editor now can copy the name/cache to the clipboard via the right-click popup menu, as well as editing and exporting.

• The LocalScopeTransformer control actor has been renamed to LocalScopeSubProcess to better align with the naming conventions.

• The DeleteFile transformer now handles arrays of strings/files as well.

• The GenericObjectEditor now has an improved undo mechanism: it is now possible to revert to a previous object when switching between objects via the class tree.

• adams-groovy: Upgraded groovy dependencies to 4.0.23 to work with Java 22

• adams-imaging:

  • The ImageObjectInfo transformer can output the rectangle/polygon area now and supports scaling.

  • Upgraded commons-imaging to 1.0.0-alpha5

  • The ImageObjectFilter can now work off a copy of the incoming report/report handler rather than modifying in-place (default).

• adams-spectral-2dim-core:

  • The RowWiseSpreadSheetSpectrumReader spectrum reader has improved support for chunked spreadsheet readers now; added support for row filtering.

  • The RowWiseSpreadSheetSampleDataReader now has support for row filtering.

• adams-spreadsheet:

  • Added chunking support to FastCsvSpreadSheetReader

  • Added support for running garbage collection in Spreadsheet file viewer when closing sheets/pages (File / GC on close).

  • The Unique values column operation now opens a modeless text dialog, enabling multiple dialogs to be open at the same time.

  • Spreadsheet tables now limit the number of characters that are displayed in the title (default: 40) to avoid overly wide columns. That can be adjusted with key "SpreadSheetTableMaxColumnNameChars" in GUIHelper.props.

• adams-visualstats: The ViewDataClickAction of the Scatterplot sink now allows the user to select the outline of a polygon and display the data points that fall within that polygon. The polygon is defined by SHIFT+left-click on vertices and SHIFT+right-click to finalize the polygon (CTRL+right-click discards any points).

• adams-weka and adams-weka-lts:

  • Upgraded jgrapht-treevisualize-weka-package to 2024.8.19 to address CVE-2022-42889 introduced by commons-text:1.5

Additions

• The BlockingSearch search plugin for the FileSystemSearch waits till the specified base search returns a non-empty results (or the specified timeout is reached).

• The ListClassUsage actor processor generates a list of all occurrences of a class (or derived classes, when enabled) in a flow.

• The LocalScopeTee control actor works like the normal Tee actor but with local scope for variables/storage.

• The LaunchTee and LaunchTrigger control actors launch their respective sub-flows in separate threads (no limit to number of threads), each with their own local scope. Can be used for launching external processes, like Docker containers that keep running.

• adams-imaging:

  • Added support for generating sub images from predefined regions that have a label associated with the LabeledRegions generator - plugin for the SubImages BufferedImageTransformer.

• adams-spectral-2dim-core:

  • Added the MultiColumnSpreadSheetSampleDataReader reader for sample data, which reads reference data row-wise, using the specified column range.

  • Added the BulkSpectrumDbWriter transformer to make bulk imports of spectra faster. The recommended setting is to use a new connection and to turn auto-commit off, to dramatically increase throughput.

• adams-spreadsheet:

  • The Overlay multi-spreadsheet operation combines multiple sheets into a single one by overlaying non-empty cells at the same coordinates, with the last non-empty cell overwriting any prior content.

Have a good weekend!

Quite a few changes this time round: a number of libraries have been upgraded to address security vulnerabilities (which may or may not apply), Weka's repeated cross-validation which was recently added to the Weka Investigator is now available in the flow as well, module adams-jep was added to allow executing Python scripts from within flows, thanks to the jep project.

Fixes

• Fixed the title of the File monitor interface (was incorrectly Flow runner).

• Runtime-only tabs in the Flow editor (e.g., from breakpoints) are no longer affected by the enabling/disabling all actions.

• adams-core: Screenshots in the manual have been updated to reflect the refreshed interfaces.

• adams-weka and adams-weka-lts:

  • Removing multiple datasets from the Weka Investigator no longer causes exceptions trying to access the selected rows.

  • Reordering datasets in the Weka Investigator also no longer causes exceptions (due to automatic data structure cleanups).

  • Screenshots in the manual have been updated to reflect the refreshed interfaces.

Changes

• Added favorites buttons to the generic object and array editor panels.

• The Favorites menu has been reworked and it is now possible to add multiple named temporary favorites, ones that only last the session. See also this video: https://youtu.be/zu_sNTUh9ZY?si=ZNK00GXsS1-A6TUH

• The tree button for selecting a class from the class tree in the GenericObjectEditor has been removed, as it has been superceded for a while now by the class tree that is shown in the left panel of the GenericObjectEditor.

• Added the Keep open option to graphical output, i.e., the window will stay open even when the flow cleans up graphical output; the user needs to explicitly close the window. See the checkbox menu item in the File menu of the window.

• Upgraded MySQL JDBC driver to 9.0.0 to address CVE-2023-22102

• Upgraded flatlaf look'n'feel to 3.4

• adams-access: Upgraded jackcess to 4.0.7

• adams-boofcv: Upgraded to 1.1.5

• adams-compress:

  • Using Apache commons compress library for LZMA now instead of lzma-java

  • Added support for lzf, lzma and tar in the File commander

  • Upgraded xz dependencty to 1.10

  • Upgraded zstd-jni dependency to 1.5.6-4

• adams-db:

  • Upgraded postgresql JDBC driver to 42.7.3 to address CVE-2024-1597

  • Upgraded hsqldb to 2.7.3

  • Upgraded sqlite-jdbc to 3.46.0.1

  • Upgraded mssql-jdbc to 12.8.0.jre11

• adams-json:

  • The JsonProducer option generator now only outputs the values that differ from their defaults values. This can be overridden by setOutputFull(true).

  • Upgraded json-smart to 2.5.1

  • Upgraded gson to 2.11.0

• adams-moa: upgraded MOA to 2024.07.0

• adams-net and adams-html: upgraded requests4j to 0.2.6

• adams-pdf: upgraded itextpdf to 5.5.13.4 to address CVE-2017-20151

• adams-rsync: upgraded rsync4j to 3.3.0-1

• adams-scripts: added -venv PATH option to exec.sh/.bat and launcher.sh/bat to activate a Python virtual environment

• adams-webservice-core: upgraded CXF to 3.6.4 to address CVE-2024-28752

• adams-weka and adams-weka-lts:

  • added favorites button for the output generators in the Weka Investigator.

  • upgraded xgboost4j_2.12 to 2.1.0, requiring glibc 2.28 or later under Linux

  • upgraded Weka to 3.9.6-fork-0.0.3 (non-LTS) and 3.9.0-fork-0.0.15 (LTS) to fix issue with AbstractClassifier trying to access batchsize and numDecimalPlaces members when deserializing old models.

Additions

• adams-compression: Added the TarExport multi-object export plugin, in addition to the existing ZipExport plugin. The export supports automatic compression based on the file extension being used, e.g., gzip (.tar.gz) or bzip2 (.tar.bz2).

• adams-jep: new module for running Python scripts via Jep (https://github.com/ninia/jep/) from within ADAMS. Requires ADAMS to be started from the shell with the correct Python environment. When running flows in parallel, the user needs to ensure that the variable names do no interfere due to global scope. Video demonstrating the use of jep: https://www.youtube.com/watch?v=26WiGDN_-KA

• adams-weka and adams-weka-lts:

  • Added support for repeated cross-validation to the flow with the transformers WekaRepeatedCrossValidationEvaluator and WekaRepeatedCrossValidationOutput.

  • Added support for the Weka Package Manager in the flow with the WekaPackageManagerAction standalone/source/transformer actors and the WekaPackageToMap conversion.

This time round, there have been a bunch of important bug-fixes and additions. The Weka Investigator can now perform repeated cross-validation, calculating statistics across the runs (the Experimenter does this per fold, the Investigator per full CV) and also a new tab for executing an experiment run on the current dataset and the chosen classifier.

Last, but not least, ADAMS turned 15 today (first commit in 2009). :-)

Fixes

• adams-pdf: PDF fonts now properly parse the attributes in the font string (Arrays.binarySearch on unsorted array returned invalid index for Italic).

• adams-weka and adams-weka-lts:

  • The Multi-Experimenter now sorts the results according to dataset/classifier order (dataset first).

  • Leave-one-out cross-validation (LOOCV) now works correctly in the Multi-Experimenter.

  • The Rename... action for renaming a column of an InstancesTable now uses the correct column index.

  • Changed the R^2 calculation to https://en.wikipedia.org/wiki/Coefficient_of_determination#Definitions

Changes

• adams-core:

  • spreadsheet formulas can now use median as well

  • upgraded commons-io to 2.16.1

• adams-imaging:

  • upgraded opex4j to 0.0.3

  • the ObjectCentersFromReport plugin in the Preview browser now allows changing the report reader

  • object finders now have a quiet flag to suppress warnings about -1 indices

• adams-spreadsheet:

  • The SpreadSheetAggregate transformer now offers the 25th and 75th quartile as well.

• adams-weka and adams-weka-lts:

  • upgraded common-csv-weka-package to 2024.7.5

Additions

• adams-imaging:

  • The image object filter ObjectsInPolygon finds annotations that fall within the specified polygon.

  • The CountObjectsInPolygon transformer outputs the number of annotations that fall within the specified polygon.

• adams-weka and adams-weka-lts:

  • Added support for repeated cross-validation to the Weka Investigator. See this video: https://youtu.be/ZlbynLCq8mc?si=HEkY0I_rQyaq6mD0

  • Added an Experiment tab to the Weka Investigator for evaluating single classifier on selected dataset. See this video: https://youtu.be/bVid9WrpE0s?si=KJcyUT33RbGiVF89

A lot of library upgrades happened again. Most notably are the introduction of quick actions in the Flow editor that can be invoked via SHIFT+right-click or SHIFT+F10. These are actor-specific actions like changing the variable name/value of a SetVariable actor or editing the boolean condition of an actor. These quick actions avoid having to go through the full-blown GenericObjectEditor dialog, saving a few mouse clicks. The ArrayProcessor received experimental parallel execution support. Items added to queues can have a delay in msec added to them, making them available at the other only once the delay has passed.

Fixes

• adams-core:

  • The Percentage generator for the ArraySubsetGenerator transformer now works as expected; added ability to oversample as well.

  • The SQL.tableExists method now works correctly with the newer MySQL JDBC driver (limiting DB meta-data to a particular database does not seem to work anymore, as it iterates over ALL databases).

  • Fixed the exception that occurred when selecting "Use default" from the popup menu in the GOE for an array property.

• adams-docker: The management interfaces for Docker containers and images now gather the output from all commands rather than just the last one.

• adams-imaging: The barcode encoders now use the correct x/y position (was off by 1).

• adams-ml: The ConfusionMatrix transformer, when generating sums to calculate percentages, no longer throws an error when probabilities instead of counts are taken into account (ie a probability column was specified).

Changes

• adams-core:

  • removed JVisualVM menu item from Help menu (user-mode: developer/debug) as it is no longer bundled with JDKs

  • added "ADAMS home directory" menu item (user-mode: developer/debug)

  • upgraded sizeofag to 1.1.0

  • The HashSet source now allows customizing the column name of the generated spreadsheet

  • upgraded commons-compress to 1.26.0

  • upgraded commons-io to 2.15.1

  • upgraded commons-codec to 1.16.1

  • upgraded processoutput4j to 0.0.12

  • GetReportValue can apply a conversion to the obtained report value now before forwarding it

  • The Flow editor now offers actor-specific quick actions via the SHIFT+right-click popup menu on a selected actor, e.g., renaming a variable name or changing the variable value. Using SHIFT+F10. this menu can be displayed directly via keyboard.

  • Reworked the notes support for files in the Preview browser, associating notes now with directory and file, allowing the user to edit the notes.

  • The Preview browser now has a Send to sub-menu in the popup menu for the local files as well, not just from the main menu.

  • CallableX actors no longer call the wrapUp()/cleanUp() methods of their referenced actor, as this should happen already.

  • Added experimental parallel execution support to the ArrayProcess control actor.

  • Added support for delayed queues, i.e., possible retrieval of queue items can be delayed by a certain delta of msecs. The retrieval delay gets defined in the EnQueue sink.

• adams-compress: added commons-codec depdendency

• adams-db:

  • upgraded hdsqldb to 2.7.2

  • upgraded sqlite-jdbc to 3.45.0.0

  • upgraded mssql-jdbc to 12.4.2.jre11

• adams-docker: The management interfaces for Docker containers and images allow pruning now as well.

• adams-imaging:

  • The YoloAnnotationsReportReader now has the -use-polygon-format option to enforce reading the data in polygon format rather than auto-detecting it.

  • The image segmentation panel now has the following buttons for controlling the visibility of layers: All, None, Invert

  • The object annotations panel now has a button for toggling the visibility of the annotations.

• adams-json: upgraded json-path to 2.9.0

• adams-maps: upgraded postgis-jdbc to 2023.1.0

• adams-nlp:

  • upgraded jsoup to 1.17.2

  • The EditDistance transformer can process string arrays of length 2 now as well rather than having to set the base string via a variable.

• adams-opencv: upgraded javacv to 1.5.10

• adams-rats-core: The EnQueue rat output now supports delayed queues with its retrieval delay option.

• adams-spectral-2dim-core:

  • GetSampleDataValue can apply a conversion to the obtained report value now before forwarding it

  • The SpectrumDisplay sink can accept arrays of spectra now, to speed up display of data

• adams-weka and adams-weka-lts: The attribute summary panel used in the Preprocess tab of the

  Investigator (displaying histogram/bar chart) now allows the user to customize the colors. The default color provider can be customized via the Preprocess.AttributeSummaryColorProvider property in the WekaInvestigatorSession.props config file or the Preferences dialog.

• adams-weka-lts: upgraded Weka to 3.9.0-fork-0.0.14 (updated package manager classes)

Additions

• adams-core:

  • added the SimplePercentage generator for the ArraySubsetGeneration transformer.

  • added the Copy file to... action to the files list in the Preview browser.

• adams-imaging: added the EncodeBarcode transformer that turns a string payload into the specified barcode, e.g., QR code.

• adams-weka and adams-weka-lts:

  • The RemoveWithLabels filter allows removing instances with labels that match a regular expression.

  • The AlignDataset filter aligns the dataset being filtered with an external dataset, e.g., making train/test sets compatible.

  • In the GenericObjectEditor, it is now possible to enclose clusterers in SingleClustererEnhancers, just like with classifiers. In addition, the opposite operation "pull up" has been added as well.

Since 2012, ADAMS has hosted its release files on Sourceforge.net.

This morning, I had an email from them in my inbox, stating that I need to remove old release files as the project is way over its 5GB file quota. To be honest, I was not aware that there was a quota and I also did not see any news that they will be enforcing such a quota.

Either way, I have removed old release files and the ADAMS website is now hosting the old releases (i.e., downloads may be slower). To further conserve space, the Debian (.deb) and Redhat (.rpm) release packages have been retired as well. But you can still download current .deb/.rpm packages via the daily snapshots.

Especially in corporate or institutional environments, users typically don't have any permissions to install software. Linux users are typically familiar with installing software in their own home directory, but Windows users are often not aware that this is even possible.

ADAMS itself does not rely on an installer, all you have to do is download a ZIP file (either release or snapshot), decompress the archive and then you are good to go - as long as you have Java installed on your system.

But what if you shouldn't have Java already on your computer? No problem, you don't need admin rights in order to get Java on your computer. You can simply download a JDK, decompress the archive and set the JAVA_HOME environment variable to that directory. ADAMS automatically looks for the JAVA_HOME environment variable.

JDKs can be downloaded from here: https://adoptium.net/temurin/releases/

E.g., Java 11 for Windows 64bit (download the zip): https://adoptium.net/temurin/releases/?variant=openjdk11&os=windows&arch=x64&package=jdk

Or Java 11 for Linux 64bit (download the tar.gz): https://adoptium.net/temurin/releases/?variant=openjdk11&os=linux&arch=x64&package=jdk

Later LTS (long term support) versions like 17 or 21 should work as well, but you only need 11 as a minimum at the time of writing.

Setting environment variables under Windows:

• Windows 10: https://www.alphr.com/environment-variables-windows-10/

• Windows 11: https://www.alphr.com/set-environment-variables-windows-11/

If you are running Linux, I presume you know how to manage environment variables via the export command in your $HOME/.bashrc. config file :-)

More detailed instructions:

• Download the JDK archive from the above URL

• Decompress the downloaded archive and move the decompress directory into your home directory. At the time of writing, you would find a directory called jdk-11.0.21+9 after decompressing the archive, that is the directory to move. E.g., on a Windows machine, I would place simply in my Documents directory. On Linux, I could place it straight in my home directory.

• Copy the location of this directory onto the clipboard (e.g., for me that would be C:UsersfracpeteDocumentsjdk-11.0.21+9)

• Create the JAVA_HOME environment variable and set its value to this directory

• Download the ZIP file for the ADAMS version that you want to use (e.g., a shapshot: https://adams.cms.waikato.ac.nz/download/snapshot/)

• Decompress the ADAMS zip file and move the directory in there into your home directory. As a Windows user, this could be your Documents directory again.

That's it for the setup. You can now start ADAMS via the start_gui.bat (Windows) or start_gui.sh (Linux) script, which is located in the bin directory of your ADAMS installation.

After the release a couple of weeks ago, I spent a considerable amount of time upgrading as many dependencies as possible to address known security vulnerabilities.

Another major change is the switch to version 8.x of the MySQL JDBC driver by default (you may need to add ?sessionVariables=sql_mode='ALLOW_INVALID_DATES' to your connection string).

Last, but not least, I've upgraded a range of actors to (optionally) make use of the new JobRunnerInstance standalone actor which can be used for better managing resources (cores/memory) when running heavily parallelized flows that, e.g., train classifiers or filter data. Going through this bottleneck, it is easier to avoid starving the system of resources by limiting the number of jobs that can run in parallel.

Fixes

• adams-core:

  • The BaseColorTextField now updates the button's color correctly under the cross-platform and FlatLaf look'n'feels.

• adams-imaging:

  • The EvaluatePrediction image segmentation operation now skips the percentage calculation when the total number of annotated pixels for a layer is 0.

Changes

• general

  • upgraded postgresql to 42.7.1

  • upgraded xercesimpl to 2.12.2

  • upgraded requests4j to 0.2.5

• adams-access: upgraded jackcess to 4.0.5

• adams-bootstrapp: upgraded bootstrapp to 0.1.13

• adams-compress:

  • upgraded compress-lzf to 1.1.2

  • upgraded zip4j to 2.11.5

  • upgraded junrar to 7.5.5

  • upgraded zstd-jni to 1.5.5-11

• adams-core:

  • The ExecutionLog flow execution listener now has the ability to specify what stages to log. Also outputs to stdout now if the log file is pointing to directory.

  • The new JobRunnerSupporter interface is for actors that can execute the resource-intense workload via a JobRunnerInstance standalone within their scope. Requires a JobRunnerSetup standalone further up in the flow, defining how to execute the jobs.

  • switched MySQL JDBC driver from 5.1.49 to 8.0.33

  • upgraded FlatLaf look'n'feel to 3.3

• adams-groovy: upgraded groovy-all to 3.0.20

• adams-html:

  • upgraded jetty-server/-http to 9.4.51.v20230217

  • upgraded nekohtml to 1.9.22

• adams-imaging:

  • using commons-codec 1.16.0 (and excluding dependency from lire)

  • upgraded zxing to 3.5.1

  • upgraded jts-core to 1.19.0

• adams-imaging-boofcv:

  • upgraded boofcv to 1.1.2

  • removed the HistogramBasedIntensityAdjustmentGray8 transformer

• adams-json:

  • upgraded json-path to 2.8.0

  • upgraded json-smart to 2.5

• adams-math: jsci now uses xercesimpl 2.12.2 (globally defined)

• adams-matlab: upgraded mfl-core to 0.5.15

• adams-net:

  • upgraded jsch to 0.2.16

  • upgraded tika-core to 1.28.5

• adams-pdf:

  • upgraded pdfbox to 2.0.30

  • upgraded icepdf-core/-viewer to 7.1.3

• adams-pyro4: upgraded pyrolite to 4.30

• adams-redis: upgraded lettuce-core to 6.3.1.RELEASE

• adams-rest: jaxws-rt now excludes woodstox-core dependency, instead of using 6.5.1 directly

• adams-spectral-2dim-core:

  • added sortOnSampleID as option to the spectrum conditions.

  • The following actors now implement the JobRunnerSupporter interface: Cleaner, Evaluator.

• adams-spreadsheet:

  • upgraded jfreechart to 1.5.4, added jcommon 1.0.24

  • The SpreadSheetFilter transformer now implements JobRunnerSupporter.

• adams-timeseries: excluding commons-codec from timeseriesForecasting dependency, using 1.16.0 instead

• adams-video: upgraded screencast4j to 0.1.5

• adams-webservice:

  • upgraded cxf to 3.6.2

  • upgraded xjc to 3.3.2

• adams-weka and adams-weka-lts: The following actors now implement the JobRunnerSupporter interface: WekaFilter, WekaTrainAssociator, WekaTrainClassifier, WekaTrainClusterer, WekaTestSetEvaluator, WekaTrainTestSetEvaluator.

• adams-yaml: upgraded snakeyaml to 2.2

Additions

• adams-core: The standalone JobRunnerInstance allows limiting resource intense jobs, like model training and filtering data, via a bottleneck within the actors' scope. That way heavily multi-threaded flows (e.g., nested Branch actors) won't starve the system of resources, such as CPU cores and memory. Actors implementing the JobRunnerSupporter interface can make use of that feature.

Retired modules

• adams-db-mysql8: removed as now superceded by 8.x dependency in adams-core module

ADAMS 24.1.0 has been released. More details and download links are available from here.

Happy New Year everyone!

We have moved from Twitter/X to Mastodon: @[email protected].

Mostly bug-fixes and addressing security vulnerabilities since the last update. Also, ADAMS is now more stable under the long-term support versions of OpenJDK 17 and 21.

Fixes

• adams-core:

  • The PolygonUtils.boundingBox method now works correctly.

  • The Display sink, when used in conjunction with the Inspect or DisplayPanelManager actors, uses the font specified by the user now.

  • The StorageValuesArray and VariablesArray source actors now properly collect errors occurring when applying the conversion, making it possible in the Flow editor to jump to the actor.

• adams-imaging:

  • The PolygonPointAnnotator now clears the points afte accepting an annotation.

  • The flow adams-imaging-annotate_objects.flow now supports the following annotation formats: ADAMS .report, OPEX .json

  • The TesseractHelper class now uses --psm instead of -psm for the page segmentation parameter when assembling the command-line arguments.

• adams-json: The GetJsonValue transformer now accepts JSON arrays again when using a proper JsonPath expression (i.e., one that starts with $).

• adams-scripts: added necessary --add-exports options to make it work under Java 17 and 21.

Changes

• adams-bootstrapp: bootstrapp upgraded to 0.1.12

• adams-core:

  • The expressions for boolean, mathematical, report math, string expressions and spreadsheet formulas now support the startswith and endswith string functions.

  • upgraded flatlaf to 3.2.5

  • upgraded jodd-core to 5.1.6

  • removed JULIANDATE/JULIANDATE_LONG DateTimeType due to newer versions of jodd-core no longer supporting it

• adams-gnuplot: the scriptlets now support quick info output

• adams-json: gson upgraded to 2.10.1

• adams-net:

  • tika-core upgraded to 1.28.3 to address CVE-2022-30973.

  • switched from javax.mail:1.5.6 to jakarta.mail:1.6.5

  • ApacheSendMail is now deprecated (broken)

  • upgraded jodd-mail to 5.1.6

• adams-opencv: javacv upgraded to 1.5.9

• adams-rsync: rsync4j upgraded to 3.2.7-5

• adams-pdf: The Headline PDF proclet now expands variables in the headline automatically.

Additions

• adams-core: added the GetListElement transformer for obtaining a specific element from a list.

• adams-imaging-ext: added the flow adams-imaging-ext_run-sam-hq.flow which allows using SAM-HQ (https://github.com/SysCV/sam-hq) via Redis for annotating images.

A lot of different changes have occurred over the last few months. There are two that stand out. First, docker related: images now get published under the waikatodatamining user; overhaul of how flows are used within docker images and updated documentation on website accordingly. Second, some interactive sources like EnterValue or the boolean condition PromptUser can now be displayed as a panel in the notification area rather than as a separate dialog. That way, dialogs will stay with the particular flow editor panel, even when starting multiple flows at the same time (which is also a new feature).

Full list of changes is below:

Fixes

• Swapping a SelectFile source with a EnterManyValues now transfers any extensions (or attached variable) as well.

• When selecting Save report from a report table with the Edit options box ticked, the writer's options dialog will only get displayed once now.

• Fixed source of memory leaks with the recent files handlers (handlers are now always set/get as shallow copies).

• AbstractTee, LoadBalancer, LocalScopeTransformer and WhileLoop now set the skip flag also in the underlying Sequence actor that manages their actors.

• The AbstractDataContainerFileWriter (e.g., the SpectrumFileWriter is derived from that) now ensures that the filename generated from the samepl ID is a valid filename.

• adams-weka and adams-weka-lts: data loaded into the Weka Investigator now adhers to the Investigator-wide undo state.

• When in headless mode, entering values in the terminal via EnterValue and EnterManyValues now makes use of provided initial values when simply pressing ENTER.

• Added support for environment variables ADAMS_USERNAME, ADAMS_USERHOME, ADAMS_USERDIR to override system properties user.name, user.home and user.dir respectively. The ADAMS_PLACEHOLDERS variables allows setting/overriding of placeholder definitions. Values in ADAMS-managed properties files can be overridden as well (check the adams-core manual for details, section Environment variables).

• adams-rats-core: The RatControl now skips ConsolePanel calls when in headless mode.

Changes

• Improved clearing of graphical displays (via: File - Clear).

• The Downloads > Docker documentation has been updated and expanded to fit the updated Docker images.

• adams-core:

  • The AbstractDataContainerFileReader transformer ancestor (e.g., for SpectrumFileReader) now supports output of the containers as an array instead of just one-by-one.

  • Added the ability to pause/resume or stop the flow from graphical actor dialogs via an optional flow control sub-menu. This is useful for flows that get launched with FlowRunner and do not have the typical Flow editor interface. Requires EXPERT user mode.

  • The Statistics menu item in the Flow editor now displays the results in the side panel rather than in a separate dialog.

  • The Flow Editor now has the additional Run all and Stop all menu items/toolbar buttons.

  • The Active menu and toolbar buttons in the Flow Editor are now hidden if no flow set as active.

  • The FlowRunner how has the -force-exit flag to properly exit the process in case 3rd-party library threads keep the application from exiting.

  • The PromptUser boolean flow condition can run in headless mode now as well and also be displayed in the notification area rather than as a dialog.

  • The EnterValue, EnterManyValues, SelectDirectory and SelectFile sources can be displayed in the notification area as well, not just as dialog.

  • added interfaces StringReportReader/Writer to allow them being used by the ReportToString/StringToReport conversions to perform conversions on the fly rather than via temp files.

  • Added MSECS_LONG, SECONDS_LONG, JULIANDATE_LONG and SERIAL_DATETIME_LONG as date time types, affecting conversions ConvertDateTimeType, DateTimeTypeToString, ExtractDateTimeField, StringToDateTimeType and transformer DateTimeTypeDifference.

  • The MathExpression transformer now accepts Byte/Short/Float/Number as well

• adams-db: upgraded sqlite-jdbc to 3.41.2.2.

• adams-imaging:

  • The ImageInfo transformer can output the image type and pixel depth now as well.

  • The Regions sub-images generator now stores the region coordinates in the report of the sub-image as well ("Coordinates[S]").

  • Object finders now have the resetIndicesIfNecessary flag that automatically resets the indices when there are missing or duplicate ones.

• adams-moa: upgraded MOA to 2023.04.0.

• adams-net:

  • The DownloadContent transformer and DownloadFile sink now stop active downloads when the flow is being stopped.

  • Upgraded jsch dependency to 0.2.9

  • The SSHConnection standalone now allows specifying a timeout (in milliseconds).

• adams-rsync: Upgraded rsync4j to 3.2.7-2

• adams-weka and adams-weka-lts:

  • When specifying an ID attribute with the WekaInstanceViewer sink then this attribute gets automatically excluded from the plot (avoids distorting the graph).

  • Weka Investigator: memory clean-up improvements.

  • Upgraded Weka versions: weka==3.9.6-fork-0.0.2, weka-lts==3.9.0-fork-0.0.13

• Docker images are now available from: https://hub.docker.com/u/waikatodatamining

Additions

• adams-core:

  • Added the FileComplete boolean condition to allow checks whether a file is complete.

  • The BytesComplete boolean condition applies the file complete check class hierarchy to byte arrays.

  • The DeleteMapValue transformer removes a key-value pair from a Java Map object.

  • The DeletePropertyValue transformer removes a property from a Java Properties object.

  • The Flow editor now has a Recent sub-menu in the Edit menu for recently used actor processors.

  • The SimpleLogging sink allows the formatted/timestamped output of objects received.

  • The FileIncomplete file use plugin makes all the file complete checks available.

  • Added unseeded number generators for doubles and integers: JavaRandomDoubleUnseeded, JavaRandomIntUnseeded.

  • The TextRenderer conversion is the equivalent of the TextRenderer transformer.

  • The ArrayToVariables transformer maps the elements of the incoming array to the specified variables. Useful when mapping fixed-length arrays such as coordinates onto variables.

  • The CurrentTime source outputs the current time as either milli-seconds or nano-seconds.

  • The ObjectPreview sink makes use of the Preview browser's handlers that can handle objects as well, not just files.

  • Added the JavaRandomByte/Unseeded random number generators for generating a sequence of bytes.

• adams-imaging:

  • Added the JPEG and PNG file complete plugins.

  • Added the ByteArrayToImageContainer and ImageContainerToByteArray conversions, which use the specified image reader/writer to manage the byte array.

  • Added the ObjectAnnotationsReportHandler for displaying the annoations over an empty image (useful for just visualizing annotation files).

• adams-json:

  • Added the NewJsonStructure source for creating empty JSON objects or arrays.

  • Added the DeleteJsonValue transformer for removing key/value pairs from JSON objects.

  • Added the JSON file complete check plugin.

• adams-net:

  • Added the Html4Display sink for displaying HTML 4 code (supports custom CSS rules).

• adams-spectral-2dim-core:

  • With the FilteredSpetrumReader and FilteredSpectrumWriter it is now possible to apply a spectrum filter within a reader/writer. Useful when working outside flows, like in the Spectrum Explorer (loading/saving spectra).

• adams-weka and adams-weka-lts:

  • The SimpleDetrend filter works like the Detrend one but does not require the attributes to contain the wave number. Instead it just uses the positional index within the set of specified attributes.

• adams-xml: Added the XML file complete check plugin.