This time round, there have been a bunch of important bug-fixes and additions. The Weka Investigator can now perform repeated cross-validation, calculating statistics across the runs (the Experimenter does this per fold, the Investigator per full CV) and also a new tab for executing an experiment run on the current dataset and the chosen classifier.

Last, but not least, ADAMS turned 15 today (first commit in 2009). :-)

Fixes

• adams-pdf: PDF fonts now properly parse the attributes in the font string (Arrays.binarySearch on unsorted array returned invalid index for Italic).

• adams-weka and adams-weka-lts:

  • The Multi-Experimenter now sorts the results according to dataset/classifier order (dataset first).

  • Leave-one-out cross-validation (LOOCV) now works correctly in the Multi-Experimenter.

  • The Rename... action for renaming a column of an InstancesTable now uses the correct column index.

  • Changed the R^2 calculation to https://en.wikipedia.org/wiki/Coefficient_of_determination#Definitions

Changes

• adams-core:

  • spreadsheet formulas can now use median as well

  • upgraded commons-io to 2.16.1

• adams-imaging:

  • upgraded opex4j to 0.0.3

  • the ObjectCentersFromReport plugin in the Preview browser now allows changing the report reader

  • object finders now have a quiet flag to suppress warnings about -1 indices

• adams-spreadsheet:

  • The SpreadSheetAggregate transformer now offers the 25th and 75th quartile as well.

• adams-weka and adams-weka-lts:

  • upgraded common-csv-weka-package to 2024.7.5

Additions

• adams-imaging:

  • The image object filter ObjectsInPolygon finds annotations that fall within the specified polygon.

  • The CountObjectsInPolygon transformer outputs the number of annotations that fall within the specified polygon.

• adams-weka and adams-weka-lts:

  • Added support for repeated cross-validation to the Weka Investigator. See this video: https://youtu.be/ZlbynLCq8mc?si=HEkY0I_rQyaq6mD0

  • Added an Experiment tab to the Weka Investigator for evaluating single classifier on selected dataset. See this video: https://youtu.be/bVid9WrpE0s?si=KJcyUT33RbGiVF89

A lot of library upgrades happened again. Most notably are the introduction of quick actions in the Flow editor that can be invoked via SHIFT+right-click or SHIFT+F10. These are actor-specific actions like changing the variable name/value of a SetVariable actor or editing the boolean condition of an actor. These quick actions avoid having to go through the full-blown GenericObjectEditor dialog, saving a few mouse clicks. The ArrayProcessor received experimental parallel execution support. Items added to queues can have a delay in msec added to them, making them available at the other only once the delay has passed.

Fixes

• adams-core:

  • The Percentage generator for the ArraySubsetGenerator transformer now works as expected; added ability to oversample as well.

  • The SQL.tableExists method now works correctly with the newer MySQL JDBC driver (limiting DB meta-data to a particular database does not seem to work anymore, as it iterates over ALL databases).

  • Fixed the exception that occurred when selecting "Use default" from the popup menu in the GOE for an array property.

• adams-docker: The management interfaces for Docker containers and images now gather the output from all commands rather than just the last one.

• adams-imaging: The barcode encoders now use the correct x/y position (was off by 1).

• adams-ml: The ConfusionMatrix transformer, when generating sums to calculate percentages, no longer throws an error when probabilities instead of counts are taken into account (ie a probability column was specified).

Changes

• adams-core:

  • removed JVisualVM menu item from Help menu (user-mode: developer/debug) as it is no longer bundled with JDKs

  • added "ADAMS home directory" menu item (user-mode: developer/debug)

  • upgraded sizeofag to 1.1.0

  • The HashSet source now allows customizing the column name of the generated spreadsheet

  • upgraded commons-compress to 1.26.0

  • upgraded commons-io to 2.15.1

  • upgraded commons-codec to 1.16.1

  • upgraded processoutput4j to 0.0.12

  • GetReportValue can apply a conversion to the obtained report value now before forwarding it

  • The Flow editor now offers actor-specific quick actions via the SHIFT+right-click popup menu on a selected actor, e.g., renaming a variable name or changing the variable value. Using SHIFT+F10. this menu can be displayed directly via keyboard.

  • Reworked the notes support for files in the Preview browser, associating notes now with directory and file, allowing the user to edit the notes.

  • The Preview browser now has a Send to sub-menu in the popup menu for the local files as well, not just from the main menu.

  • CallableX actors no longer call the wrapUp()/cleanUp() methods of their referenced actor, as this should happen already.

  • Added experimental parallel execution support to the ArrayProcess control actor.

  • Added support for delayed queues, i.e., possible retrieval of queue items can be delayed by a certain delta of msecs. The retrieval delay gets defined in the EnQueue sink.

• adams-compress: added commons-codec depdendency

• adams-db:

  • upgraded hdsqldb to 2.7.2

  • upgraded sqlite-jdbc to 3.45.0.0

  • upgraded mssql-jdbc to 12.4.2.jre11

• adams-docker: The management interfaces for Docker containers and images allow pruning now as well.

• adams-imaging:

  • The YoloAnnotationsReportReader now has the -use-polygon-format option to enforce reading the data in polygon format rather than auto-detecting it.

  • The image segmentation panel now has the following buttons for controlling the visibility of layers: All, None, Invert

  • The object annotations panel now has a button for toggling the visibility of the annotations.

• adams-json: upgraded json-path to 2.9.0

• adams-maps: upgraded postgis-jdbc to 2023.1.0

• adams-nlp:

  • upgraded jsoup to 1.17.2

  • The EditDistance transformer can process string arrays of length 2 now as well rather than having to set the base string via a variable.

• adams-opencv: upgraded javacv to 1.5.10

• adams-rats-core: The EnQueue rat output now supports delayed queues with its retrieval delay option.

• adams-spectral-2dim-core:

  • GetSampleDataValue can apply a conversion to the obtained report value now before forwarding it

  • The SpectrumDisplay sink can accept arrays of spectra now, to speed up display of data

• adams-weka and adams-weka-lts: The attribute summary panel used in the Preprocess tab of the

  Investigator (displaying histogram/bar chart) now allows the user to customize the colors. The default color provider can be customized via the Preprocess.AttributeSummaryColorProvider property in the WekaInvestigatorSession.props config file or the Preferences dialog.

• adams-weka-lts: upgraded Weka to 3.9.0-fork-0.0.14 (updated package manager classes)

Additions

• adams-core:

  • added the SimplePercentage generator for the ArraySubsetGeneration transformer.

  • added the Copy file to... action to the files list in the Preview browser.

• adams-imaging: added the EncodeBarcode transformer that turns a string payload into the specified barcode, e.g., QR code.

• adams-weka and adams-weka-lts:

  • The RemoveWithLabels filter allows removing instances with labels that match a regular expression.

  • The AlignDataset filter aligns the dataset being filtered with an external dataset, e.g., making train/test sets compatible.

  • In the GenericObjectEditor, it is now possible to enclose clusterers in SingleClustererEnhancers, just like with classifiers. In addition, the opposite operation "pull up" has been added as well.

Since 2012, ADAMS has hosted its release files on Sourceforge.net.

This morning, I had an email from them in my inbox, stating that I need to remove old release files as the project is way over its 5GB file quota. To be honest, I was not aware that there was a quota and I also did not see any news that they will be enforcing such a quota.

Either way, I have removed old release files and the ADAMS website is now hosting the old releases (i.e., downloads may be slower). To further conserve space, the Debian (.deb) and Redhat (.rpm) release packages have been retired as well. But you can still download current .deb/.rpm packages via the daily snapshots.

Especially in corporate or institutional environments, users typically don't have any permissions to install software. Linux users are typically familiar with installing software in their own home directory, but Windows users are often not aware that this is even possible.

ADAMS itself does not rely on an installer, all you have to do is download a ZIP file (either release or snapshot), decompress the archive and then you are good to go - as long as you have Java installed on your system.

But what if you shouldn't have Java already on your computer? No problem, you don't need admin rights in order to get Java on your computer. You can simply download a JDK, decompress the archive and set the JAVA_HOME environment variable to that directory. ADAMS automatically looks for the JAVA_HOME environment variable.

JDKs can be downloaded from here: https://adoptium.net/temurin/releases/

E.g., Java 11 for Windows 64bit (download the zip): https://adoptium.net/temurin/releases/?variant=openjdk11&os=windows&arch=x64&package=jdk

Or Java 11 for Linux 64bit (download the tar.gz): https://adoptium.net/temurin/releases/?variant=openjdk11&os=linux&arch=x64&package=jdk

Later LTS (long term support) versions like 17 or 21 should work as well, but you only need 11 as a minimum at the time of writing.

Setting environment variables under Windows:

• Windows 10: https://www.alphr.com/environment-variables-windows-10/

• Windows 11: https://www.alphr.com/set-environment-variables-windows-11/

If you are running Linux, I presume you know how to manage environment variables via the export command in your $HOME/.bashrc. config file :-)

More detailed instructions:

• Download the JDK archive from the above URL

• Decompress the downloaded archive and move the decompress directory into your home directory. At the time of writing, you would find a directory called jdk-11.0.21+9 after decompressing the archive, that is the directory to move. E.g., on a Windows machine, I would place simply in my Documents directory. On Linux, I could place it straight in my home directory.

• Copy the location of this directory onto the clipboard (e.g., for me that would be C:UsersfracpeteDocumentsjdk-11.0.21+9)

• Create the JAVA_HOME environment variable and set its value to this directory

• Download the ZIP file for the ADAMS version that you want to use (e.g., a shapshot: https://adams.cms.waikato.ac.nz/download/snapshot/)

• Decompress the ADAMS zip file and move the directory in there into your home directory. As a Windows user, this could be your Documents directory again.

That's it for the setup. You can now start ADAMS via the start_gui.bat (Windows) or start_gui.sh (Linux) script, which is located in the bin directory of your ADAMS installation.

After the release a couple of weeks ago, I spent a considerable amount of time upgrading as many dependencies as possible to address known security vulnerabilities.

Another major change is the switch to version 8.x of the MySQL JDBC driver by default (you may need to add ?sessionVariables=sql_mode='ALLOW_INVALID_DATES' to your connection string).

Last, but not least, I've upgraded a range of actors to (optionally) make use of the new JobRunnerInstance standalone actor which can be used for better managing resources (cores/memory) when running heavily parallelized flows that, e.g., train classifiers or filter data. Going through this bottleneck, it is easier to avoid starving the system of resources by limiting the number of jobs that can run in parallel.

Fixes

• adams-core:

  • The BaseColorTextField now updates the button's color correctly under the cross-platform and FlatLaf look'n'feels.

• adams-imaging:

  • The EvaluatePrediction image segmentation operation now skips the percentage calculation when the total number of annotated pixels for a layer is 0.

Changes

• general

  • upgraded postgresql to 42.7.1

  • upgraded xercesimpl to 2.12.2

  • upgraded requests4j to 0.2.5

• adams-access: upgraded jackcess to 4.0.5

• adams-bootstrapp: upgraded bootstrapp to 0.1.13

• adams-compress:

  • upgraded compress-lzf to 1.1.2

  • upgraded zip4j to 2.11.5

  • upgraded junrar to 7.5.5

  • upgraded zstd-jni to 1.5.5-11

• adams-core:

  • The ExecutionLog flow execution listener now has the ability to specify what stages to log. Also outputs to stdout now if the log file is pointing to directory.

  • The new JobRunnerSupporter interface is for actors that can execute the resource-intense workload via a JobRunnerInstance standalone within their scope. Requires a JobRunnerSetup standalone further up in the flow, defining how to execute the jobs.

  • switched MySQL JDBC driver from 5.1.49 to 8.0.33

  • upgraded FlatLaf look'n'feel to 3.3

• adams-groovy: upgraded groovy-all to 3.0.20

• adams-html:

  • upgraded jetty-server/-http to 9.4.51.v20230217

  • upgraded nekohtml to 1.9.22

• adams-imaging:

  • using commons-codec 1.16.0 (and excluding dependency from lire)

  • upgraded zxing to 3.5.1

  • upgraded jts-core to 1.19.0

• adams-imaging-boofcv:

  • upgraded boofcv to 1.1.2

  • removed the HistogramBasedIntensityAdjustmentGray8 transformer

• adams-json:

  • upgraded json-path to 2.8.0

  • upgraded json-smart to 2.5

• adams-math: jsci now uses xercesimpl 2.12.2 (globally defined)

• adams-matlab: upgraded mfl-core to 0.5.15

• adams-net:

  • upgraded jsch to 0.2.16

  • upgraded tika-core to 1.28.5

• adams-pdf:

  • upgraded pdfbox to 2.0.30

  • upgraded icepdf-core/-viewer to 7.1.3

• adams-pyro4: upgraded pyrolite to 4.30

• adams-redis: upgraded lettuce-core to 6.3.1.RELEASE

• adams-rest: jaxws-rt now excludes woodstox-core dependency, instead of using 6.5.1 directly

• adams-spectral-2dim-core:

  • added sortOnSampleID as option to the spectrum conditions.

  • The following actors now implement the JobRunnerSupporter interface: Cleaner, Evaluator.

• adams-spreadsheet:

  • upgraded jfreechart to 1.5.4, added jcommon 1.0.24

  • The SpreadSheetFilter transformer now implements JobRunnerSupporter.

• adams-timeseries: excluding commons-codec from timeseriesForecasting dependency, using 1.16.0 instead

• adams-video: upgraded screencast4j to 0.1.5

• adams-webservice:

  • upgraded cxf to 3.6.2

  • upgraded xjc to 3.3.2

• adams-weka and adams-weka-lts: The following actors now implement the JobRunnerSupporter interface: WekaFilter, WekaTrainAssociator, WekaTrainClassifier, WekaTrainClusterer, WekaTestSetEvaluator, WekaTrainTestSetEvaluator.

• adams-yaml: upgraded snakeyaml to 2.2

Additions

• adams-core: The standalone JobRunnerInstance allows limiting resource intense jobs, like model training and filtering data, via a bottleneck within the actors' scope. That way heavily multi-threaded flows (e.g., nested Branch actors) won't starve the system of resources, such as CPU cores and memory. Actors implementing the JobRunnerSupporter interface can make use of that feature.

Retired modules

• adams-db-mysql8: removed as now superceded by 8.x dependency in adams-core module

ADAMS 24.1.0 has been released. More details and download links are available from here.

Happy New Year everyone!

We have moved from Twitter/X to Mastodon: @TheAdamsFlow@mstdn.social.

Mostly bug-fixes and addressing security vulnerabilities since the last update. Also, ADAMS is now more stable under the long-term support versions of OpenJDK 17 and 21.

Fixes

• adams-core:

  • The PolygonUtils.boundingBox method now works correctly.

  • The Display sink, when used in conjunction with the Inspect or DisplayPanelManager actors, uses the font specified by the user now.

  • The StorageValuesArray and VariablesArray source actors now properly collect errors occurring when applying the conversion, making it possible in the Flow editor to jump to the actor.

• adams-imaging:

  • The PolygonPointAnnotator now clears the points afte accepting an annotation.

  • The flow adams-imaging-annotate_objects.flow now supports the following annotation formats: ADAMS .report, OPEX .json

  • The TesseractHelper class now uses --psm instead of -psm for the page segmentation parameter when assembling the command-line arguments.

• adams-json: The GetJsonValue transformer now accepts JSON arrays again when using a proper JsonPath expression (i.e., one that starts with $).

• adams-scripts: added necessary --add-exports options to make it work under Java 17 and 21.

Changes

• adams-bootstrapp: bootstrapp upgraded to 0.1.12

• adams-core:

  • The expressions for boolean, mathematical, report math, string expressions and spreadsheet formulas now support the startswith and endswith string functions.

  • upgraded flatlaf to 3.2.5

  • upgraded jodd-core to 5.1.6

  • removed JULIANDATE/JULIANDATE_LONG DateTimeType due to newer versions of jodd-core no longer supporting it

• adams-gnuplot: the scriptlets now support quick info output

• adams-json: gson upgraded to 2.10.1

• adams-net:

  • tika-core upgraded to 1.28.3 to address CVE-2022-30973.

  • switched from javax.mail:1.5.6 to jakarta.mail:1.6.5

  • ApacheSendMail is now deprecated (broken)

  • upgraded jodd-mail to 5.1.6

• adams-opencv: javacv upgraded to 1.5.9

• adams-rsync: rsync4j upgraded to 3.2.7-5

• adams-pdf: The Headline PDF proclet now expands variables in the headline automatically.

Additions

• adams-core: added the GetListElement transformer for obtaining a specific element from a list.

• adams-imaging-ext: added the flow adams-imaging-ext_run-sam-hq.flow which allows using SAM-HQ (https://github.com/SysCV/sam-hq) via Redis for annotating images.

A lot of different changes have occurred over the last few months. There are two that stand out. First, docker related: images now get published under the waikatodatamining user; overhaul of how flows are used within docker images and updated documentation on website accordingly. Second, some interactive sources like EnterValue or the boolean condition PromptUser can now be displayed as a panel in the notification area rather than as a separate dialog. That way, dialogs will stay with the particular flow editor panel, even when starting multiple flows at the same time (which is also a new feature).

Full list of changes is below:

Fixes

• Swapping a SelectFile source with a EnterManyValues now transfers any extensions (or attached variable) as well.

• When selecting Save report from a report table with the Edit options box ticked, the writer's options dialog will only get displayed once now.

• Fixed source of memory leaks with the recent files handlers (handlers are now always set/get as shallow copies).

• AbstractTee, LoadBalancer, LocalScopeTransformer and WhileLoop now set the skip flag also in the underlying Sequence actor that manages their actors.

• The AbstractDataContainerFileWriter (e.g., the SpectrumFileWriter is derived from that) now ensures that the filename generated from the samepl ID is a valid filename.

• adams-weka and adams-weka-lts: data loaded into the Weka Investigator now adhers to the Investigator-wide undo state.

• When in headless mode, entering values in the terminal via EnterValue and EnterManyValues now makes use of provided initial values when simply pressing ENTER.

• Added support for environment variables ADAMS_USERNAME, ADAMS_USERHOME, ADAMS_USERDIR to override system properties user.name, user.home and user.dir respectively. The ADAMS_PLACEHOLDERS variables allows setting/overriding of placeholder definitions. Values in ADAMS-managed properties files can be overridden as well (check the adams-core manual for details, section Environment variables).

• adams-rats-core: The RatControl now skips ConsolePanel calls when in headless mode.

Changes

• Improved clearing of graphical displays (via: File - Clear).

• The Downloads > Docker documentation has been updated and expanded to fit the updated Docker images.

• adams-core:

  • The AbstractDataContainerFileReader transformer ancestor (e.g., for SpectrumFileReader) now supports output of the containers as an array instead of just one-by-one.

  • Added the ability to pause/resume or stop the flow from graphical actor dialogs via an optional flow control sub-menu. This is useful for flows that get launched with FlowRunner and do not have the typical Flow editor interface. Requires EXPERT user mode.

  • The Statistics menu item in the Flow editor now displays the results in the side panel rather than in a separate dialog.

  • The Flow Editor now has the additional Run all and Stop all menu items/toolbar buttons.

  • The Active menu and toolbar buttons in the Flow Editor are now hidden if no flow set as active.

  • The FlowRunner how has the -force-exit flag to properly exit the process in case 3rd-party library threads keep the application from exiting.

  • The PromptUser boolean flow condition can run in headless mode now as well and also be displayed in the notification area rather than as a dialog.

  • The EnterValue, EnterManyValues, SelectDirectory and SelectFile sources can be displayed in the notification area as well, not just as dialog.

  • added interfaces StringReportReader/Writer to allow them being used by the ReportToString/StringToReport conversions to perform conversions on the fly rather than via temp files.

  • Added MSECS_LONG, SECONDS_LONG, JULIANDATE_LONG and SERIAL_DATETIME_LONG as date time types, affecting conversions ConvertDateTimeType, DateTimeTypeToString, ExtractDateTimeField, StringToDateTimeType and transformer DateTimeTypeDifference.

  • The MathExpression transformer now accepts Byte/Short/Float/Number as well

• adams-db: upgraded sqlite-jdbc to 3.41.2.2.

• adams-imaging:

  • The ImageInfo transformer can output the image type and pixel depth now as well.

  • The Regions sub-images generator now stores the region coordinates in the report of the sub-image as well ("Coordinates[S]").

  • Object finders now have the resetIndicesIfNecessary flag that automatically resets the indices when there are missing or duplicate ones.

• adams-moa: upgraded MOA to 2023.04.0.

• adams-net:

  • The DownloadContent transformer and DownloadFile sink now stop active downloads when the flow is being stopped.

  • Upgraded jsch dependency to 0.2.9

  • The SSHConnection standalone now allows specifying a timeout (in milliseconds).

• adams-rsync: Upgraded rsync4j to 3.2.7-2

• adams-weka and adams-weka-lts:

  • When specifying an ID attribute with the WekaInstanceViewer sink then this attribute gets automatically excluded from the plot (avoids distorting the graph).

  • Weka Investigator: memory clean-up improvements.

  • Upgraded Weka versions: weka==3.9.6-fork-0.0.2, weka-lts==3.9.0-fork-0.0.13

• Docker images are now available from: https://hub.docker.com/u/waikatodatamining

Additions

• adams-core:

  • Added the FileComplete boolean condition to allow checks whether a file is complete.

  • The BytesComplete boolean condition applies the file complete check class hierarchy to byte arrays.

  • The DeleteMapValue transformer removes a key-value pair from a Java Map object.

  • The DeletePropertyValue transformer removes a property from a Java Properties object.

  • The Flow editor now has a Recent sub-menu in the Edit menu for recently used actor processors.

  • The SimpleLogging sink allows the formatted/timestamped output of objects received.

  • The FileIncomplete file use plugin makes all the file complete checks available.

  • Added unseeded number generators for doubles and integers: JavaRandomDoubleUnseeded, JavaRandomIntUnseeded.

  • The TextRenderer conversion is the equivalent of the TextRenderer transformer.

  • The ArrayToVariables transformer maps the elements of the incoming array to the specified variables. Useful when mapping fixed-length arrays such as coordinates onto variables.

  • The CurrentTime source outputs the current time as either milli-seconds or nano-seconds.

  • The ObjectPreview sink makes use of the Preview browser's handlers that can handle objects as well, not just files.

  • Added the JavaRandomByte/Unseeded random number generators for generating a sequence of bytes.

• adams-imaging:

  • Added the JPEG and PNG file complete plugins.

  • Added the ByteArrayToImageContainer and ImageContainerToByteArray conversions, which use the specified image reader/writer to manage the byte array.

  • Added the ObjectAnnotationsReportHandler for displaying the annoations over an empty image (useful for just visualizing annotation files).

• adams-json:

  • Added the NewJsonStructure source for creating empty JSON objects or arrays.

  • Added the DeleteJsonValue transformer for removing key/value pairs from JSON objects.

  • Added the JSON file complete check plugin.

• adams-net:

  • Added the Html4Display sink for displaying HTML 4 code (supports custom CSS rules).

• adams-spectral-2dim-core:

  • With the FilteredSpetrumReader and FilteredSpectrumWriter it is now possible to apply a spectrum filter within a reader/writer. Useful when working outside flows, like in the Spectrum Explorer (loading/saving spectra).

• adams-weka and adams-weka-lts:

  • The SimpleDetrend filter works like the Detrend one but does not require the attributes to contain the wave number. Instead it just uses the positional index within the set of specified attributes.

• adams-xml: Added the XML file complete check plugin.

Never seem to get around posting all the changes that are happening... Most notably, there is now a default new theme in ADAMS using the FlatLAF look and feel, a modern theme for Java Swing. A lot of work went into the imaging modules again, now with more flexibility in terms of how to annotate (polygons: tracing or clicking) and support for custom tools. The new adams-imaging-ext module incorporates deep-learning based tools that aid the user in annotating images (eg using pretrained models or DEXTR).

Fixes

• added support for anti-aliased text in GOE editors and text overlays for improved readability.

• adams-core:

  • updated simple-directory-chooser to 0.0.3 to fix problem with clicking on bookmarks not properly open directory tree.

  • The notification area in the Flow editor no longer splits the error output on ":" but on the top-level actor name, to avoid incorrect splits when colons are part of the actor name.

  • The storage panel of the Breakpoint control panel now updates correctly when the data changes.

  • The Go to menu item in the tree popup menu of the Flow editor now works as expected when there are more than two actor handlers present (going up in the tree).

  • In the Flow editor, a copied single actor (with potential subtree) can now be pasted in the GOE again (the internal representation gets converted to a commandline on demand, which the GOE can parse).

  • The SetTextRenderer for turning Java Set-like data structures into text now starts the numbering at 1 rather than at 2.

  • Validating a flow in the Flow editor now captures more variables.

  • Fixed several NullPointerExceptions relating to property change notifications when using the FlatLAF look and feel.

  • The Parameters tab in the Flow editor no longer shows options that are of type Actor.

  • Added a renderer for Set objects, e.g., used when inspecting storage objects.

• adams-meta: The getActors(Class) method of the AbstractStandaloneGroup class now returns all actors of that class, not just the first one.

• adams-rats-core: The stopExecution method of the Cron input Rat is now synchronized to avoid race conditions in stopping the scheduler.

• adams-redis: The RedisConnection standalone now uses the specified host/port rather than just the default ones.

Changes

• adams-addons-all: added adams-docker as dependency

• adams-annotator: added adams-docker as dependency

• adams-compress: upgraded zip4j to 2.11.3

• adams-core:

  • The ImageHandler in the Preview browser now allows the image reader to be selected by the user.

  • The Preview browser now accepts setups from other content handlers in its options dialog when pasting a setup. It also has menu items for adjusting the window to half (of the screen) width/height.

  • The object inspection tree in the breakpoint view of the Flow editor now uses lazy expansion to speed up display of large (and deeply nested) objects.

  • The GenericObjectEditor now displays log messages in the user interface as well, e.g., when pasting of a setup via the "∨" button (top right corner) goes wrong.

  • upgraded debian-maven-plugin to 1.0.22

  • The ProgressBar sink now has the ability for displaying a button that stops flow.

  • DirectoryValueDefinition and FileValueDefinition now can output absolute paths via the useAbsolutePath option.

  • The Flow editor now has menu items in the window menu for half (of the screen) width/height

  • What options are being displayed in the GenericObjectEditor depends on the current user-mode (Program -> User mode) and what minimum user mode has been defined for an option. E.g., the logging level is only displayed in EXPERT mode now. This allows hiding of options that are not relevant for most users.

  • The DisplayPanelManager sink now allows specifying the location of the divider between entry names and the displayed entry.

  • Upgraded to FlatLAF 3.0 and tweaked JTree parameters to improve appearance.

  • using FlatLAFLight as default look'n'feel now.

  • Submenus in the popup menu of the Flow editor actor tree no longer require a custom class, instead a different definition in the FlowEditor.props file can be used (property: Tree.PopupMenu).

• adams-imaging:

  • The Preview browser handlers ObjectAnnotationsHandler, ObjectCentersFromReport, ObjectLocationsFromReport and ObjectLocationsFromSpreadSheet now allow the user to specify the image reader to use; also use a table rather than a list for displaying the located objects and now offer a search panel; the object panel can be detached into a separate frame as well.

  • The ObjectLocationsHandler in the Preview browser now supports a panel with the object locations.

  • The YoloAnnotationsReportReader and YoloAnnotationsReportWriter now support annotations in polygon format as well (index plus normalized x/y polygon coordinates).

  • The Draw transformer now accepts plain BufferedImage objects as well.

  • The Image plugin for the Draw transformer now allows specifying an alpha value for transparency as well.

  • The OpexObjectLocationsReader/Writer now support per object meta-data as well.

  • Added support for tools in the ImageObjectAnnotator transformer interface. The Annotator tool lets you switch how you annotate objects (eg from bbox to polygon) while annotating. With Edit existing annotations can be modified (resizing bboxes, moving/adding vertices) and with Move annotations can be moved (dragged with mouse or moved via arrow keys).

• adams-json: The JsonObjectToMap conversion can turn JSON arrays into lists now as well (via option).

• adams-matlab: The Mat5StructToMap now handles structs with more elements than 1x1 correctly and allows the user to specify the index of a specific element to retrieve.

• adams-ml-app: added adams-docker as dependency

• adams-moa: excluding slf4j-log4j12 from moa dependency now, to avoid duplicate logger class.

• adams-nlp: excluding the outdated commons-lang3 dependency from com.kennycason:kumo-core and using version 3.7 instead explicitly.

• adams-redis: added support for authentication and experimental SSL parameters (useSSL, StartTLS, SSLVerifyMode) to the RedisConnection standalone.

• adams-rsync: upgraded rsync4j to 3.2.7-1.

• adams-spectral-3way-core: excluding log4j-slf4j-impl transitive dependency from multiway-algorithms now to avoid duplicate logger class.

• adams-spreadsheet: The Spreadsheet viewer now has a Window menu, offering to open a new viewer window or adjusting the current one to half width/height.

• adams-weka and adams-weka-lts:

  • The operations Merge, Randomize and RandomSubset that are available in the table listing the datasets now keep their last settings.

  • The Investigator now has a Window menu, offering to open a new Investigator window or adjusting the current one to half (of the screen) width/height.

• adams-yaml: upgraded snakeyaml to 1.33

• adams-xml: upgraded xstream to 1.4.20 to address CVE-2022-40151 and CVE-2022-41966.

Additions

• adams-audio: The SpeechPlaybackHandler allows playback of WAV/MP3 files and displays the content of the associated .txt file.

• adams-core:

  • The ObjectViewer sink allows the user to inspect any object with ADAMS' built-in inspection capabilities.

  • The Flow editor now has the Parse error item in the Run menu which allows to the user to paste an error output (which contains actor names) from the console. This error then gets parsed and the user can navigate to the actors (if possible).

  • The LastLineMatches file use scheme simply checks whether the last line in a text file matches the specified regular expression (useful under Linux).

  • SelectDirectory/SelectFile/EnterValue actor can be "swapped" in the Flow editor with a EnterManyValues instance now.

• adams-docker: basic support for executing docker commands. Linux/Mac only or WSL2, as it relies on the docker command-line client for execution.

• adams-imaging:

  • The ObjectAnnotationsHandler now has an OverlapRemoval cleaner, e.g., for only keeping predictions with the highest score.

  • The ImageClassificationHandler overlays label (and optional score) onto an image that was read with the specified report reader.

  • The ImageClassificationSpreadSheetReportReader loads spreadsheets with image classification predictions and stores the highest label and its score in the report.

  • The SimpleOverlay image segmentation container operation turns base image and layers into a combined image that can be used in reports or saved to disk; CompareAnnotationWithPrediction generates images from annotation vs prediction, highlighting the overlappoing, missed and additional pixels.

  • The ImageSegmentationViewer allows the viewing of ImageSegmentationContainer tokens, making it easy to toggle layers for better inspection.

  • The AnyImageToByteArray and BufferedImageToByteArray conversions turn an image into a byte array of a certain format (e.g., JPG or PNG).

  • The ByteArrayToBufferedImage generates an image container from the incoming byte array representing an image (e.g., JPG or PNG).

  • The PolygonPointAnnotator allows the user to annotate polygons by left-clicking on each vertex (ENTER accepts, ESCAPE discards).

  • The PolygonFill tool allows you to define a polygon in the ImageSegmentationAnnotator transformer that then gets filled with either foreground or background color. The polygon gets defined by left-clicking on the individual vertices.

  • Added meta-plotters for annotation outlines, labels and shapes that apply the sub-plotters subsequently.

  • Added vertex outline plotters for object annotations (bbox and polygon).

• adams-imaging-ext:

  • A new module with imaging tools that rely on external tools, like deep learning frameworks that run in docker containers.

  • DEXTR (Deep Extreme Cut) can aid the human annotator in annotating shapes for image segmentation (via transformer ImageSegmentationAnnotator).

  • OPEX can obtain object detection predictions from an OPEX-emitting docker container (via transformer ImageObjectAnnotator).

• adams-json:

  • With ImageClassificationJsonReportReader it is possible to read JSON files that contain label/score pairs generated as predictions by image classification frameworks.

  • The JsonArrayToList conversion turns a JSON array into a Java list.

  • The ListToJsonArray conversion turns a Java list into a JSON array.

• adams-matlab:

  • The DoubleMatrixToMat5Array conversion turns a double matrix into a 2-dimensional Matlab array.

  • The MapToMat5Struct conversion turns a Java map (nested maps, spreadsheets, Matlab arrays) into a Matlab struct object.

  • The Mat5ArrayToString conversion generates a human-readable string from a Matlab matrix.

  • The StringToMat5Array conversion parses a string like [1, 2; 3, 4] into a Matlab matrix.

  • The MultiMatrixOperation transformer applies a matrix operation to the matrices passing through and outputs the results.

  • The Mat5CharToString conversion turns a Char object into a string.

• adams-spreadsheet: The AutoWidthTabularSpreadSheetReader/Writer pair allows parsing/outputting of tabular text files with dynamic column widths. For reading, the column width is being determined based on the header row. For writing, the column width is adjusted to fit all content in the output.

• adams-weka and adams-weka-lts:

  • added the Remove test set action to the Investigator, which uses the RemoteTestInstances filter under the hood for removing instances of a test set from a source dataset (using a unique ID).

  • added the Soerensen-Dice metric for nominal classes (https://en.wikipedia.org/wiki/S%C3%B8rensen%E2%80%93Dice_coefficient).

This time round we have a bigger update coming: further modules retired, added experimental module for initial OpenCV image processing support, the Matlab module got extended massively (now can create structs/matrices and populate them, not just read them), a number of dependencies were upgraded and, works now properly under Java 17 and, last but not least, simplified switching of look'n'feel from the main menu (and added support for modern FlatLaf look'n'feel).

Fixes

• adams-core:

  • The documentation of the DeleteFile standalone now states that it only deletes files.

  • The storage tab in the control panel in the Flow editor when reading a breakpoint now keeps the last select item in the table. Therefore it updates that item whenever the breakpoint is reached again.

• adams-matlab:

  • The transformer Mat5FileInfo now sorts the output when enabled.

  • The transformer Mat5StructInfo now queries the input token for the correct input type and sorting works now as expected.

• adams-spreadsheet: The SpreadSheetDisplay sink can be accessed via CallableActorScreenshot when inside a DisplayPanelManager now as well.

Changes

• adams-core:

  • Removed the following actors (which had been marked as deprecated for a long time):

    • ConditionalStandalone

    • ConditionalSource

    • ConditionalTransformer

    • ConditionalSink

  • The package adams.flow.condition.test, which was only used by these actors, has been removed as well.

  • The SelectArraySubset transformer now has an initialSelection range parameter with which you can select which items get selected to start with and an initialSearch parameter (search must be allowed) to immediately show only a subset.

  • Now uses the simple-directtory-chooser library (https://github.com/waikato-datamining/simple-directory-chooser) for providing a way of selecting directories in the user interface.

  • The GenericObjectEditor used by the Flow editor and in other locations are now remembering size and location.

  • Simplified switching of Look'n'Feel (main menu: Program > Look'n'feel) and added support for FlatLaf (https://www.formdev.com/flatlaf/).

  • The HasSize boolean condition now allows the specification of an upper bound as well.

  • The control panel, when reaching a breakpoint, now allows the user to specify a rendering limit (token and storage; underlying renderer must support that), to adjust how much data is being displayed.

  • The SelectDirectory source actor now allows the selection of multiple directories.

  • The MapToString conversion now outputs array values in a readable format.

• adams-db: upgraded hsqldb to 2.7.1 to address CVE-2022-41853

• adams-heatmap: removed deprecated filter HeatmapCropToCentroid (replaced by HeatmapCrop/CropToCentroid)

• adams-json: upgraded gson to 2.8.9 to address CVE-2022-25647

• adams-matlab:

  • upgraded mfl-core to 0.5.11

  • renamed MatlabStructInfo to Mat5StructInfo for consistency

• adams-meka: upgraded meka to 1.9.7 (to work with Java 17)

• adams-moa: upgraded MOA to 2022.10.17 (ADAMS-internal release of fork; minus deep learning classifiers/libraries)

• adams-pdf:

  • The page ranges of the PDFExtract and PDFRenderPages transformers are now unordered (i.e., indices will not get sorted).

  • Replaced the outdated Intarsys libraries with modern IcePDF (https://github.com/pcorless/icepdf).

• adams-rest: upgraded jackson-databind to 2.14.0-rc1 to address CVE-2022-42003

• adams-spectral-3way-core: upgraded multiway-algorithms to 0.0.13 and lombok to 1.18.24 to work with Java 17

• adams-spreadsheet:

  • The SpreadSheetRowBuffer transformer now allows the user to specify a monitor variable for resetting the buffer (row -> sheet).

  • The HasRows boolean condition now allows the specification of an upper bound as well.

• adams-weka and adams-weka-lts:

  • The WekaInstanceBuffer transformer now allows the user to specify a monitor variable for resetting the buffer (instance -> instances).

  • WekaPredictionsToSpreadSheet and WekaPredictionsToInstances now have useOriginalIndices on by default to avoid accidental misalignment when adding attributes from the test data.

Additions

• adams-core:

  • The EnterManyValues source can now query for arrays using the ArrayValueDefinition plugin.

  • The DeleteDir standalone can delete sub-directories matching a regular expression or itself.

  • The HasLength boolean condition can ensure a minimum/maximum length of an array.

  • The CollectionSize transformer outputs the size (ie number of stored elements) of any collection.

  • The IsArm64 boolean condition checks whether the system architecture is Arm 64-bit.

  • The DistinctValueCellRenderingCustomizer renderer uses different a background color for each distinct value in the spreadsheet.

  • The ColorProviderBasedColorGradientGenerator can be used for generating colors using a color provider (eg useful when generating images from matrices with unique values).

• adams-imaging: added the DoubleMatrixToBufferedImage conversion, which turns a heatmap into an image.

• adams-matlab:

  • added the Mat5ArrayInfo transformer to obtain information on the incoming array (eg dimensions).

  • added the Mat5ArraySubset transformer for obtaining arbitrary subsets (n-dimensional arrays) of arrays.

  • added the Mat5ArrayToDoubleMatrix conversion.

  • added the IsMat5Array boolean flow condition.

  • added the IsMat5Matrix boolean flow condition.

  • added the NewMat5Matrix source actor for creating an empty matrix.

  • added the Mat5GetMatrixElement transformer for retrieving a single element from a n-dimensional matrix.

  • added the Mat5SetMatrixElement transformer for setting a single element in a n-dimensional matrix.

  • added the NewMat5Struct source actor for creating an empty struct.

  • added the Mat5GetStructField transformer for retrieving an array by name from a struct.

  • added the Mat5SetStructField transformer for setting an array by name in a struct.

• adams-opencv: new module for image processing using JavaCV's wrapper around OpenCV.

Retired modules

The following modules have been retired and moved to https://github.com/waikato-datamining/adams-discontinued:

• adams-dex-app: due to retiring RabbitMQ modules

• adams-jython: Jython hasn't progressed beyond Python 2.7 for many years

• adams-rabbitmq: focus shifted, no longer required

• adams-rabbitmq-rats: due to retiring adams-rabbitmq