A lot of different changes have occurred over the last few months. There are two that stand out. First, docker related: images now get published under the waikatodatamining user; overhaul of how flows are used within docker images and updated documentation on website accordingly. Second, some interactive sources like EnterValue or the boolean condition PromptUser can now be displayed as a panel in the notification area rather than as a separate dialog. That way, dialogs will stay with the particular flow editor panel, even when starting multiple flows at the same time (which is also a new feature).

Full list of changes is below:

Fixes

• Swapping a SelectFile source with a EnterManyValues now transfers any extensions (or attached variable) as well.

• When selecting Save report from a report table with the Edit options box ticked, the writer's options dialog will only get displayed once now.

• Fixed source of memory leaks with the recent files handlers (handlers are now always set/get as shallow copies).

• AbstractTee, LoadBalancer, LocalScopeTransformer and WhileLoop now set the skip flag also in the underlying Sequence actor that manages their actors.

• The AbstractDataContainerFileWriter (e.g., the SpectrumFileWriter is derived from that) now ensures that the filename generated from the samepl ID is a valid filename.

• adams-weka and adams-weka-lts: data loaded into the Weka Investigator now adhers to the Investigator-wide undo state.

• When in headless mode, entering values in the terminal via EnterValue and EnterManyValues now makes use of provided initial values when simply pressing ENTER.

• Added support for environment variables ADAMS_USERNAME, ADAMS_USERHOME, ADAMS_USERDIR to override system properties user.name, user.home and user.dir respectively. The ADAMS_PLACEHOLDERS variables allows setting/overriding of placeholder definitions. Values in ADAMS-managed properties files can be overridden as well (check the adams-core manual for details, section Environment variables).

• adams-rats-core: The RatControl now skips ConsolePanel calls when in headless mode.

Changes

• Improved clearing of graphical displays (via: File - Clear).

• The Downloads > Docker documentation has been updated and expanded to fit the updated Docker images.

• adams-core:

  • The AbstractDataContainerFileReader transformer ancestor (e.g., for SpectrumFileReader) now supports output of the containers as an array instead of just one-by-one.

  • Added the ability to pause/resume or stop the flow from graphical actor dialogs via an optional flow control sub-menu. This is useful for flows that get launched with FlowRunner and do not have the typical Flow editor interface. Requires EXPERT user mode.

  • The Statistics menu item in the Flow editor now displays the results in the side panel rather than in a separate dialog.

  • The Flow Editor now has the additional Run all and Stop all menu items/toolbar buttons.

  • The Active menu and toolbar buttons in the Flow Editor are now hidden if no flow set as active.

  • The FlowRunner how has the -force-exit flag to properly exit the process in case 3rd-party library threads keep the application from exiting.

  • The PromptUser boolean flow condition can run in headless mode now as well and also be displayed in the notification area rather than as a dialog.

  • The EnterValue, EnterManyValues, SelectDirectory and SelectFile sources can be displayed in the notification area as well, not just as dialog.

  • added interfaces StringReportReader/Writer to allow them being used by the ReportToString/StringToReport conversions to perform conversions on the fly rather than via temp files.

  • Added MSECS_LONG, SECONDS_LONG, JULIANDATE_LONG and SERIAL_DATETIME_LONG as date time types, affecting conversions ConvertDateTimeType, DateTimeTypeToString, ExtractDateTimeField, StringToDateTimeType and transformer DateTimeTypeDifference.

  • The MathExpression transformer now accepts Byte/Short/Float/Number as well

• adams-db: upgraded sqlite-jdbc to 3.41.2.2.

• adams-imaging:

  • The ImageInfo transformer can output the image type and pixel depth now as well.

  • The Regions sub-images generator now stores the region coordinates in the report of the sub-image as well ("Coordinates[S]").

  • Object finders now have the resetIndicesIfNecessary flag that automatically resets the indices when there are missing or duplicate ones.

• adams-moa: upgraded MOA to 2023.04.0.

• adams-net:

  • The DownloadContent transformer and DownloadFile sink now stop active downloads when the flow is being stopped.

  • Upgraded jsch dependency to 0.2.9

  • The SSHConnection standalone now allows specifying a timeout (in milliseconds).

• adams-rsync: Upgraded rsync4j to 3.2.7-2

• adams-weka and adams-weka-lts:

  • When specifying an ID attribute with the WekaInstanceViewer sink then this attribute gets automatically excluded from the plot (avoids distorting the graph).

  • Weka Investigator: memory clean-up improvements.

  • Upgraded Weka versions: weka==3.9.6-fork-0.0.2, weka-lts==3.9.0-fork-0.0.13

• Docker images are now available from: https://hub.docker.com/u/waikatodatamining

Additions

• adams-core:

  • Added the FileComplete boolean condition to allow checks whether a file is complete.

  • The BytesComplete boolean condition applies the file complete check class hierarchy to byte arrays.

  • The DeleteMapValue transformer removes a key-value pair from a Java Map object.

  • The DeletePropertyValue transformer removes a property from a Java Properties object.

  • The Flow editor now has a Recent sub-menu in the Edit menu for recently used actor processors.

  • The SimpleLogging sink allows the formatted/timestamped output of objects received.

  • The FileIncomplete file use plugin makes all the file complete checks available.

  • Added unseeded number generators for doubles and integers: JavaRandomDoubleUnseeded, JavaRandomIntUnseeded.

  • The TextRenderer conversion is the equivalent of the TextRenderer transformer.

  • The ArrayToVariables transformer maps the elements of the incoming array to the specified variables. Useful when mapping fixed-length arrays such as coordinates onto variables.

  • The CurrentTime source outputs the current time as either milli-seconds or nano-seconds.

  • The ObjectPreview sink makes use of the Preview browser's handlers that can handle objects as well, not just files.

  • Added the JavaRandomByte/Unseeded random number generators for generating a sequence of bytes.

• adams-imaging:

  • Added the JPEG and PNG file complete plugins.

  • Added the ByteArrayToImageContainer and ImageContainerToByteArray conversions, which use the specified image reader/writer to manage the byte array.

  • Added the ObjectAnnotationsReportHandler for displaying the annoations over an empty image (useful for just visualizing annotation files).

• adams-json:

  • Added the NewJsonStructure source for creating empty JSON objects or arrays.

  • Added the DeleteJsonValue transformer for removing key/value pairs from JSON objects.

  • Added the JSON file complete check plugin.

• adams-net:

  • Added the Html4Display sink for displaying HTML 4 code (supports custom CSS rules).

• adams-spectral-2dim-core:

  • With the FilteredSpetrumReader and FilteredSpectrumWriter it is now possible to apply a spectrum filter within a reader/writer. Useful when working outside flows, like in the Spectrum Explorer (loading/saving spectra).

• adams-weka and adams-weka-lts:

  • The SimpleDetrend filter works like the Detrend one but does not require the attributes to contain the wave number. Instead it just uses the positional index within the set of specified attributes.

• adams-xml: Added the XML file complete check plugin.

Never seem to get around posting all the changes that are happening... Most notably, there is now a default new theme in ADAMS using the FlatLAF look and feel, a modern theme for Java Swing. A lot of work went into the imaging modules again, now with more flexibility in terms of how to annotate (polygons: tracing or clicking) and support for custom tools. The new adams-imaging-ext module incorporates deep-learning based tools that aid the user in annotating images (eg using pretrained models or DEXTR).

Fixes

• added support for anti-aliased text in GOE editors and text overlays for improved readability.

• adams-core:

  • updated simple-directory-chooser to 0.0.3 to fix problem with clicking on bookmarks not properly open directory tree.

  • The notification area in the Flow editor no longer splits the error output on ":" but on the top-level actor name, to avoid incorrect splits when colons are part of the actor name.

  • The storage panel of the Breakpoint control panel now updates correctly when the data changes.

  • The Go to menu item in the tree popup menu of the Flow editor now works as expected when there are more than two actor handlers present (going up in the tree).

  • In the Flow editor, a copied single actor (with potential subtree) can now be pasted in the GOE again (the internal representation gets converted to a commandline on demand, which the GOE can parse).

  • The SetTextRenderer for turning Java Set-like data structures into text now starts the numbering at 1 rather than at 2.

  • Validating a flow in the Flow editor now captures more variables.

  • Fixed several NullPointerExceptions relating to property change notifications when using the FlatLAF look and feel.

  • The Parameters tab in the Flow editor no longer shows options that are of type Actor.

  • Added a renderer for Set objects, e.g., used when inspecting storage objects.

• adams-meta: The getActors(Class) method of the AbstractStandaloneGroup class now returns all actors of that class, not just the first one.

• adams-rats-core: The stopExecution method of the Cron input Rat is now synchronized to avoid race conditions in stopping the scheduler.

• adams-redis: The RedisConnection standalone now uses the specified host/port rather than just the default ones.

Changes

• adams-addons-all: added adams-docker as dependency

• adams-annotator: added adams-docker as dependency

• adams-compress: upgraded zip4j to 2.11.3

• adams-core:

  • The ImageHandler in the Preview browser now allows the image reader to be selected by the user.

  • The Preview browser now accepts setups from other content handlers in its options dialog when pasting a setup. It also has menu items for adjusting the window to half (of the screen) width/height.

  • The object inspection tree in the breakpoint view of the Flow editor now uses lazy expansion to speed up display of large (and deeply nested) objects.

  • The GenericObjectEditor now displays log messages in the user interface as well, e.g., when pasting of a setup via the "∨" button (top right corner) goes wrong.

  • upgraded debian-maven-plugin to 1.0.22

  • The ProgressBar sink now has the ability for displaying a button that stops flow.

  • DirectoryValueDefinition and FileValueDefinition now can output absolute paths via the useAbsolutePath option.

  • The Flow editor now has menu items in the window menu for half (of the screen) width/height

  • What options are being displayed in the GenericObjectEditor depends on the current user-mode (Program -> User mode) and what minimum user mode has been defined for an option. E.g., the logging level is only displayed in EXPERT mode now. This allows hiding of options that are not relevant for most users.

  • The DisplayPanelManager sink now allows specifying the location of the divider between entry names and the displayed entry.

  • Upgraded to FlatLAF 3.0 and tweaked JTree parameters to improve appearance.

  • using FlatLAFLight as default look'n'feel now.

  • Submenus in the popup menu of the Flow editor actor tree no longer require a custom class, instead a different definition in the FlowEditor.props file can be used (property: Tree.PopupMenu).

• adams-imaging:

  • The Preview browser handlers ObjectAnnotationsHandler, ObjectCentersFromReport, ObjectLocationsFromReport and ObjectLocationsFromSpreadSheet now allow the user to specify the image reader to use; also use a table rather than a list for displaying the located objects and now offer a search panel; the object panel can be detached into a separate frame as well.

  • The ObjectLocationsHandler in the Preview browser now supports a panel with the object locations.

  • The YoloAnnotationsReportReader and YoloAnnotationsReportWriter now support annotations in polygon format as well (index plus normalized x/y polygon coordinates).

  • The Draw transformer now accepts plain BufferedImage objects as well.

  • The Image plugin for the Draw transformer now allows specifying an alpha value for transparency as well.

  • The OpexObjectLocationsReader/Writer now support per object meta-data as well.

  • Added support for tools in the ImageObjectAnnotator transformer interface. The Annotator tool lets you switch how you annotate objects (eg from bbox to polygon) while annotating. With Edit existing annotations can be modified (resizing bboxes, moving/adding vertices) and with Move annotations can be moved (dragged with mouse or moved via arrow keys).

• adams-json: The JsonObjectToMap conversion can turn JSON arrays into lists now as well (via option).

• adams-matlab: The Mat5StructToMap now handles structs with more elements than 1x1 correctly and allows the user to specify the index of a specific element to retrieve.

• adams-ml-app: added adams-docker as dependency

• adams-moa: excluding slf4j-log4j12 from moa dependency now, to avoid duplicate logger class.

• adams-nlp: excluding the outdated commons-lang3 dependency from com.kennycason:kumo-core and using version 3.7 instead explicitly.

• adams-redis: added support for authentication and experimental SSL parameters (useSSL, StartTLS, SSLVerifyMode) to the RedisConnection standalone.

• adams-rsync: upgraded rsync4j to 3.2.7-1.

• adams-spectral-3way-core: excluding log4j-slf4j-impl transitive dependency from multiway-algorithms now to avoid duplicate logger class.

• adams-spreadsheet: The Spreadsheet viewer now has a Window menu, offering to open a new viewer window or adjusting the current one to half width/height.

• adams-weka and adams-weka-lts:

  • The operations Merge, Randomize and RandomSubset that are available in the table listing the datasets now keep their last settings.

  • The Investigator now has a Window menu, offering to open a new Investigator window or adjusting the current one to half (of the screen) width/height.

• adams-yaml: upgraded snakeyaml to 1.33

• adams-xml: upgraded xstream to 1.4.20 to address CVE-2022-40151 and CVE-2022-41966.

Additions

• adams-audio: The SpeechPlaybackHandler allows playback of WAV/MP3 files and displays the content of the associated .txt file.

• adams-core:

  • The ObjectViewer sink allows the user to inspect any object with ADAMS' built-in inspection capabilities.

  • The Flow editor now has the Parse error item in the Run menu which allows to the user to paste an error output (which contains actor names) from the console. This error then gets parsed and the user can navigate to the actors (if possible).

  • The LastLineMatches file use scheme simply checks whether the last line in a text file matches the specified regular expression (useful under Linux).

  • SelectDirectory/SelectFile/EnterValue actor can be "swapped" in the Flow editor with a EnterManyValues instance now.

• adams-docker: basic support for executing docker commands. Linux/Mac only or WSL2, as it relies on the docker command-line client for execution.

• adams-imaging:

  • The ObjectAnnotationsHandler now has an OverlapRemoval cleaner, e.g., for only keeping predictions with the highest score.

  • The ImageClassificationHandler overlays label (and optional score) onto an image that was read with the specified report reader.

  • The ImageClassificationSpreadSheetReportReader loads spreadsheets with image classification predictions and stores the highest label and its score in the report.

  • The SimpleOverlay image segmentation container operation turns base image and layers into a combined image that can be used in reports or saved to disk; CompareAnnotationWithPrediction generates images from annotation vs prediction, highlighting the overlappoing, missed and additional pixels.

  • The ImageSegmentationViewer allows the viewing of ImageSegmentationContainer tokens, making it easy to toggle layers for better inspection.

  • The AnyImageToByteArray and BufferedImageToByteArray conversions turn an image into a byte array of a certain format (e.g., JPG or PNG).

  • The ByteArrayToBufferedImage generates an image container from the incoming byte array representing an image (e.g., JPG or PNG).

  • The PolygonPointAnnotator allows the user to annotate polygons by left-clicking on each vertex (ENTER accepts, ESCAPE discards).

  • The PolygonFill tool allows you to define a polygon in the ImageSegmentationAnnotator transformer that then gets filled with either foreground or background color. The polygon gets defined by left-clicking on the individual vertices.

  • Added meta-plotters for annotation outlines, labels and shapes that apply the sub-plotters subsequently.

  • Added vertex outline plotters for object annotations (bbox and polygon).

• adams-imaging-ext:

  • A new module with imaging tools that rely on external tools, like deep learning frameworks that run in docker containers.

  • DEXTR (Deep Extreme Cut) can aid the human annotator in annotating shapes for image segmentation (via transformer ImageSegmentationAnnotator).

  • OPEX can obtain object detection predictions from an OPEX-emitting docker container (via transformer ImageObjectAnnotator).

• adams-json:

  • With ImageClassificationJsonReportReader it is possible to read JSON files that contain label/score pairs generated as predictions by image classification frameworks.

  • The JsonArrayToList conversion turns a JSON array into a Java list.

  • The ListToJsonArray conversion turns a Java list into a JSON array.

• adams-matlab:

  • The DoubleMatrixToMat5Array conversion turns a double matrix into a 2-dimensional Matlab array.

  • The MapToMat5Struct conversion turns a Java map (nested maps, spreadsheets, Matlab arrays) into a Matlab struct object.

  • The Mat5ArrayToString conversion generates a human-readable string from a Matlab matrix.

  • The StringToMat5Array conversion parses a string like [1, 2; 3, 4] into a Matlab matrix.

  • The MultiMatrixOperation transformer applies a matrix operation to the matrices passing through and outputs the results.

  • The Mat5CharToString conversion turns a Char object into a string.

• adams-spreadsheet: The AutoWidthTabularSpreadSheetReader/Writer pair allows parsing/outputting of tabular text files with dynamic column widths. For reading, the column width is being determined based on the header row. For writing, the column width is adjusted to fit all content in the output.

• adams-weka and adams-weka-lts:

  • added the Remove test set action to the Investigator, which uses the RemoteTestInstances filter under the hood for removing instances of a test set from a source dataset (using a unique ID).

  • added the Soerensen-Dice metric for nominal classes (https://en.wikipedia.org/wiki/S%C3%B8rensen%E2%80%93Dice_coefficient).

This time round we have a bigger update coming: further modules retired, added experimental module for initial OpenCV image processing support, the Matlab module got extended massively (now can create structs/matrices and populate them, not just read them), a number of dependencies were upgraded and, works now properly under Java 17 and, last but not least, simplified switching of look'n'feel from the main menu (and added support for modern FlatLaf look'n'feel).

Fixes

• adams-core:

  • The documentation of the DeleteFile standalone now states that it only deletes files.

  • The storage tab in the control panel in the Flow editor when reading a breakpoint now keeps the last select item in the table. Therefore it updates that item whenever the breakpoint is reached again.

• adams-matlab:

  • The transformer Mat5FileInfo now sorts the output when enabled.

  • The transformer Mat5StructInfo now queries the input token for the correct input type and sorting works now as expected.

• adams-spreadsheet: The SpreadSheetDisplay sink can be accessed via CallableActorScreenshot when inside a DisplayPanelManager now as well.

Changes

• adams-core:

  • Removed the following actors (which had been marked as deprecated for a long time):

    • ConditionalStandalone

    • ConditionalSource

    • ConditionalTransformer

    • ConditionalSink

  • The package adams.flow.condition.test, which was only used by these actors, has been removed as well.

  • The SelectArraySubset transformer now has an initialSelection range parameter with which you can select which items get selected to start with and an initialSearch parameter (search must be allowed) to immediately show only a subset.

  • Now uses the simple-directtory-chooser library (https://github.com/waikato-datamining/simple-directory-chooser) for providing a way of selecting directories in the user interface.

  • The GenericObjectEditor used by the Flow editor and in other locations are now remembering size and location.

  • Simplified switching of Look'n'Feel (main menu: Program > Look'n'feel) and added support for FlatLaf (https://www.formdev.com/flatlaf/).

  • The HasSize boolean condition now allows the specification of an upper bound as well.

  • The control panel, when reaching a breakpoint, now allows the user to specify a rendering limit (token and storage; underlying renderer must support that), to adjust how much data is being displayed.

  • The SelectDirectory source actor now allows the selection of multiple directories.

  • The MapToString conversion now outputs array values in a readable format.

• adams-db: upgraded hsqldb to 2.7.1 to address CVE-2022-41853

• adams-heatmap: removed deprecated filter HeatmapCropToCentroid (replaced by HeatmapCrop/CropToCentroid)

• adams-json: upgraded gson to 2.8.9 to address CVE-2022-25647

• adams-matlab:

  • upgraded mfl-core to 0.5.11

  • renamed MatlabStructInfo to Mat5StructInfo for consistency

• adams-meka: upgraded meka to 1.9.7 (to work with Java 17)

• adams-moa: upgraded MOA to 2022.10.17 (ADAMS-internal release of fork; minus deep learning classifiers/libraries)

• adams-pdf:

  • The page ranges of the PDFExtract and PDFRenderPages transformers are now unordered (i.e., indices will not get sorted).

  • Replaced the outdated Intarsys libraries with modern IcePDF (https://github.com/pcorless/icepdf).

• adams-rest: upgraded jackson-databind to 2.14.0-rc1 to address CVE-2022-42003

• adams-spectral-3way-core: upgraded multiway-algorithms to 0.0.13 and lombok to 1.18.24 to work with Java 17

• adams-spreadsheet:

  • The SpreadSheetRowBuffer transformer now allows the user to specify a monitor variable for resetting the buffer (row -> sheet).

  • The HasRows boolean condition now allows the specification of an upper bound as well.

• adams-weka and adams-weka-lts:

  • The WekaInstanceBuffer transformer now allows the user to specify a monitor variable for resetting the buffer (instance -> instances).

  • WekaPredictionsToSpreadSheet and WekaPredictionsToInstances now have useOriginalIndices on by default to avoid accidental misalignment when adding attributes from the test data.

Additions

• adams-core:

  • The EnterManyValues source can now query for arrays using the ArrayValueDefinition plugin.

  • The DeleteDir standalone can delete sub-directories matching a regular expression or itself.

  • The HasLength boolean condition can ensure a minimum/maximum length of an array.

  • The CollectionSize transformer outputs the size (ie number of stored elements) of any collection.

  • The IsArm64 boolean condition checks whether the system architecture is Arm 64-bit.

  • The DistinctValueCellRenderingCustomizer renderer uses different a background color for each distinct value in the spreadsheet.

  • The ColorProviderBasedColorGradientGenerator can be used for generating colors using a color provider (eg useful when generating images from matrices with unique values).

• adams-imaging: added the DoubleMatrixToBufferedImage conversion, which turns a heatmap into an image.

• adams-matlab:

  • added the Mat5ArrayInfo transformer to obtain information on the incoming array (eg dimensions).

  • added the Mat5ArraySubset transformer for obtaining arbitrary subsets (n-dimensional arrays) of arrays.

  • added the Mat5ArrayToDoubleMatrix conversion.

  • added the IsMat5Array boolean flow condition.

  • added the IsMat5Matrix boolean flow condition.

  • added the NewMat5Matrix source actor for creating an empty matrix.

  • added the Mat5GetMatrixElement transformer for retrieving a single element from a n-dimensional matrix.

  • added the Mat5SetMatrixElement transformer for setting a single element in a n-dimensional matrix.

  • added the NewMat5Struct source actor for creating an empty struct.

  • added the Mat5GetStructField transformer for retrieving an array by name from a struct.

  • added the Mat5SetStructField transformer for setting an array by name in a struct.

• adams-opencv: new module for image processing using JavaCV's wrapper around OpenCV.

Retired modules

The following modules have been retired and moved to https://github.com/waikato-datamining/adams-discontinued:

• adams-dex-app: due to retiring RabbitMQ modules

• adams-jython: Jython hasn't progressed beyond Python 2.7 for many years

• adams-rabbitmq: focus shifted, no longer required

• adams-rabbitmq-rats: due to retiring adams-rabbitmq

Apart from retiring a bunch of unused modules, upgrading a range of dependencies, I've switched the codebase to allow Java 11 syntax. Even though only switching to 11 for the code, the compiled binaries should work with OpenJDK 17 (the next long-term-support version of OpenJDK).

Fixes

• The MemoryMonitor standalone now stops the monitoring when the flow stops.

• The ListVariableUsage actor processor (used by Find usages in the Flow editor tree popup) now lists boolean options as well.

• EnterManyValues now restores time/date/datetime values correctly, the ListSelectionValueDefinition now can have multi-selection enabled.

• adams-imaging: The PascalVOCObjectLocationsReader report reader no longer fails with a NullPointerException if the segmented node is not present.

• adams-weka and adams-weka-lts: The WekaSpreadSheetToPredictions transformer now handles labels correctly that are only present in the predicted column.

Changes

• adams-access: upgraded jackcess to 4.0.2

• adams-compress:

  • upgraded compress-lzf to 1.1

  • upgraded zip4j to 2.11.2

  • upgraded junrar to 7.5.3

  • upgraded xz to 1.9

  • upgraded zstd-jni to 1.5.2-4

• adams-core:

  • Upgraded mysql-connector-java to 5.1.49 to address Cannot connect: java.sql.SQLException: Unknown system variable 'query_cache_size' error

  • Upgraded jclasslocator to 0.0.22 for speeding up classpath analysis

  • The CounterInit standalone and transformer now support initializing strings with a specific value

  • The Preview browser now allows fixing the preview handler, i.e., it does not get reloaded from its .props file with the next file. This is useful when using two or more preview windows with different handlers for the same file type/extension.

  • The HashSet source now allows the user to specify a conversion for converting the hashset elements into custom object types in the spreadsheet (default: string).

  • The Flow editor now records what actor handlers are being used for enclosing other actors to offer a common shortlist in the new, compact Enclose item of the tree popup menu.

  • The Make conditional action in the Flow editor now uses ConditionalTrigger/SubProcess/Tee for sources/transformers/sinks that don't have a conditional equivalent.

  • The Expression plot post-processor can handle plot containers without X values now.

• adams-db-mysql8: Upgraded mysql-connector-java to 8.0.30

• adams-excel:

  • Apache POI dependencies upgraded to 4.1.2

  • The ExcelSpreadSheetReader can use names in the sheet range now, also improved the handling of cell types (e.g., formulas and errors)

  • The ExcelSpreadSheetWriter now allows customizing the formats of time, date and date/time cells. Also handles boolean cells now.

• adams-groovy: upgraded groovy-all dependency to 3.0.8

• adams-html: upgraded jetty-server to 9.4.48.v20220622

• adams-imaging:

  • The IntersectOverUnionRatio object overlap scheme can now either use bounding boxes or polygons for the IoU calculation.

  • The JAI Resize BufferedImage transformer now has a forcePercentage option to allow upscaling images.

• adams-jython: upgraded jython-standalone to 2.7.3

• adams-matlab: The Mat5SpreadSheetReader can use names in the sheet range now

• adams-math: * The LOWESS, SavitzkyGolay and SimpleSavitzkyGolay plot post-processors can handle plot containers without X values now.

• adams-ml: The ConfusionMatrix transformer can now enumerate the labels (in case of very long labels) and outputs a second spreadsheet with the index/label relation.

• adams-net: switched to com.github.mwiede fork of jsch library

• adams-odf: The ODFSpreadSheetReader can use names in the sheet range now

• adams-pdf:

  • The SpreadSheet PDF proclet allows customizing the table width, horizontal alignment and relative column widths.

  • upgraded pdfbox to 2.0.27

• adams-python: moved Pyro4 related code into adams-pyro4 module

• adams-random: The transformers ArrayFolds and ArraySubSample now can turn off creating a deep copy of the elements before transferring them to the target array.

• adams-rats-django: updated to work with Django 4.1.x and Python 3.8+

• adams-rest: upgraded jackson-databind to 2.13.4 to address CVE-2022-42004

• adams-rsync: upgraded rsync4j-windows64 to 3.2.3-10

• adams-spreadsheet:

  • The SpreadSheetDisplay sink now allows to hide the column with the row index and use a simple header.

  • The GnumericSpreadSheetReader can use names in the sheet range now

  • The NewSpreadSheet source now can have the headers supplied via an array, which takes precedence over the comma-separated list.

• adams-webservice: upgraded Apache CXF to 3.5.3

• adams-yaml: Upgraded snakeyaml to 1.32 to address CVE-2022-25857 and CVE-2022-38752

Additions

• adams-core:

  • The ArrayDisplay sink allows displaying array elements in tabs, horizontally, vertically or in a grid.

  • Added the MapVariableIterator for iterating over key/value pairs of a map, storing key/value in user-specified variables.

  • Added the Copy (commandline) option to the Flow editor popup menu (Copy...) to allow pasting in the GenericObjectEditor.

  • Added the StringToCharArray and CharArrayToString conversions.

• adams-imaging:

  • The Regions plugin for the Subimages buffered image transformer allows the user to specify arbitrary rectangles (and their annotations) to be extracted.

  • The ImageObjectOverlapMatrix transformer generates a matrix between overlapping annotations and predictions, which can be turned into a confusion matrix.

  • Additional (non-JAI) Resize BufferedImage transformer added for resizing images.

  • The MergeOverlaps image object filter can combine multiple, overlapping objects into a single one.

  • The MergeOverlaps overlapping object removal scheme combines overlapping objects rather than removing them.

  • The DetermineOverlappingAreas transformer computes areas of overlapping objects between annotations and predictions.

• adams-pdf:

  • Added the FixedText PDF proclet that allows arbitrary text to be added (and variables get expanded automatically) without having to load a file from disk.

  • The ConfusionMatrix PDF proclet inserts a confusion matrix with better highlighting of the diagonal and (optionally) the other cell values.

• adams-pyro5: same as adams-pyro4, but uses the pyrolite library for communicating with Pyro5

• adams-spreadsheet:

  • Added the conversions SpreadSheetToRowArray and RowArrayToSpreadSheet for managing a spreadsheet as row array and apply existing array operators.

Retired modules

The following modules have been retired and moved to https://github.com/waikato-datamining/adams-discontinued:

• adams-cqengine: module retired due to no downstream projects making use of it

• adams-groovy-webservice: module retired due to com.github.groovy-wslite:groovy-wslite project being abandoned

• adams-osm: display of maps in Swing not well supported

• adams-tensorflow: module retired due to limited usefulness

• adams-deeplearning-spectral-app: module retired (no added benefit since adams-tensorflow got retired)

• adams-twitter: no real benefit of module

• adams-twitter-rats: due to adams-twitter getting retired