A bunch of changes and updates happened since last time in relation to spreadsheet handling (general and spectral data related). One thing that I've been trying to sort out was creating screenshots of tables. You can now use Send to -> Export table as image on pretty much any native ADAMS table to create a screenshot of a table.

Fixes

• The ListVariableUsage actor processor now also inspects VariableNameValuePair (and arrays) and BaseObject arrays for variables.

Changes

• The search panel received an overhaul, including an added clear button. The Preview browser now performs incremental search for a more efficient search.

• The spreadsheet handlers in the Preview browser now allow the user to select a custom cell rendering, e.g., for highlighting certain values.

• Most tables should now offer Send to capability and the new Export table as image action to export the current table as an image in various formats. With the Copy table as image, the table gets copied as image to the clipboard.

• The SpreadSheetDisplay sink can now be used in conjunction with the CallableActorScreenshot actor.

• The Preview browser supports export of tables as images now as well.

• commons-io/commons-io upgraded to 2.7

• adams-imaging:

  • The ObjectLocationsFromReport operation for the Draw transformer now allows varying the shape color as well, just like in the Preview browser.

  • Object location overlays can now fall back on the bounding box if the polygon is too small (eg from a bad mask). Also applies to the Preview browser and Draw transformer plugin.

• adams-matlab: improved handling of character cells in the MatlabArrayToSpreadSheet conversion.

• adams-net: The HttpRequest transformer now has an option for defining the mime-type of the data, default is application/octect-stream.

• adams-weka and adams-weka-lts: The WekaInstancesDisplay sink can now be used in conjunction with the CallableActorScreenshot actor now.

• adams-spreadsheet: The SpreadSheetInsertRow transformer can interpret the value now as blank or comma-separated values as well. The value(s) can now be forced to be STRING as well.

• adams-spectral-2dim-core: the JSON format for spectra has been overhauls, using separate arrays for wave numbers and amplitudes now to reduce file size. Also, either the full report or specific reference+meta-data values can be written/read.

Additions

• The HeatmapCellRenderingCustomizer cell rendering customizer for spreadsheet tables allows one to color in the background of numeric cells based on the global min/max and the specified color generator. Makes it easier to spot extreme values.

• The HasElements boolean flow condition checks whether the incoming object is an array and has at least specified number of elements.

• The HasSize boolean flow condition checks whether the incoming collection has at least the specified size.

• The ReportArrayToMap conversion turns the a report array into a map using the specified report field as key.

• The MergeReportFromMap transformer allows the reports passing through (either Report or MutableReportHandler objects) to be merged with reports available from a map in storage. This way, meta-data can be read separately and stored in a map in storage and then easily attached.

• adams-imaging:

  • Added general support for point annotations, which can be used for pose estimation.

  • Added support for reading/writing DeepLabCut CSV annotation files

• adams-spectral-2dim-core:

  • The Flatliner outlier detector picks up on spectra that only consist of one value.

  • The SpreadSheetColumnsToSpectra conversion generates multiple spectra from a spreadsheet, more flexible than SpreadSheetColumnToSpectrum. The SpreadSheetRowsToSpectra conversion works similar, but row-wise instead of column-wise.

  • The SpreadSheetColumnsToSampleData conversion generates sample data objects (= reports) from columns from a spreadsheet. The SpreadSheetRowsToSampleData conversion does the same using rows from a spreadsheet.

  • The SampleDataArrayToMap conversion generates a map from the array, using the sample ID as key for the map.

  • The SpreadSheetRowGenerator transformer allows you to generate spreadsheet rows from spectra using various generators.

• adams-heatmap: Added the MultiHeatmapOperation transformer, which allows applying an operation to an array of heatmaps.

• adams-spreadsheet:

  • The SpreadSheetUseRowAsHeader conversion replaces the values in the header row with the ones from the specified data row.

  • The SpreadSheetColumnsToReport and SpreadSheetRowsToReport conversions convert columns/rows in a spreadsheet containing meta-data into Report objects.

More work was done on the adams-matlab module and on image annotation support.

Fixes

• The SelectArraySubset transformer now updates its JList correctly when getting subsequently called, not just the first time round.

• Very long cell values in tables (> 1000 characters) now get excluded from calculating the optimal column width to avoid slow calculation and extremely wide tables.

• adams-meta: The InactiveStandalone/Source/Transformer/Sink actors now copy the actors provided to the constructors rather than just using the reference.

• adams-spreadsheet: Formulas in spreadsheets now resolve values of cell references to their native value, not just trying to convert it to double.

• adams-weka and adams-weka-lts:

  • The WekaAttributeSelection transformer no longer puts the ranked order of attributes back into sorted order.

  • Changing the attribute weights in an Instances table now gets applied to the correct column when displaying the instance weights column.

  • Exporting the output to PDF no longer duplicates the content of text files (due to two exporters available).

  • The Data generator source in the Weka Investigator works again.

  • String and relational values no longer get rendered as HTML as this turned out to be too slow when displaying a table.

Changes

• adams-weka: upgraded multisearch-weka-package to 2021.2.17

• adams-spreadsheet: deprecated SpreadSheetCommonIDs, SpreadSheetDifference in favor of MultiSpreadSheetOperation

• adams-timeseries: upgraded timeseriesForecasting Weka package to 1.1.27

• adams-matlab: the Mat5SpreadSheetReader can now retrieve arrays from struct objects as well

• adams-weka and adams-weka-lts:

  • Attribute weights now get displayed in the header row when showing weights.

  • The Revert action in the Weka Investigator now monitors files for changes (last modified timestamp) and gets enabled when either the file on disk got changed or it has been modified in the Investigator itself.

  • The SimpleArffLoader and SimpleArffSaver now allow the user to specify the file encoding.

Additions

• The MapToStorageValues transformer transfers a map object into internal storage.

• adams-imaging:

  • The ConfusionMatrix plugin for the ImageSegmentationContainerOperation transformer generates a confusion matrix of the annotated pixels versus predicted ones and the CountPixels one simply outputs a spreasdheet with counts per layer (of non-black pixels).

  • The IndexedPNGImageHandler for the Preview browser allows overriding the colors of PNG files that use an indexed color palette, e.g., when output by image segmentation models.

  • The BlueChannelImageHandler (using the new BlueChannelColorizer image transformer under the hood), interprets the values in the blue channel of an image as color indices and applies the colors from the supplied color provider accordingly.

• adams-spreadsheet: added the MultiSpreadSheetOperation transformer which applies the specified operation to the incoming spreadsheet array (CommonIDs, Sum, Difference, Merge).

• The HasProperty boolean condition can be used to check whether objects have a certain property name, e.g., when updating object values via the UpdateProperty transformer.

• adams-matlab:

  • The MatlabStructToMap conversion turns a Matlab struct object into a map (field/array relation).

  • The IsMatlabStruct boolean condition checks whether the token represents a Matlab struct object.

  • The MatlabStructInfo transformer allows outputting information on a Matlab struct object.

  • With the flow adams-matlab-inspect_file.flow you can inspect .mat files interactively.

• adams-weka and adams-weka-lts: The Text directory source in the Weka Investigator allows the loading of directories with plain text documents as datasets.

Since I'll be out of the office the next few weeks, just a quick update on what has happened since the last post:

Fixes

• Restricted stopping is now working as expected in conjunction with LocalScopeTransformer and LocalScopeTrigger.

• adams-event: The (internal) schedulers are now per flow rather than global, i.e., the same flow can be run in parallel and stopped without affecting the other instance.

Changes

• Switched to using Java 11 for builds, but source and target version are still Java 8 for the time being.

• The LineByLineTextReader now has a -max-lines option to limit the number of lines read.

• adams-webservice: upgraded Apache CXF to 3.3.9

• adams-excel: upgraded Apache POI to 3.17

• adams-event: updated quartz-scheduler to 2.3.2

• adams-imaging: The ObjectCentersOverlayFromReport, ObjectLocationsOverlayFromReport and ObjectLocationsOverlayFromSpreadSheet overlays (and corresponding plugins for the Preview browser) now allow varying the color of the shape as well. This is useful when displaying mostly one type of object in the image.

Additions

• The ZipArrays source actor allows the row-based iteration through arrays of the same length that are available in storage (like Python's zip operation).

• adams-weka and adams-weka-lts: added FFT filter (unsupervised, attribute).

• Added adams-matlab module for reading/writing binary Matlab .mat files.

• adams-imaging:

  • The ApplyMask multi-image operation applies an image mask to another image (pixels in image get set to black if mask pixel is black).

  • The ObjectAnnotationsMask BufferedImage transformer applies the object annotations in an image's report to the image and sets all other pixels to