Updates 2018/12/20

Just a short list of updates, due to upcoming release. Main new feature is an SQL workbench for running SELECT, UPDATE, DELETE queries from within ADAMS.

Fixes

  • adams-rest: added missing Maven dependencies to make it work with Java 9+.

Changes

  • Added button to the control panel for debugging flows, to highlight the current actor in the editor window.
  • adams-weka: upgraded Weka dependency to 3.9.3-fork-0.0.3, fixes duplicate output in console for LogBack and log4j
  • adams-weka-lts: upgraded Weka dependency to 3.9.0-fork-0.0.8, fixes duplicate output in console for LogBack and log4j

Additions

  • adams-weka and adams-weka-lts: added menu item for Weka's SimpleCLI
  • adams-spreadsheet: added SQL Workbench for running SQL queries on SQL databases.

Updates 2018/12/07

A fair amount of work was going on in the Weka-related modules, with a new Compare tab getting introduced to the Weka Investigator and a number of new filters. The content handlers in the Preview browser can be customized now, provided they have customizable options.

Fixes

  • When swapping SetVariable transformer with standalone (and vice versa), the transfer for variable of variableValue now works. Also added transfer for expandValue, valueType properties.
  • String content on the clipboard is now only parsed in the Flow editor when performing an actual paste. This is to avoid lengthy wait times when switching between flows or generating popup menu when the clipboard content is large.
  • Pasting from clipboard in file or directory chooser panels now updates the history as well.
  • adams-rats: meta ratinput/output schemes now call the base initReception/Transmission methods; the Cron input scheme now installs the cronjob in the initReception method (to avoid race conditions), DeQueue configures queue in initReception, Socket sets up server socket in initReception, Subscribe configures subscription in initReception.
  • Updating a value using a field with a different data type as currently stored in a report didn't update the field itself, the old key was kept instead. Problematic when updating reference data for spectra, attempting to fix the data type of values.
  • adams-spreadsheet: fixed the quick info of the SpreadSheetAddRowID conversion.
  • adams-spectral-2dim-core: the Preview browser now handles simple spectrum files (.spec) again.
  • adams-weka and adams-weka-lts:
    • fixed setting of numZeroes in RemoveWithZeroes filter.
    • the WekaFileChooser now lists all loaders/savers again
  • adams-net: fixed generation of output file name in SFTPGet transformer.
  • adams-rats: bulk actions now only get applied to checked ones, not all; checking/unchecking RatControl panel now uncheckings the RatsControl panel.

Changes

  • NumericStringCompare, as used by the Sort transformer, now providers better error messages when regexp replacement fails.
  • Content handlers in the Preview browser can be customized now (only some offer cutomizable options)
  • The Flow editor now has an Actors help menu item, which brings up class help just for actors. Additional copy operations got movied into Copy... sub-menu to keep menu cleaner, also added the Copy acto name action to sub-menu.
  • adams-spreadsheet: The spreadsheet table now has menu items in the header popup menu for setting column width/widths manually.
  • adams-spectral-3way-core: added option to the SimpleEEMReader to denormalize the amplitudes using the stored normalization factor.
  • adams-python: PythonEnvironment now allows updating the PYTHONPATH environment variable (append, prepend, replace).
  • adams-imaging:
    • The Grid scheme of the SubImages BufferedImageTransformer algorithm now offers options for overlaps on x and y axis, to avoid cutting through objects of interest.
    • Renamed transformer ImageReader to ImageFileReader and sink ImageWriter to ImageFileWriter to make them consistent with other actors that read/write files.
  • adams-weka and adams-weka-lts:
    • added dataset-weights-weka-package 2015.6.16 dependency
    • SimpleArffLoader and SimpleArffSaver now handle instance weights
    • The Weka Investigator now has a popup menu on tabs (right-click) and also allows saving/loading of parameters of individual tabs. Data tab can display extra column with instance weights now. Closing tabs no longer prompts a dialog, as closing can be undone now (via Tab menu).

Additions

  • Added InList boolean condition for checking whether incoming string is in list of strings.
  • The CollectionToArray transformar turns any collection (eg java.util.List or java.util.Set) into an array.
  • The StorageCollectionInsert transformer updates a collection object in storage with the object(s) passing through.
  • The Find in files interface allows you to search for files in a directory that contain a certain search string. Search text can be either a simple string or a regular expression.
  • adams-ml: added the CrossValidation scheme to the file-based dataset generation for generating cross-validation folds.
  • adams-weka and adams-weka-lts:
    • The WeightsBasedResample filter (package weka.filters.unsupervised.instance) allows you to resample a dataset using its weights.
    • The AccumulatedLWLWeights filter (package weka.filters.unsupervised.instance) generates weights for dataset using LWL-based approach, accumulating them and normalizing them to [0;1].
    • The RemoveWithWeights filter (package weka.filters.unsupervised.instance) removes instances outside the specified range.
    • The new Compare tab in the Weka Investigator allows you to compare two datasets via a common ID attribute (string or nominal): lists common IDs and ones that are only in first or second dataset, visual comparison of internal numeric values in separate tab (mainly aimed at spectral data).
  • adams-imaging: added simple framework for generating negative regions (regions without annotations) using annotated images as input for automating negative region generation: NegativeRegions transformer and class hierarchy of schemes.

Updates 2018-11-09

Not really a theme this time, some libraries got upgraded to fix security vulnerabilities, a general framework for file-based dataset generation (with CNTK Faster-RCNN a concrete implementation), the Weka Investigator got two new tabs (Fast ICA, Data query) and other bits and bobs.

Fixes

  • adams-visualstats: setting x/y index in the scatter plot no longer shifts the panel (getX/getY where overriding the methods of the JComponent class)
  • adams-weka and adams-weka-lts: The WekaTrainClusterer transformer now displays the selected post-processor correctly in its quick info.

Changes

  • upgraded commons-compress dependency to 1.18 due to security vulnerability: https://nvd.nist.gov/vuln/detail/CVE-2018-11771
  • The Copy transformer now has option for specifying type of copy.
  • The StringMatcher transformer as a new flag outputIndices for outputting the 1-based indices instead of the strings.
  • The File commander now has an extra column listing the file extension, to make sorting by file type easier.
  • The Preview browser now offers a simple note-taking functionality: adding notes via popup menu in files list; clearing/viewing/saving of notes via main menu.
  • adams-pdf: upgraded pdfbox to 2.0.12+ to fix https://nvd.nist.gov/vuln/detail/CVE-2018-11797
  • adams-rsync: upgraded rsync4j to 3.1.2-13 (OpenSSH 7.9 binaries for Windows)
  • adams-weka:
    • upgraded Weka to 3.9.3-fork-0.0.2
    • upgraded matrix-algorithms to 0.0.12
    • The ClusterCenters clusterer post-processor now allows computation of median for numeric attributes and most common label for nominal ones.
  • adams-weka-lts:
    • upgraded Weka to 3.9.0-fork-0.0.7
    • upgraded matrix-algorithms to 0.0.12
    • The ClusterCenters clusterer post-processor now allows computation of median for numeric attributes and most common label for nominal ones.
  • adams-spectral-3way-core:
    • SimpleXYZReader and SimpleXYZWriter now support gzip compression
    • feature generators are now more robust when trying to access incorrect report values.
  • adams-spreadsheet:
    • SpreadSheetAddFormulaColumn now has a placeholder for the last row
    • SpreadSheetAddFormulaRow now has a placeholder for the last column
    • The expression language for LookUpUpdate now has has/remove methods for checking whether a variable is present and to remove it respectively.

Additions

  • Added a transformer for reversing array order: ArrayReverse
  • adams-imaging: transformer for cleaning up overlapping images: DeleteOverlappingImageObjects
  • adams-cntk: added preview handler for CNTK FasterRCNN ROIs: CNTKFasterRcnnRois
  • adams-weka and adams-weka-lts:
    • added experimental FastICA (independent component analysis) support to the Investigator as new tab (ICA).
    • added the WekaClustererPostProcessor transformer to allow for multiple post-processing of a container output by a WekaTrainClusterer actor.
    • added the following clusterer post-processors: AddCluster (adds the predicted cluster), ClusterCounts (records how many instances get clustered in a certain cluster).
    • added new tab to the Weka Investigator for sql-like queries: Data query
  • adams-spectral-2dim-core: added experimental FastICA support to the SpectrumExplorer.
  • adams-ml: added simple framework for preparing/generating datasets from files, using: PrepareFileBasedDataset and GenerateFileBasedDataset.
  • adams-cntk: added file-based dataset generator plugin CNTKFasterRCNN.
  • adams-spectral-3way-core:
    • added Shift filter, which allows shifting of x, y or z values.
    • Upgraded multiway-algorithms dependency to 0.0.8

Updates 2018/10/19

Quite a bit of work underneath the hood has happened, UI improvements relating to the Flow editor, improved previewing for spectral data types and lots of little changes and fixes.

Fixes

  • The ListStructureModifyingActors actor processor now skips disabled actors (used when debugging).
  • GOE editors that handle actor references like callable actors now skip any actors that are disabled.
  • Rearranged the Flow editor tree popup menu a bit to streamline usage: combined external actor operations; moved less commonly used operations down.
  • Added support for Flow editor tree popup menu extensions, ie menu items that get inserted after/before an existing menu item (Tree.PopupMenuExtensions in FlowEditor.props settings file).
  • The FileInfoHandler of the Preview Browser switched to using java.nio.file.Files.XXX methods (due to bug in java.io.File.XXX methods, see https://bugs.java.com/bugdatabase/view_bug.do?bug_id=6203387); added owner, posix permissions, regular file, symbolic link to the output.
  • Recent refactoring of the generic plot panel and its axes, resulted in invalid formattters getting assigned. This resulted in the Memory monitor x axis showing the HH:mm plus the Java Epoch.
  • BOMs (byte order marks; used by UTF variants) are now handled properly by the CsvSpreadSheetReader, the generic loading of text files and the various TextFileReader schemes.
  • adams-imaging:
    • Annotating objects is no longer generating negative heights or widths when not starting selection rectangle in top-right corner.
    • The Objects sub-image algorithm now ensures that the sub-image dimensions do not exceed the original image.
  • adams-ml: The ConfusionMatrix transformer now handles unclassified predictions (aka missing values).
  • adams-spreadsheet:
    • SpreadSheetDisplay now adheres to the useOptimalColumnWidth flag.
    • Spreadsheet file viewer: now hides rows with missing values when applying filters; Copy column now copies only the visible cell values instead of all.
    • The SpreadSheetMerge transformer now removes duplicate unique columns correctly (failed to remove correct ones if no prefix/ID was used), when using the keepOnlySingleUniqueID flag.
  • adams-weka and adams-weka-lts:
    • WekaPredictionsToSpreadSheet (also used in generating the confusion matrix view in the Investigator) now ensures that labels are treated as strings now, to avoid numeric conversions for labels like '00' or '07'.
    • WekaPredictionsToSpreadSheet, WekaPredictionsToInstances and the Investigator now correctly align the original instances with the predictions of cross-validation when using a grouped split generator.
  • adams-spectral-2dim-core: The OpusSpectrumReaderExt now applies the scaling factor for Y defined in the file when reading the amplitudes.

Changes

  • Switched from Swing widgets to ADAMS ones: JButton, JCheckBox, JComboBox, JTextArea, JTextField, JTextPane. This makes it easier to introduce additional functionality in the top-level class.
  • The Command and Exec sources and the Exec sink now support setting of custom environment variables.
  • Added option for conversion to be applied to values, to have better control over types for hashset-handlers: HashSetInit (standalone/transformer), HashSetAdd (transformer), HashSet (boolean condition).
  • The parsers for BooleanExpression, MathematicalExpression and StringExpression now allow expression variables (not to be confused with flow variables) to be strings consisting of alphanumeric chars and "_" (but must start with uppercase letter to distinguish them from built-in functions), as well as enclosing via single quotes instead of just "[]".
  • Report tables are now using a case-insensitive sort on the field names.
  • The NewReport source now uses BaseClassname instead of String for the report classname.
  • BaseClassname now has its own GOE editor, which allows you to search for classes.
  • The GenericArrayEditor now allows the sorting (asc/desc) of the elements.
  • The SetContainerValue control actor now allows storage of non-standard, additional meta-data via the force flag.
  • adams-rats: The Exec input and output now support setting of custom environment variables.
  • adams-spreadsheet: The LookUpUpdate parser now allows enclosing of expression variables in "[]" as well, aligning it with the other parsers.
  • adams-spectral-3way-core: upgraded multiway-algorithms dependency to 0.0.7.
  • adams-spectral-2dim-core:
    • Added -lenient flag to the InstanceGenerator transformer to only log errors instead of returning them (and possibly stopping the flow).
    • The spectrum preview in the Preview browser and the object renderer when debugging a flow now display spectrum and sample data via a tabbed pane, rather than just the spectrum.
  • adams-compress: upgraded unrar dependency to 3.0.0
  • adams-net:
    • removed obsolete org.apache.httpcomponents/httpclient dependency
    • upgraded tika-core dependency to 1.19.1

Additions

  • Added the PullUpActor tree editor popup menu and keyboard action, which moves all sub-actors of a mutable actor handler (like Tee or Trigger) up one level and removes the enclosing actor handler.
  • Added little test dialog for date formatting/parsing in Developer user mode in the Maintenance menu.
  • adams-ml: added basic filter framework for spreadsheet-based datasets, SavitzkyGolay, PLS, MultiStream, MultiBatch.
  • adams-weka and adams-weka-lts: added OuterProductAnalysis filter (Fabricio S.Terra, Raphael A.Viscarra Rossel, Jose A.M.Dematte (2019). Spectral fusion by Outer Product Analysis (OPA) to improve predictions of soil organic C. Geoderma. 335:35-46. URL https://doi.org/10.1016/j.geoderma.2018.08.005.).
  • adams-spectral-2dim-core:
    • Added new AbstractMultiSpectrumFilter schemes: Add and Subtract.
    • Source for listing sample IDs of sample data with no associated spectra: OrphanedSampleDataIdSupplier
    • Added handlers in the Preview browser for spectrum formats: ASCIIXY, CML, DPT, JCampDX, JCampDX2, Json, MPS, Opus, OpusExt, Relab, SPA, SPC, SpecLib.

Updates 2018/09/16

It has been a while, I know. Been visiting clients and preparing to go live with some large ADAMS-based systems. On the data transformation side of things, there are a few more PLS variants available. For image processing, there is now better support for VIA/VGG annotations and data available. The experimental CNTK Java integration has been removed, too hard to maintain as it is tied to a specific CNTK version.

Fixes

  • The Flow editor no longer saves expanded external flows.

Changes

  • The Copy (plain text) operation in the Flow editor tree popup menu now uses the compact format as well.
  • The Flow control actor now allows to specify a restart manager. This allows the monitoring of flow(s)/setup file(s) by a flow that runs as a background service (daemon or Windows service), triggering its restart.
  • Added a GOE editor for the BaseInterval class.
  • adams-rsync: updated the rsync4j library to 3.1.12-12
  • adams-weka: upgraded multisearch library to 2018.8.16
  • adams-weka and adams-lts:
    • upgraded matrix-algorithms to 0.0.11
    • the filter PartitionedMultiFilter2 now allows specification of prefixes for the filters rather than just using 'filtered'
  • adams-imaging:
    • MergeObjectLocations can now just merge all locations when using minOverlapRatio=0.0 and overlapAction=KEEP
    • ViaAnnotations transformer has been removed.
  • adams-cntk: removed the experimental CNTK Java bindings, as they too hard to maintain and strictly limited to a specific release of CNTK.
  • adams-cntk-weka and adams-cntk-weka-lts: removed any code that relied on the experimental CNTK Java bindings.

Additions

  • Added menu item Copy actor path to the Flow editor's popup menu.
  • Added class hierarchy for conversions that swap out objects and concrete SwapObjects implementation for replacing one object with another, using their command-lines.
  • adams-meta:
    • added actor wrappers for auto-generated actors, which can get removed using the RemoveAutogeneratedActors processor: AutogeneratedStandalone, AutogeneratedSource, AutogeneratedTransformer, AutogeneratedSink
    • added actor wrappers for inactive actors (actors that don't get executed, but kept for reference), which can get activated again using the ReactivateActors processor: InactiveStandalone, InactiveSource, InactiveTransformer, InactiveSink
  • adams-net:
    • added MarkdownToHTML conversion for converting Markdown string into HTML string
  • adams-weka and adams-lts:
    • added more PLS algorithms: DIPLS, PRM, VCPLS
    • added SwapPLS conversion for replacing PLS algorithm (with optional migration of number of components)
  • adams-imaging:
  • adams-spectral-2dim-core: added the RowWiseSpreadSheetSampleDataReader sample data reader.

Updates 2018/07/26

The last few weeks I have been busy preparing the migration of a large, commercial ADAMS-based application for processing spectral data to a newer version. This new version uses Rats instead of cronjobs to improve processing speed of data and making predictions. Due to these flows now containing several 10,000 actors, a lot of speed improvements were necessary to make them easier to view and edit in the Flow editor. That introduced also a number of bugs (a lot of rewriting happened behind the scenes), but it has stabilized a lot now.

Fixes

  • The undo is now fixed in the Flow editor, with the flow no longer disappearing when performing an undo operation. Due to changes of the underlying data structure introduced the last couple of weeks, for speeding up the read/write of flows from/to disk.
  • Using now nested lists for copying/pasting actors on the clipboard for better performance, also using a faster check whether nodes are on clipboard for pasting.
  • The popup menu in the GOE failed on java.io.File objects due to a wrong cast to PlaceholderFile, which got fixed.
  • The GOE could end in an endless loop, firing property events that hammered the Swing thread, resulting in a lock up of the user interface. Removing an unnecessary call for firing property events, the lock up disappeared.
  • Reading of flows in the old nested format now works again.
  • File choosers that allow the editing of options of the selected file type (reader/writer) now ensure that the selected filter and the last handler used are of the same class when showing the dialog, resetting the last handler if that's not the case. Otherwise, cases can occur where the selected filter is not presented in the GenericObjectEditor.
  • adams-weka and adams-weka-lts:
    • The WekaFilter transformer no longer re-initializes the filter if loading from disk/actor/storage and initializeOnce is checked.
    • The CrossValidationFoldGenerator classes now report back the actual number of folds in use (eg when using -1 for LOO, this depends on the data), which in turn makes the WekaCrossValidationExecution work properly when using GroupedCrossValidationFoldGenerator in conjunction with LOO.
    • search methods that implement RankedOutputSearch (eg PrincipalComponents) now output the attribute importance from the ranking instead of the attribute index
  • adams-rats:
    • The Rat actor now disables the performLazySetup option if the sub-flow contains at least one Breakpoint actor (debugging doesn't work in conjunction with lazy setup).
    • The Subscribe rat input now ignores any notifications if the Rat actor is paused.
    • The RatControl actor how has optional support for bulk actions, ie checked Rat actors can be paused/resumed/stopped/started.
  • adams-spectral-2dim-core: fixed selection of instance generator when exporting visible spectra in the Spectrum Explorer: it is now possible to select from a class hierarchy

Changes

  • Introduced the adams.flow.processor.ActorProcessor interface, replacing the usage of the AbstractActorProcessor class.
  • The DeserializeToStorage standalone now accepts pairs of storage name / file name to reduce the number of actors necessary when loading lots of models. Storage name and/or file name can be variables as well.
  • The Flow editor now prompts whether to perform checks before saving a when it is considered large (default is 200 actors); occurs on a per-flow basis and can be reverted by simply checking/unchecking the checkbox in the menu.
  • Actors that implement the AtomicExecution interface (eg Tee or Branch) now allow the user to specify a timeout after which the sub-flow gets stopped nonetheless (when finishedBeforeStopping is set).
  • Upgraded lanterna to 3.0.1 (for providing an ncurses-like terminal interface).
  • adams-spreadsheet: The SpreadSheetSetCell transformer can use a cell finder now to locate the cells to update with the specified value as well (eg for updating all cells with missing values or a value less than a certain minimum).
  • adams-weka and adams-weka-lts: switched to updated Weka versions (3.9.2-fork-0.0.2 and 3.9.0-fork-0.0.5 respectively) that include patch for MyHeap usage in weka.core.neighboursearch.LinearNNSearch, avoiding the the number of elements cannot exceed the initially set maximum limit exception.

Additions

  • added Clear clipboard to menu of Flow editor
  • added MapToReport conversion
  • Added the ListObjectCommandlines actor processor for listing object command-lines of a specific class.
  • Added actor processor for updating the number of threads used by classes: ChangeNumThreads
  • adams-spreadsheet:
    • The SpreadSheetCellSelector transformer lets you select cells in a spreadsheet to forward in the flow.
    • The SpreadSheetRandomSystematicSample transformer picks a random systematic sample of rows from the incoming spreadsheet.
    • added Clear clipboard to menu of Spreadsheet file viewer
  • adams-weka and adams-weka-lts:
    • The PredictionTrend output generator for the classification tab of the Weka Investigator plots actual and predicted values in a single plot, sorted by actual. This allows spotting of trends.
    • Added NIPALS and SparsePLS
  • adams-meta: The ProcessorActor transformer allows you to process an incoming flow with a specific actor processor (eg listing variables or command-lines).
  • adams-spectral-2dim-core: added instance generator based solely on report fields and with dedicated class field: FieldInstanceGeneratorWithClass

Updates 2018/06/26

Finished some major refactoring related to loading and saving flows. Well, actually replacing an ugly hack from two years ago with a clean implementation. Properly decoupling user interface and readers/writers for flows. That and some further optimizations in the Flow editor resulted in nice speed improvements: a flow with 38,000 actors now loads in about 3 seconds on my laptop. BTW this is the largest flow that I have generated so far!

Further CPU profiling also revealed some other code locations which could be optimized, improving startup time of flows.

Fixes

  • The Preview browser now properly updates the display when selecting a different handler for serialized object.
  • SetManyVariables standalone and transformer now use a list internally, which improves performance of the addVariablePair method when setting hundreds of variables.
  • User interface improvements in the Flow editor that relate to long running background tasks, they no longer block the interface.
  • General Flow editor improvements to speed things up with flows that have 10s of thousands of actors.
  • The Switch and IfThenElse actor now clean up the current token once execution finishes, to avoid unnecessary memory usage.

Changes

  • The Image viewer now uses a multi-page instead of multi-tab interface as well.
  • Copy/paste of flow nodes now uses a custom DataFlavour on the clipboard to speed up checks whether pasting is available; added "Copy (plain text)" for old approach.
  • Switched to sizeofag 1.0.4. This enabled the Inspect memory sub menu in the Flow editor, displaying the memory consumption of the selected actor (detailed per class or just overall size).
  • An alternative visualvm executable can be specified now by the user through preferences (or the JVisualVM.props properties file). JVisualVM got dropped from JDKs starting with Java 9. The project is now maintained separately and you can download it from here: https://visualvm.github.io/
  • adams-rsync: upgraded rsync4j to 3.1.2-10.
  • adams-rats: a Rat actor can free up memory now with the wrapUpAfterExecution flag, which is available in conjunction with performLazySetup.
  • adams-weka and adams-weka-lts: upgraded to matrix-algorithms 0.0.4.
  • adams-webservice: upgraded Apache CXF to 3.2.5.

Additions

  • Added simple source actor for just keep the flow running: Idle. This is useful when using an event-based flow (eg using Rat actors) rather than a data driven one.
  • The ForceFlush control actor enforces actors implementing FlushSupporter (currently DumpFile and WekaInstanceDumper) to flush their buffers.
  • With the ArrayToChunks transformer you can split an array into sub-arrays of a specified length.
  • adams-weka and adams-weka-lts: meta-classifier that can use a fallback model for making predictions: Fallback
  • adams-rats: the ExecuteRats control actor allows execution of Rat actors in MANUAL mode whenever a token passes through.
  • adams-compress: added support for Zstd de/compression library (https://en.wikipedia.org/wiki/Zstd).
  • adams-spectral-2dim-core: added the MaxWaves outlier detector for enforcing an upper limit on wave numbers.
  • adams-spectral-3way-core: added support for feature generation through the ThreeWayDataFeatureGenerator transformer and its associated class hierarchy of feature generators.

Updates 2018/05/25

A lot of work went into the 3-way data module, as well as proper support for polygons in the image annotation framework. The audio module also got a bit of love.

Furthermore, some user interfaces got switched from multi-tab to multi-page ones, freeing up vertical screen real estate (most people of have wides-screen monitors!).

Fixes

  • External actors that use a file monitor can once again use variables in the file name for the external flow.
  • The flow panel and flow tree now get focused better, to enable a smoother keyboard interaction.
  • adams-imaging: The ImageAnnotator now uses a scrollpane to display more buttons than fit on screen.
  • adams-weka and adams-weka-lts: the MultiSearch meta-classifier now allows you to select the search algorithm for the parameter space.

Changes

  • CsvSpreadSheetWriter now allows an empty quote character, which will suppress any quoting.
  • Switched the following applications from multi-tab interface to a multi-page one in order to manage more tabs/pages at the same time without decreasing the vertical screen real estate (these pages can be detached/reattached):
    • Flow editor
    • adams-imaging: Image processor
    • adams-heatmap: Heatmap viewer
    • adams-spreadsheet: Spreadsheet file viewer

Additions

  • The implicit variable flow_filename_restore contains a file name based on the full file name of the flow, but with the .props extension. To be used for interactive actors like EnterManyValues to restore the entered values.
  • Added Favorite actor (here, after, beneath) keyboard actions (Ctrl+E, Ctrl+Alt+E, Ctrl+Shift+E).
  • adams-spectral-2dim-core: added meta spectrum reader that reads spectra from zip files: ZippedSpectrumReader.
  • adams-spectral-3way-core:
    • added heatmap viewer for 3-day data, like 3D fluorescent spectra (EEM)
    • added experimental support for building and applying the multi-way algorithms available from: https://github.com/waikato-datamining/multiway-algorithms
    • added lots of conversion to and fro tensors, 3-way data and spreadsheets
  • adams-audio:
    • With the AudioRecorder source it is possible to record audio from a microphone.
    • The WaveToAmplitudes conversion extracts amplitudes from Wave object, absolute or normalized
    • Added Resample wave filter for using different sample rate.
    • Added Cut wave filter for cutting out a portion of a wave.
    • Added sink for playing back audio files: AudioPlayback
  • adams-compress: added support for xz compression via Xz/UnXz transformers and XzTextFileReader.
  • adams-imaging:
    • Added the Grayscale buffered image transformer with customizable lumiance parameters via a class hierarchy (supporting BT 601, 709 and 2020).
    • The Rotate object filter rotates objects by the specified angle.
    • ViaAnnotations transformer generates annotations for VIA (VGG Image Annotator - http://www.robots.ox.ac.uk/~vgg/software/via/)
    • Added support for polygon annotations via the PolygonPainter (default is RectanglePainter).

Updates 2018/04/20

Spent a lot of time trying to get things working with Java 9. The usual backwards compatibility mantra got a serious beating with Java 9. ADAMS now compiles with Java 9, but I still recommend to stick with Java 8 for a while.

Futhermore, there is now a separate module for ffmpeg called adams-ffmpeg, rather than being tucked away in the adams-video module. In addition to that, I added an experimental module for audio processing called adams-audio.

Fixes

  • Now compiles with Java 9.
  • Auto-registering of flows now registers after the flow ID has been initialized rather than with 0.
  • adams-imaging:
    • The draw operations ObjectCentersFromReport and ObjectLocationsFromReport now use the user-specified font.
    • The image overlays ObjectCentersOverlayFromReport and ObjectLocationsOverlayFromReport now use the user-specified font.

Changes

  • Upgraded sizeofag to 1.0.2.

  • Upgraded jclasslocator to 0.0.10.

  • Removed archetype module.

  • Added keyboard shortcuts to the Flow editor for bringing up the actor tree to search for an actor to insert, skips the GOE dialog: Ctrl+J (here), Ctrl+Alt+J (after), Ctrl+Shift+J (beneath)

  • adams-spectral-2dim-core: The Evaluator transformer now allows to turn off

    cloning of container storing the evaluation results.

  • adams-webservices: updated Apache CXF to 3.2.4 to make it compile with Java 9 (10 doesn't work yet, but the upcoming 3.2.5 release has a fix for that).

  • adams-meka: Upgraded to MEKA 1.9.2.

  • adams-imaging:

    • The draw operations ObjectCentersFromReport and ObjectLocationsFromReport now offer x/y offsets for the label.
    • The image overlays ObjectCentersOverlayFromReport and ObjectLocationsOverlayFromReport now offer x/y offsets for the label.
    • ImageObjectInfo now allows to output the INDEX_STRING and INDEX_INT as well.

Additions

  • Added support for publish/subscribe: InitPublishSubscribe standalone for initializing pub/sub data structure in storage and the Publish sink for broadcasting the data to all the subscribers of the specified pub/sub data structure.
  • Added platform specific file-in-use checks: Windows and NonWindows (Default combines both).
  • Added the JsonObjectToMap conversion to turn a JSON object into a simple map.
  • Added the ExtractSimpleContainerContent conversion to retrieve the content of simple containers.
  • Added the Go to menu item to the Flow editor popup menu, which allows you to jump to the reference actor (eg a callable actor).
  • adams-rats:
    • added support for using sockets for sending and receiving data, aptly named Socket for input and output (adams-rats-sockets.flow).
    • added support for publish/subscribe with Subscribe rat input and Publish rat output. Simpler and more efficient than queues for notifying multiple Rat actors at once.
    • added Django web application for remote control of Rat actors that are available through the RatControl actor, using a REST webservice for communication.
  • adams-weka: The SocketFacade meta-classifier communicates via sockets with a prediction scheme, can be used to interface with any programming language as long as communication occurs via sockets.
  • adams-imaging:
    • added BasicStatistics feature generator (for buffered images).
  • adams-audio: new module for some basic audio processing of WAV files, generating spectrograms (as image or tabular data).
  • added separate adams-ffmpeg module, to make ffmpeg functionality available in adams-audio and adams-video.
  • adams-ml: added basic framework for calculating summary statistics with the SummaryStatistics transformer.
  • adams-spectral-2dim-core: added the SimpleSpectralInstanceGenerator instances generator, which does not output a class attribute in the generated Weka Instances object.