Final update for the year

Here is the final update for this year. Later on today, I will prepare a new release.

'Tis the season! ;-)

Well, actually mainly because of all the GUI improvements and the new adams-nlp module.

Fixes

  • Renaming variables, callable actors, events and undo/redo no longer result in the flow wobbling around (better queuing in Swing thread)
  • Improved the layout of the search panels, allowing for proper resizing and avoiding accidental hiding of search box

Changes

  • Plots now offer a Copy plot menu item from the right-click menu, to copy a screenshot of the plot to the clipboard
  • Count control actor now offers a monitor variable which allows resetting the internal count whenever the variable changes value
  • The Create callable actor menu item from the Flow editor's right-click menu now allows the creation callable actors under CallableActors, GridView and TabView.

Additions

  • Copy to clipboard target got added to the Send to functionality, which allows you to copy text, tables and graphical components (= screenshots) to the system's clipboard
  • TextOverlay added for simple text overlays in plots (eg SequencePlotter/SimplePlot); gets used by the Weka plugin for classifier errors to display key on lines drawn on screen

Stay tuned!

updates 4/12

Last week, I gave a data mining workshop using ADAMS in Kuching, Malaysia. The material of the workshop is available from my github account.

I noticed that Windows is not particular great in regards to displaying unicode characters (e.g., emoticons). During my workshop we were also covering twitter sentiment analysis and showing all emoticons as little squares is not a very useful. My Linux Mint displayed them without any issues, on the other hand.

Thanks to these issues and the workshop, there have been a number of changes:

Fixes

  • Flow editor: annotations/quick info/input+output menu items in the View menu now get correctly updated when switching tabs
  • run/run.bat start scripts now specify the correct java-cup jar library in the CLASSPATH
  • Flow editor: Parameters tab now displays inline scripts in a more sensible format

Changes

  • added support for reading/writing flow files (.flow, .flow.gz format) with different file encodings (eg UTF-8/16)

Additions

  • adams-weka: added more string tokenizers and new token cleaner class hierarchy
  • adams-core: SwitchedSource actor allows you to output tokens based on boolean conditions, similar to the Switch control actor
  • adams-video: simple VLC-based video player got added by my student Steven Brown - requires VLC to be installed locally
  • added dialog for customizing fonts of widgets, e.g., to select more suitable font for displaying various unicode characters
  • the module adams-nlp has left the incubating stage and got moved to adams-addons

Have a good weekend!

upcoming release

Just a heads-up that I'll be making a new release tomorrow.

Since last week, only a few things changed:

  • fixed a flicker issue when processing videos in the flow and displaying the individual frames with a custom zoom in the ImageViewer (zoom got reset to 100% before changed to custom zoom)
  • adams-weka: added WekaGetCapabilities transformer to retrieve capabilities from classes like Filter, Classifier and Clusterer. The WekaCapabilitiesToInstances and WekaCapabilitiesToSpreadSheet conversion use this information. The former to generate random data, the latter to output the capabilities in tabular format.
  • the Flow editor now allows you to define keyboard actions (eg adding or enclosing actors using a specific actor) to speed up flow development (see manual "Keyboard actions")

Fingers crossed that tomorrow's release goes smoothly. :-)

update 13/11

It's this time of the week again and there have been a number of updates again.

Fixes

  • adams-core: the Inspect control actor no longer interprets skip as cancel
  • adams-core: the BreakPoint actor no longer locks the application when attempting to stop the flow inside a block that uses atomic execution (= finishedBeforeStopping option in Tee/Trigger/etc)

Changes

  • adams-core: the ImagePanel now allows undo/redo through the right-click popup menu as well
  • adams-imaging: Crop paint selection plugin offers undo/redo and also stores the x/y and width/height of the crop (in relation to the original image); PaintSelection supports undo/redo now too
  • option parsing of integer types now handles float-type strings (eg 11.0) as well, by parsing them as float and the retrieving the correct value - this is useful when dealing with numeric values stored in reports, as the default type there is float (even if you store an integer)
  • adams-meka: MEKA has been upgraded to 1.9.0
  • Index/Range (and spreadsheet/WEKA related classes) now allow enforcing of numeric indices by preceding them with "#" (eg "#13"); all names (columns/attributes/labels) can be enclosed with double quotes now
  • tool tips in the GenericObjectEditor have been improved; e.g., classes that implement ExampleProvider (eg Index/Range) now automatically append the example text

Additions

  • adams-core: new actor processors ListAllVariables and ListAllStorageNames
  • adams-weka: LeanMultiScheme is an extended version of Weka's MultiScheme meta-classifier with better memory management
  • adams-core: ReportToVariables and ReportToStorage transformers allow the transfer for multiple report values into variables/storage items in one go
  • adams-core: ModuleInfo source actor outputs information on the ADAMS module that make up the ADAMS application

Have a good weekend!

time for an update

I'm running out of space on my piece of paper here, so I think it's time for an update. ;-)

Fixes

  • fixed some regressions in the flow editor due to changes relating to better Swing thread queuing
    • when copy/pasting actor with variable, variable disappeared
    • creating a callable actor didn't no update the tree immediately
  • the showProperties option of the ImageViewer sink now works inside the DisplayPanelManager as well

Changes

  • added the DATETIMEMSEC type to the DateTimeType enum
  • added DATETIMEMSEC support to the following conversions
    • ConvertDateTimeType
    • DateTimeTypeToString
    • ExtractDateTimeField
    • StringToDateTimeType
  • added DATETIMEMSEC support to the following actors
    • DateTimeTypeDifference
    • SpreadSheetConvertCells
    • SpreadSheetConvertHeaderCells
  • Flow editor: long files now get shortened for the tab; hovering over the tab reveals the full name
  • Cast control actor no longer throws a cast exception when using Unknown
  • data containers now list Notes and Reports in the debug inspection as well
  • adams-spreadsheet: SpreadSheetSetCell and SpreadSheetGetCell now support Row objects as well apart from SpreadSheet ones
  • adams-heatmap: the report table in the HeatmapDisplay can be hidden now
  • native ADAMS file choosers now offer a Filter text field which allows the display of files/dirs to be restricted to the ones that contain this substring (case-sensitive) - useful when looking for a single file or group of files in directories with 1000s of files
  • adams-weka: the WekaModelReader now has an option makeThreadSafe which attempts to make the loaded classifier thread-safe, if necessary (using the ThreadSafeClassifierWrapper mentioned below)

Additions

  • adams-heatmap: added the CountValues meta-generator
  • adams-heatmap: added TextOverlay for HeatmapDisplay sink
  • adams-core: added support for named counters, i.e., counters that count how often a string passes through
    • CounterInit (standalone/transformer) - initializes/resets a named counter in internal storage
    • CounterAdd (transformer) - increments the counter for the string representation of the token passing through
    • Counter (source) - outputs the names with their associated counts as spreadsheet
  • adams-weka: added an interface ThreadSafeClassifier that indicates that a classifier is thread-safe (Weka classifiers/filters aren't by design) and a generic ThreadSafeClassifierWrapper meta-classifier that attempts to make the wrapped classifier safer (there are no guarantees, though; if the code is bad by design, this won't help either)

Just a heads-up... There will be a new release coming up in the next couple of weeks (as part of my preparations for a data mining workshop end of November).

update 24/10

It's been a while since the last update on ADAMS, as I've been busy with our commercial projects and development mainly occurring in these frameworks. However, there were some changes that also affected the public ADAMS code:

Fixes

  • ConsoleWindow standalone now correctly displays the window
  • fixed launching Terminal on Windows (for Windows users: you know terminals as command-prompts)
  • StorageForLoop source now implements the VariableUpdater interface to make the variable usage check pass

Changes

  • ConsoleWindow now allows saving all the messages coming through to a file

Additions

  • adams-weka: the Veto meta-classifier favors the specified label if it is predicted by at least the specified number/percentage of classifiers in the ensemble. If not, then Vote with majority rule is used.
  • adams-weka: the SuppressModelOutput meta-classifier is useful if you want to suppress any large model output, e.g., from large ensembles
  • adams-weka: with the ThresholdedBinaryClassification meta-classifier you can move the default probability threshold of 0.5 to decide which label is selected to another value
  • adams-weka: the RemoveOutliers filter uses the outlier detection framework introduced here, removing all instances that get marked as outliers

Have a good weekend!

Friday update

It's time to bring you up-to-date what's been happening under the hood of ADAMS lately. Well, for one thing, I got invited to give a demo at the BigMine 2015 workshop as part of KDD 2015 in Sydney. Eventually, a paper will be out.

The other major thing is that we definitely stick now with Java 8, mainly due to the better support of Java 8 in regards to file handling (java.nio.Files namely) and atomic moves of files/directories. I had the problem with a commercial product (and, of course, Windows) that files got move, but Windows still thought that they might be in use. Resulted in failures to process files. Rather annoying. However, moving to atomic moves, this problem has gone away now.

Fixes

  • AbstractChooserPanel and its derived component no longer doubles up the entry when pasting text into it.
  • PropertySheetPanelPage (used in wizards) now correctly updates the buttons when values change.
  • Fixed a performance issue in the Instance Explorer when selecting the color for many selected rows.
  • The Clear graphical output menu item in the Flow editor was a bit to aggressive in the cleanup and left the editor in a broken state (NullPointerExceptions galore).

Updates

  • The message option of the EnterManyValues source handles variables now as well, automatically expanding them
  • The storage panel, as used in the Breakpoint's user interface, now allows the export of selected objects, just like the object tree.
  • SpreadSheetTable/Model now allow you to toggle between cell value and cell type view. Sometimes it's easier to spot a string value where a number should be with this functionality (via popup menu).
  • Instance Explorer now allows you to use the value of an attribute as ID rather than the row number.
  • Tables now offer copy row/column as well through their popup menus.
  • HashSetAdd transformer now accepts spreadsheets and arrays of objects as well, for speeding up the add operation.
  • The CsvSpreadSheetReader received a performance boost option, which can be used for spreadsheets that have uniform column types: it allows you to specify a number of rows to use for identifying the column type automatically. The speedup on my laptop for a 2400 column by 2000 row spreadsheet with mainly floating values was 50% (30s instead of 60s).

Additions

  • WekaGeneticAlgorithm transformer added, which allows you to use the DarkLord genetic algorithm now also in the flow, not just through the wizard.
  • String conversion dialog added to the maintenance section (requires developer user mode), allows you to backquote/unbackquote etc strings.
  • RemoveTestInstances filter (package weka.filters.unsupervised.instance) allows you to remove rows from the data passing through (only first batch), by matching the IDs from a selected test set against the current data. Allows you to avoid manually removal of instances from a data set (hold out dataset) when training a classifier on the full dataset.

Have a great weekend!