ADAMS (Posts about control charts)

updates from a busy week

FracPete — Thu, 11 Jun 2015 19:20:51 GMT

It's been a busy week and debugging support has seen quite some improvements.

Debugging

PathBreakpoint now allows you to select the actor through a dialog rather than having to enter the path manually
due to some API changes to input consumers, it is now possible to inspect the current token when stepping through the flow manually, rather than only when reaching a Breakpoint control actor
instead of only outputting a textual representation of tokens (inspection panel), a class hierarchy for custom renderers has been implemented plain text (used so far; fallback), images, spreadsheets, reports, Weka Instance(s), timeseries
these rendering classes are now also used when previewing items from the internal storage ("storage" tab)
started work on adding ability to edit items in internal storage; at this stage primitives and arrays of primitives are supported, as well as objects that can be manipulated through the GenericObjectEditor (more will come)

Here is the summary of the remaining updates:

Fixes

The vertical marker paintlet used by the SequencePlotter now uses the actual minimum/maximum of the canvas, not just the one derived from the data

Changes

control charts plots can now be initialized with a container from the ControlChart transformer and subsequently display data using the last known limits; furthermore, added support for post-processsing, e.g., limiting the number of data points visible at any given time
spreadsheets are now editable in the viewer; the SpreadSheetDisplay sink offers the new "readOnly" option (default = readonly)

Additions

OpenFile sink opens the incoming file with the application associated with it as determined by the operating system
MissionControl control actor is a minimalistic interface for pausing/resuming/stopping a flow; useful when starting a flow from the commandline using the FlowRunner class instead of from the GUI
InputOutputListener control actor allows you to listen in on the tokens go into and coming out from a sub-flow by forwarding them to callable actors for further processing

Thanks for the feedback regarding the debugging functionality and have a good weekend!

updates 5/6

FracPete — Fri, 05 Jun 2015 04:05:38 GMT

It's been a short, but busy week - Monday was Queen's Birthday holiday. Not only lots of data mining related work, but also lots of changes happening to ADAMS. The changes mainly improved usability in the Flow editor.

Fixes

MetaPartitionedMultiFilter now handles duplicate regular expressions, e.g., when filtering the same subset of attributes with several different filters

Changes

The notification area in the flow editor (eg displaying errors) now offers a "Jump to" menu item from its right-click popup menu. It lists all the actor names that were listed in the message, allowing you to quickly get to an actor that generated the error.
adams-spreadsheet: the LookUpAdd transformer now accepts spreadsheets as well for updating the look up table, not just pairs of objects
CopyFile/MoveFile now support the option for multiple attempts, in case the I/O operation fails (useful on Windows with its file locking)
adams-spreadsheet: the LookUpAdd transformer now accepts spreadsheets as well for updating the look up table, not just pairs of objects

Additions

Control charts are now available through "Chart" functionality of the spreadsheet viewer tool (adams-spreadsheet module).

Breakpoints

The breakpoint/debugging support in the flow has been reworked extensively, in order to make it much more user-friendly and efficient. This was possible by combining and extending the functionality of the "Debug" flow execution listener, which offered similar functionality to the "Breakpoint" control actor. However, setting it up was a bit tedious.

The new Breakpoint actors no longer have a GUI at all, they simply configure the "Debug" flow execution listener in a complete automatic fashion. By moving the control panel from the Breakpoint actor into the Debug listener, the added benefit is now that you only have a single window for all your breakpoints. Large flows with multiple breakpoints quickly became hard to debug, with many breakpoints simply called "Breakpoint" (uhm.. which one was this again?). Furthermore, the actor that is currently suspended, it now highlighted in the flow, so you can see where you are at. A new functionality is that you can now step through a flow, one actor at a time. This allows you to quickly check how variables/storage change, without having to stop the flow and add a new breakpoint at another location.

Breakpoints can be added/removed/modified/disabled/enabled at runtime very easily, using the new "Breakpoints" tab in the control panel (see attachment breakpoints_controlpanel.png).

The line-wrap bug in the token inspection panel has finally been fixed. When right-clicking in the object tree, you can now either copy the string representation to the clipboard or export it to various file formats. At the moment, spreadsheets, images and Weka datasets are supported apart from plain text.

Another minor, but significant modification: I added a "Run (debug)" menu item to the flow editor, which allows you to step through a flow, actor by actor, without having to have any Breakpoint control actors present.

That's all for this week!

weekly roundup 29/5

FracPete — Fri, 29 May 2015 03:45:57 GMT

Time to let you in on the latest changes...

Something that has been in the making for a bit, but was accelerated through the bug-hunt for locked files under Windows (a feature I always hated!):

ADAMS now requires Java 8 for building and running.

And here is the rest:

Fixes

SpreadSheetAggregate is now more robust in regards to failures, also handles empty input spreadsheets now correctly (no longer adds a phantom row)

Changes

TimeseriesDisplay now offers a "plot updater" option, which allows you to specify at what interval to update the display (or only at the end)
StringToValidVariableName conversion now has the additional "pad" option to output a full variable like "@{cool_variable}" instead of "cool_variable"
Quote conversion now offers "force" (always quote it, even if not necessary) and "double-up" (double quotes rather than using backslashes for escaping them) options
UnQuote conversion offers the "double-up" option like Quote as well, reversing the doubling up, of course
SpreadSheetToDouble/StringMatrix conversions now allow user to specify the range of rows/columns to turn into a matrix
data container writers (= transformers) now allow the specification on how to generate the file: AUTOMATIC, DATABASE_ID, ID, SUPPLIED current behavior was AUTOMATIC, which either uses DB-ID or if that is not available the ID of a container. With SUPPLIED you can create your own filename (without path) Affected: TimeseriesFileWriter, HeatmapFileWriter
EnterManyValues now has support for displaying a help via a tip text hint (when hovering over the field) and allows the output of a spreadsheet (default so far) and key value pairs

Additions

added support for control charts (adams-visualstats) c, u, np, p, Xbar R, Xbar S

https://en.wikipedia.org/wiki/Control_chart

http://www.qimacros.com/control-chart-formulas/
added a "System performance" menu item to the ADAMS main menu under "Help" (for checking performance of servers in cloud or local)

Have a good weekend!