Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog and this project adheres to Semantic Versioning.

But we group the changes as follows:

  • Breaking Changes when a change requires manual activity at update.
  • Bug Fixes when we fix broken functionality.
  • New Features for added functionality.
  • Improvements for other changed parts.

Table of Contents

[Unreleased]

Breaking Changes

  • Add compatibility for Java 8+ (10 included).
  • Add compatibility for Java 9+ for drag'n'drop. Due to technical limitations, the old way of TransferHandler#importData(JComponent, Transferable) needs to be called instead of the newer TransferHandler#importData(TransferSupport). Thus drag'n'drop implementations that use the newer API are not supported anymore. Actions with drag'n'drop need to be re-recorded.

Bug Fixes

  • Fix drag'n'drop actions were not loadable to replay those actions.

New Features

Improvements

  • Move the menu item "Ignored Components" to the review module (so it can be used with a review license).

[2.4.1] (2018-08-07)

Bug Fixes

  • Fix apply of recheck result.replay to update the sut state.
  • Fix showing of result.replay if file is directly opened with retest via start parameters.
  • Fix recheck not being able to open the gui.

New Features

  • Add gui command to cli so that one can pass parameters to start the gui (e.g. result.replay)

[2.4.0] (2018-07-19)

Breaking Changes

  • Remove dependency of persisted classes in result.replay files to Google Guava, which also hard-wired the dependency to a certain version thereof. This change means that result.replay files created with older versions of retest cannot be read anymore.

Improvements

  • Use XPath style for paths, i.e. /Component_0 is now /Component[1]. A migration strategy has been included (see retest migrate --help for usage).
  • Various internal changes to improve the user experience with the retest library JAR.

[2.3.2] (2018-07-05)

Bug Fixes

  • Fix obfuscation which removed some code from the retest library JAR.

[2.3.1] (2018-07-04)

Bug fixes

  • Fix various minor bugs.

Improvements

  • Adapt CI/CD pipeline to also create a small, obfuscated retest JAR, that can be more easily put on Maven Central.
  • Extract some missing code into open source retest-model and fix obfuscation of proprietary code accordingly.

[2.3.0] (2018-06-26)

Improvements

[2.2.0] (2018-06-05)

Breaking Changes

  • When replaying with multiple windows that contain the same element (e.g. a file menu), the correct window containing the recorded element will be clicked. However, you need to record the action again.

Bug Fixes

  • Handle remotes without GUIs (e.g. servers) via de.retest.remote.withoutGui. If the property is set to true within a generic launch script, retest will immediately return on the particular remote(s) when actions are broadcasted, but will still monitor it for dynamic waits.

New Features

  • Add SwingElement#findElements( By ) to the retest API to find contained elements that match a certain criterion.
  • There is now proper handling of external JARs that extent retest with custom components or otherwise custom functionality. You need to set the property de.retest.customization.clientJars and point it to a ; separated list of JARs that contain e.g. your custom TestContext implementation.
  • Actions, suites and execsuites can now be directly shown in their folder from inside the GUIs respective views. This allows for easy modification (rename, copy, delete).
  • Add status icon to dialog for converting suites so that one directly sees if the CSV file is valid or contains errors.

[2.1.1] (2018-04-27)

Bug Fixes

  • Fix multiple differences cannot be applied if they reference the same element. If, for instance, a button was inserted and, as a result, the following button was shifted, both couldn't be applied at once.
  • Fix ignored components not being loaded (after a restart) when using a custom context.

Improvements

  • When the execsuite to be updated (by clicking "Apply" in the report) has changed in the meantime, this led to an error message. Now, this will overwrite intermediate changes, giving more power to the user, but also more responsibility.

[2.1.0] (2018-04-04)

Breaking Changes

  • de.retest.defaultListener.blacklistComponentClasses is replaced by de.retest.listener.blacklist and now addresses all components.

Bug Fixes

  • Fix launch scripts which led to no output via CLI.
  • Internal bug fixes to improve and steady recording of action sequences.

New Features

  • Add possibility to use parameters in actions which can be used to convert one suite into multiple execsuites at once. For more information please take a look at the retest documentation.

Improvements

  • Enable more static instrumentation via -Xbootclasspath/p:, also for button listeners.
  • It is now possible to blacklist components from being listened to (and action recorded) via the property de.retest.listener.blacklist, which replaces de.retest.defaultListener.blacklistComponentClasses.

[2.0.2] (2018-03-26)

Breaking Changes

  • Fix NullPointerException if various components return null instead of text. This includes a fix of a trimming issue, which may causes differences but can be easily accepted in review.
  • Fix default component listener which sometimes was removed by accident, causing unknown components not being recorded anymore. If you are using -Xbootclasspath/p: to point to retest's statically instrumented java.awt.Component, you have to manually remove the class and then start your SUT via retest twice when you use this version for the first time.

Improvements

  • Handle client code that returns null as renderer for table cells or table header cells.

[2.0.1] (2018-03-15)

Bug Fixes

  • Fix IllegalStateException (No ElementDifferenceTreeNode for the given compDiff ... and No InsertedDeletedElementTreeNode for the given compDiff ...) in review.

Improvements

  • Increased the default timeout of de.retest.targetNotFound.waitingTime from 5s to 30s.

[2.0.0] Nim Chimpsky (2018-03-07)

Breaking Changes

  • The underlying data model is changed to allow the modification of recorded actions. This results in a non-compatibility with recordings from retest version 1.x. Migration requires manual steps, if you find yourself in that situation, please contact our support so we can help you.
  • The de.retest.equinox.hooks JAR is not needed anymore and, therefore, removed from the retest installation. Consequently, it can be removed from any SUT installation as well.
  • Drastically improved performance of loading and saving result.replays by using a binary format. As a consequence, old reports prior to retest 2.x cannot be opened anymore.
  • Ant and the internal Java API (e.g. TestReplayer) are replaced with the new CLI (see below) and are not supported anymore.

Bug Fixes

  • Fixed bug with replaying clicks with modifiers (Shift, Control, ...).

New Features

  • Add a preferences dialog to show all parameters, that are configured in the properties file.
  • New screen "Adaption Rules" allows to specify rules for inserting, skipping or changing actions in the BeanShell scripting language.
  • New tab in "Edit Action Sequence" allows to review details of the action target and action parameters as well as to edit them.
  • retest now offers a full-blown command line interface (CLI) for convert, replay, generate, update, and migrate. Just type retest --help (or -h) for more information.
  • Windows user are now able to use a native EXE file for starting the retest GUI, whereas the available batch script offers access to the CLI. *nix users can use the given executable for both GUI and CLI.
  • Pixel comparison of screenshots for components was already available via de.retest.swing.paintingComponents.packages property. Now, the resulting differences are also shown in review.
  • Show a dialog if retest started without a valid license.
  • Add a Selenium-style API for creating tests programmatically. Combined with the recheck API, this allows e.g. to create tests without knowing which data the test interacts with. And it is a convenient fallback, if a test needs to do programmatic stuff that can't be done from the GUI (if/else, loops). One can even mix recordings and code. See documentation for more details.
  • Add recheck API for programmatic difference testing. Creates individual files per check and can be updated like execsuites.
  • Add shortcuts for most buttons (New, Open, Save, Save as) inside the GUI.
  • If dynamic instrumentation of SUT bytecode is not possible or to costly, do static instrumentation via -Xbootclasspath/p:.

Improvements

  • Rollout of new corporate design (colors, logo, images).
  • Table clicks now display their modifier (Shift, Control, ...) in action sequences.
  • Dialogs showing screenshots are not modal anymore but scrollable and resizable.
  • Fonts are more reliable across different operating systems, because the font size, type, and family are now checked individually and their respective defaults are respected.
  • Improved table performance during replay by only taking table screenshots (instead of rows and cells additionally).
  • JScrollPane and JSplitPane are now more stable across platforms.
  • If the current target cannot be found, we poll every 500 ms for 5 s (both configurable via de.retest.targetNotFound.sleepTime and de.retest.targetNotFound.waitingTime) until the target eventually occurs. This makes for much more robust and fast test execution.

[1.5.5] (2017-10-11)

Bug Fixes

  • Fixed a NullPointerException in DragDropListener that occurred when the drop target was null.

[1.5.4] (2017-10-09)

Bug Fixes

  • Set default language to English instead of German.

[1.5.3] (2017-09-22)

Bug Fixes

  • Fixed java.lang.RuntimeException: Not implemented ... for MenuItems without label.

[1.5.2] (2017-09-20)

Bug Fixes

  • Fixed a java.lang.Error: Destination component not connected to component tree hierarchy that occurred in some rare cases.

[1.5.1] (2017-09-13)

Bug Fixes

  • Fixed a NullPointerException when replaying generated suites.
  • Fixed that the start-up script didn't allow spaces in the path.
  • Fixed a NullPointerException when a window has an empty title.

[1.5.0] (2017-09-05)

Bug Fixes

  • Fixed that identifying attributes were not ignored correctly (i.e. still appeared as difference in the report, although marked as ignored).
  • Fixed ClassCastException when executing a action with invalid target.
  • Fixed that in some cases, applying all changes did not make the exec suite green.
  • Fixed AssertionError triggered in de.retest.swing.tree.TreeRowResolver in rare cases.
  • Fixed NullPointerException and other smaller problems.

New Features

  • Add progress bar to review when loading/applying huge replay results.
  • Add indication to editing of suites and test if the referenced tests or actions are missing. The missing files should be replaced or, if not possible, the suite/test should be deleted as they cannot be used anymore. Please note that last mentioned can break even more suites and tests.
  • Add recording of shortcut combinations (e.g. Ctrl + P).

Improvements

  • Make GUI responsive when loading/applying huge replay results.

[1.4.1] (2017-07-18)

Bug Fixes

  • Fixed NullPointerException when recording expansion/collapsing actions on a tree.

Improvements

  • An empty code location (configured different PARENT_PACKAGE) and a missing code location (no javaagent configured) are now treated the same, resolving some special cases in which this difference led to a TargetNotFoundException.
  • retest tries to resolve situations in which it doesn't find the target automatically, e.g. by clicking on an 'ok'-button if a dialog is shown where this is the only option. However, to not run into endless-loops (e.g. if the dialog just reappears), it does so only 5 times. The number of attempts to resolve such situations is now configurable via the de.retest.swing.targetNotFoundMaxHandlingAttempts property.

[1.4.0] (2017-07-10)

Breaking Changes

  • Changed table headers to allow gathering of its column headers. Since additional components are found in screens with tables, they need to be accepted manually.
  • Add font, background color, foreground color and enabled-status as additional criteria per component, which may lead to additional changes when running tests. However, simply accepting the changes after update should suffice.
  • TargetNotFoundHandler renamed to ExecutionHook, which is more general and powerful (see New Features).
  • JComboBox has now children components which will result in additional components. Thus they need to be accepted manually.

Bug Fixes

  • Fixed that, in some cases, tab-clicks were not recorded.
  • Fixed recording of text modification after right click.
  • Fixed that when editing a table cell the wrong component is retrieved. This should have no effect on the current action sequences.
  • Fixed that the Accept all checkbox did not work correctly if differences were accepted from another difference.
  • Fixed that removing values from or adding values to a JComboBox results in the wrong item being selected (was solely based on row index).

New Features

  • Drag'n'Drop is now supported if executed inside the program. Drag'n'Drop from/to the outside (i.e. operating system or other programs) is not yet supported.
    • Fully supported: Text components (e.g. JTextField) if the drop action modifies the text property.
    • Partially supported: JList, JTable and JTree. While the Drag'n'Drop should be recorded, there may be problems while replaying:
      • The drop action might not be replayed properly. A current not working example is dragging onto a table cell that is being edited. Since the insertion between two characters of the dragged text is tied to the cursor location of the text field, the Drag'n'Drop will always insert the text at the end. This is because the Drag'n'Drop action does not update the component dropped on.
      • The initial drag action is not replayed. This non critical if the object is copied.
      • If one of the above is causing problems, the component in question needs a custom implementation. In this case please contact support.
  • Components can be now analyzed whether they do custom painting and, if so, the screenshots of the images is compared. Two comparison implementations are available via the de.retest.image.DifferenceCalculator property: ExactImageDifferenceCalculator and FuzzyImageDifferenceCalculator. To enable this comparison, please use the property de.retest.swing.paintingComponents.packages to define packages of which painting components should the screenshot be taken off.
  • Added ability to convert suites to execsuites via ant.
  • Clicking on unknown components is now supported. Please note that this feature should only be used rarely since it is quite unstable to changes. If you need a component to be supported in a more stable way (because you need it often), please contact the support. Since not all components support a listener, some components may become not clickable anymore. We are currently creating a fix for this. As a temporary fix, please use de.retest.defaultListener.blacklistComponentClasses to add components that will be ignored by the default listener.
  • Added a loading feature so Tests and Suites can be edited.
  • Inserted and skipped actions are now represented in the test report and shown in the GUI. However, they cannot yet be 'approved' to be permanently added to the test.
  • Instead of a TargetNotFoundHandler, we now have a more powerful ExecutionHook, which can insert or skip arbitrary actions (not just in case of a TargetNotFoundException). The property de.retest.maxInsertedActions (default 5) is used to limit the number of inserted actions, in case the user config results in an endless loop.

Improvements

  • Removed tree selection listeners. Clicking on a tree is now only one action.
  • Now using TreeTable in the GUI for creating tests, suites, converting action sequences and in the review module to visualize the structure and relation of action sequences, tests and suites.
  • Add count of number of rows of tables, trees and lists if not all children are tracked in detail (i.e. if their number exceeds de.retest.maxChildComponents).
  • Show error message if a report cannot be loaded into ReView.
  • Improved the file path to be relative to the retest-Workspace. This is for example used in the JFileChooser.
  • The selectable values in a JComboBox are now analyzed properly as children. Thus removing values should now be recognized.
  • Improve migration by automatically compressing and deleting temporary files. Moreover, migration is now done in parallel.

[1.3.5] (2017-06-27)

Bug Fixes

  • Fixed an arguable bug, where certain operations outside the event dispatch thread caused a java.lang.Error.

Improvements

  • Updated the demo license.

[1.3.4] (2017-06-27)

Bug Fixes

  • Fixed an arguable bug, where a NullPointerException was thrown, if the implementation of the TableCellEditor returned null for a call to getTableCellEditorComponent(javax.swing.JTable, java.lang.Object, int row, int column) – which according to specification it shouldn't...

[1.3.3] (2017-05-30)

Bug Fixes

  • Fixed an arguable bug, where a NullPointerException was thrown, if the implementation of the JTable returned null for a call to getCellEditor(int row, int column) - which according to specification it shouldn't...

[1.3.2] (2017-05-15)

Bug Fixes

  • Fixed a bug, where the de.retest.maxChildComponents parameter was influencing the number of table rows that were counted.

[1.3.1] (2017-05-13)

Bug Fixes

  • Fixed replay of rightclick for tables where the renderer / editor is not a simple component, but contains children.
  • Two minor bugfixes for rare cases.

[1.3.0] (2017-05-02)

Breaking Changes

  • Changed trees to allow for complex layouts. This requires all actions with trees to be recorded and converted again.

Bug Fixes

  • Fixed recording of clicks on JList.
  • Fixed clicks on JTree being recorded twice.
  • Fixed clicks on JCheckBox in tables being recorded twice.
  • Fixed recording of double clicks on JTextfield.

New Features

  • It now is possible to ignore individual attributes of individual components.
  • Surili now also tries to insert values from the Big List of Naughty Strings.
  • Right click on tables, trees, table headers, labels and text fields can be recorded and replayed.
  • Added a new option that removes the signature of all .jar when retest launches. This is because if the application has signed .jar its launch will take longer. To use this feature it must be toggled with de.retest.checkAndRemoveJarSignature=true. However this causes a short delay when retest is started. It is recommended that this option should only be used if necessary.
  • Add property to ignore threads while waiting for stabilization.

Improvements

  • Improved user feedback when retest could not find a component on a dialog (TargetNotFoundException).
  • Increased default for de.retest.maxChildComponents to 1,500 child components to tackle medium-sized lists, tables, and trees.
  • Better logging during update process when there're missing file permissions.

[1.2.3] (2017-04-19)

Bug Fixes

  • Fixed a bug where in some special cases modal dialogs where not recognized.

Improvements

  • Ignore Java2D Queue Flusher active waiting thread.

[1.2.2] (2017-04-10)

Bug Fixes

  • Fixed obfuscation preferences where an MalformedParameterizedTypeException was thrown.

[1.2.1] (2017-04-03)

Bug Fixes

  • Update demo license.

[1.2.0] (2017-03-28)

Bug Fixes

  • A DefaultUncaughtExceptionHandler no longer keeps retest from terminating the SUT properly.
  • Fixed IllegalArgumentException: Only directories are allowed! that occurred when Surili opened a JFileChooser.
  • Fixed NullPointerException for tables without table headers.

Improvements

  • A dialog will be shown when saving or loading files fails.
  • Added menu hover to improve self explanatory of clickable content.
  • Added support button for direct access to retest's service desk.
  • HSQLDB is now supported by the ThreadMonitor.
  • Added property de.retest.instrumentation.shouldInstrument to allow to completely skip instrumentation, along with a warning if used. This is intended only for debugging purposes.
  • Support invisible java.awt.Components and provide a stub for easy extension.
  • Show English descriptions for screens.

[1.1.1] (2017-03-15)

Bug Fixes

  • In special situations where the java.util.logging framework is used, the SUT could only be started once.

[1.1.0] (2017-03-08)

Breaking Changes

  • Removing de as the language in the default config, possibly making it necessary to set it, if so preferred.
  • Differences on the file system are now completely ignored. As a result, actions via JFileChooser must be rerecorded.

Bug Fixes

  • Properly mark screenshots of differences for nested windows.
  • Inactive windows due to modal dialogs are now correctly ignored by the monkey.
  • Fixed automatic invocation of (general and specific) suite hooks for action sequences within ReCapture.
  • Fixed IOException thrown when proxy settings were used for local addresses.
  • Fixed IllegalStateException if a replay result was opened where an additional window was shown in the SUT.
  • Fixed a bug which shadowed child differences in a few situations. Hence, former hidden differences may be visible now.
  • Various minor bug fixes.

New Features

  • Now all changes of an action can be accepted at once via an "Accept all" checkbox in the table header.
  • Now the (always up-to-date) documentation in the web is linked in the HTML descriptions.
  • Partly English translation of the GUI (without the HTML description).

Improvements

  • The screen to edit actions is polished.
  • Black- and whitelisting is now on the same page of the monkey wizard, making it harder to confuse them or create invalid configurations.
  • Errors and warnings now occur within the startup shell, which makes it a lot easier to detect issues like invalid licenses.
  • Randomly missing Popup$HeavyWeightWindows no longer cause differences.

[1.0.0] Ham - The First One (2017-02-09)

Bug Fixes

  • It is not possible anymore to apply the same changes twice to an executable suite.
  • Fixed encoding, various typos, and formatting issues of the HTML description in the GUI.
  • Fixed regression with ignored proxy settings.

New Features

  • After applying changes to executable suites and/or ignored components, a popup shows which files are affected and reminds to commit them.
  • Adding the possibility to have the monkey test UI components exclusively (whitelisting).
  • Adding a screen to view the currently ignored components.
  • After generating tests with Surili, the best suite is now exported as an executable suite.
  • The color markings of selected components is now different for ignoring (grey), blacklisting (red) and whitelisting (green).
  • In ReView, there is now a table on the level of the action that shows all differences of that action and allows to accept them right there.

Improvements

  • Showing report summary when initially opening a replay.result file, summarizing execution time, number of suites, tests, actions, number of differences and errors and such.
  • Behavior in regard to ignored.components is now consistent with updating execution suites - changes to ignored components are only applied when someone presses Apply Changes.
  • Update the inline documentation on the right side.
  • Add extended online documentation at https://retest.de/docs/de/.
  • Several improvements in demo application.

[0.12.0] (2017-01-30)

Breaking Changes

  • Changed application packaging, the complete code is now in one fat JAR. Paths to classpath and javaagent in startup scripts or build.xml must be adjusted regarding to this change.

Bug Fixes

  • A lot of bug fixes in Surili module.

New Features

  • Added a demo application as show case.

[0.10.2] (2017-01-26)

Bug Fixes

  • Fixed automatic invocation of specific suite hooks within ReCapture.

[0.10.1] (2017-01-05)

Bug Fixes

  • Fixed two different OutOfMemoryError leaks, triggered if one opens too many GUI screenshot popups.
  • Fixed IndexOutOfBoundsException if same changes were applied to same execsuite multiple times.
  • Corrected displaying of error in details view (was stuffed into one line).

[0.10.0] (2016-12-22)

Breaking Changes

  • Proxy settings changed, please take a look in retest-template.properties.
  • For lists, tables, and trees, at most 500 child components are being recorded in order to improve overall performance and file sizes. This value can be adjusted via the property de.retest.maxChildComponents.

Bug Fixes

  • Fixed race condition between clean up task and (external) report creator.
  • Fixed IllegalStateException "No state AND no stateDifference is illegal!" in ReView.
  • Huge components no longer cause differences per se.
  • Fixed several corner cases regarding path differences, e.g. sometimes the wrong table cell was clicked.
  • RePlay no longer allows starting multiple SUTs by accident.
  • Closing a window is now recorded again.
  • Fix for OSGI applications, where the bundle de.retest.equinox.hooks was not found.
  • Fixed a bug, where the folder latest was not cleared from XML files.

Improvements

  • Stabilized and secured outgoing connections, especially with proxy setups.
  • Better error messages for configuration errors.

[0.9.0] (2016-11-17)

Breaking Changes

  • The license file must be moved to retest workspace with name retest.license, or specified with system property de.retest.license.File.
  • The classes referenced in Ant build.xml have changed, please inherit from new build.xml in the release.
  • The sut-javaagent is now placed directly in install directory with fix name retest-sut-javaagent.jar.
  • The equinox-hooks-fragment is now placed directly in install directory with fix name retest-equinox-hooks-fragment.jar.

Bug Fixes

  • Fixed OutOfMemoryError in ReView.
  • Removed old update folders.
  • Added support for combo chooser again.
  • Surili now excludes submenus if the corresponding menu or menubar is excluded as well.
  • Fixed bug where table cells where wrongly associated to one another.
  • Fixed bug where JFileChooser was not recorded and replayed correctly.
  • A lot of bugfixes for Surili Ant task.

New Features

  • Write a live test anything protocol (TAP) after each test execution. This can be displayed within the build server.
  • Made Screenshots resizable in report.
  • Let Surili optionally start at the end of an ExecSuite instead of anywhere.
  • Limit number of test reports via de.retest.cleanUp.maxFolders (default 10) and/or de.retest.cleanUp.maxDirectorySizeInMByte (default 1GB).
  • Added support for editable combo box.
  • Test reports are created when Surili is executed.

Improvements

  • Simplifyed path serialization to reduce file size.
  • Made screenshots clickable in report.
  • Added summary of changes in actions details view within ReView.
  • Added support for disabled Tabs.