Release notes 7.0.2

Fixes

ARIMAX: Name of fields was incorrectly displayed in "Model coefficients" output port when using normalizer with reference category (for example,Indicator). The error was corrected (#9693).

Megaladata Desktop: When loading the application window selecting its maximum size, the window title was beyond the upper edge of the screen because of the error in the external library. To avoid this situation, the application is always launched in the normal window mode.

Megaladata Desktop: The second and subsequent instances of the application were launched with some delay. The situation was rectified.

Megaladata Integrator: The following error was corrected: "Object reference not set to an instance of an object". It occurred if Rows array that contained the elemenet equal to null was tranferred to Integrator via REST request to the Table port (#9795).

Megaladata Integrator: The Date/Time parameter value was incorrectly transferred from Megaladata Integrator to Megaladata Server if the time zone was specified in the request using Z postfix. For example, for GET request with the value of 2015-07-31T18:30:00Z parameter for UTC+03:00 as a response 2015-07-31T18:30:00 value instead of the expected 2015-07-31T21:30:00 was received (#9650).

Python: When preparing the output table by pandas.DataFrame columns, the "Variable" data kind was received whereas the data kind must be initialized according to the data type. The error was corrected (#9775).

Python: The values related to the integer type were incorrectly recorded in the output column related to the variant type when executing the code in a separate process. When executing the similar code inside the Megaladata process, the user received the following message: "MegaladataError: Transformation of the passed Python object into the variable type value is not supported". The errors were corrected (#9692, #9689).

Workflow: When hovering a cursor over the Notes selection element, it is enclosed in the frame (#9718).

Chart visualizer: Series were displayed in reverse order for the column chart in the accumulation mode. The error was observed in the packages prepared in the earlier Megaladata versions that were also opened in Megaladata 7.0 and higher. Corrections were introduced (#9244).

Data Quality visualizer: The word "Spaces" was displayed instead of values in the histogram of categorical data in the Caption field. The problem was observed not for all text fields and not in all versions. The error was corrected (#9772).

Data Quality visualizer: The error in detailing of the data for outliers in the fields with continuous kind of values was corrected. Previously, when there were both outliers and extreme values among the values of the field with the continuous data, also extreme values could get into detailing of outliers in the list (#9780).

Data Quality visualizer: Values of parameters were not updated when changing data set. The situation was rectified (#9227, #9680).

Binary сlassification assessment visualizer: Construction of NPV (Negative Predictive Value) and OPR (Overall Precision Rate) profiles was corrected for borderline cases: for NPV when there were only "events", and for OPR when there were only "events" or only "non-events".

Cube visualizer: Access violation error was corrected. It occurred when adding many fields to the area of dimensions after receiving "Too many dimensions selected" message and when trying to expand the dimension hereafter (#9238).

Cube visualizer: When many fields are sequentially added to the area of the Cube dimension, the user can receive the following message: "Too many dimensions selected". Then the last field added to the dimension area moved to the list of fields but in this case it looked like as the field that was located in the area of dimensions (it was highlighted in bold). The erroneous cancellation of field dragging was fixed. The field remains in the dimension list after the following warning: "Too many dimensions selected" (#9655).

Cube visualizer: When sorting in the Cube by "Percent by column" and "Percent by rows" views, in some cases order of cells could be incorrect. The situation was rectified (#9468).

Cube visualizer: If the user deactivated the node on which the Cube was configured that was in "Defer layout update" mode, and then the Cube was activated again, and the dimension filter was opened, the user could see an empty or incorrect list of dimensions in the filter window, or Access violation error could occur. Error of behavior was corrected (#9660).

Cube visualizer: The value of 9223372036854775807 measure when using Maximum aggregation was incorrectly displayed as -1. The situation was rectified (#9779).

Cube visualizer: When formatting values of the dimensions related to the integer type, large values (exceeding 9007199254740991) were displayed with accuracy loss. The error was corrected (#9778).

Cube visualizer: When testing the format, the package locale was not considered in the Formatting window. The situation was rectified (#9777).

Cube visualizer: The fields previously used as measures and hidden in the cross table were not considered in Configure dimensions window as available fields that could be added to dimensions. The error was corrected (#9727).

Statistics visualizer: If "Count order statistics" process was cancelled after 50% of execution, the button that started this process remained in an inactive state. Error of behavior was corrected (#8662).

Statistics visualizer: The error in the algorithm of median, upper and lower quartiles calculation was corrected. When calculating the order statistics index, off-by-one error error was made. Calculations for the data sets that included more than 1000 rows were erroneous (#9786).

Statistics visualizer: Display of the caption value for the continuous field in the visualizer histogram was corrected. If the package locale was changed and the histogram was displayed for the countinuous Real or Date/Time field, it was observed that the value was displayed in the default locale but not in the one selected for the package.

Table visualizer: After configuration of formatting for the column with integers, the numbers exceeding 2^57 were displayed in the exponential format. Error of behavior was corrected (#6528).

Table visualizer: Unnecessary data preparation was removed in the case of repeated activation of the Table deactivated when sorting or filtering (#9702).

Visualizers (Table, Statistics and Chart): Different errors were corrected, including Access violation error. They occurred if the node was deactivated while performing asynchronous computations in a separate stream in the visualizer. For example, errors occurred if the node with large data set was deactivated on the workflow when the visualizer configured on this node was opened in a separate tab:

• Table in which data filtering and sorting were started.
• Statistics in which calculation of the order statistics and unique values started using the toolbar command was performed.
• Chart in which the new field was dragged to the area of chart construction (#5766).

Visualizer: Different errors occurred, including Access violation errors when deactivating the visualizer with the open detailing when the detailing visualizer was activated (#9746).

Node execution: Access violation error was corrected. It occurred in case of the simultaneous configuration restoration of two Node execution nodes for one workflow. Such a situation could arise, for example, if two Execution nodes in which "Save configuration of the selected node" checkbox was deselected, executed one and the same Supernode and their output ports were connected to the input of one and the same Union node. Error probability depended heavily on operating system and hardware (#9563).

Node execution, Loop: Access violation error was corrected. It occurred when deleting Execution and Loop nodes with open property inspector (#9751).

Grouping, Table to variables, Statistics visualizer: Access violation error was corrected. It occurred when calculating median for the field the values of which were such that no value got in range [mean — root-mean-square error; mean + root-mean-square error]. Such situation is possible, for example, in connection with accuracy loss when calculating root-mean-square deviation (#8721, #9774).

Tree to table: The wizard ignored selection of the node that was an array but not a container. After introduced changes the corrected array node that is not a container and the "Global node index" can be selected separately (namely, selection checkbox of one of these nodes doesn't have influence on selection checkbox of the other one). The wizard saves the node selection checkbox that is an array but not a container, and this checkbox is not reset when setup is completed (#9263).

Replace: When adding the new row to the replacement table, the newly added replacement was not displayed. To display the data, it was required to switch to another field and then return back. Error of behavior was corrected (#9675).

Replace: The following error was corrected: "Cannot read properties of null (reading '1')". It could occur when changing the width of "Replacement method" column in the wizard. The error occurred seldom. In some cases when changing the width of "Replacement Method" column, the wizard window was completely blocked. The error was corrected (#9453).

Replace: The replacement table columns changed their sizes when adding many replacements, when scrolling the list of fields (if the list of fields was long), when "Sort" button was pressed multiple times if there were more than 2 exact match fields. Error of behavior was corrected (#6139).

Import from database: The line break characters in the SQL query text were deleted in the SQL query editor of the import from DB wizard in the case of repeated opening of the wizard. In the result, the node start was finished with the request to DB execution error. The regression error occurred in version 7.0.0 was corrected (#9652).

Import from database: Access violation error was corrected. It occurred when defining the control variable usage type of Import from database node inside the Loop (#8685, #9467).

Import from XML file: The following error was corrected: "It is not possible to define the sibling corresponding the schema". It occurred when importing XML with optional groups of elements. If the XSD schema contained the choice group with optional elements (with minOccurs="0") but the group as such was not marked as the optional one, the xml that contained null data of such group returned an error when importing (#9643).

Import from XML file: If the XML document contained the elements that were marked as any in the schema, the following error occurred while importing: "Maximum number of "/person{}/any" element occurrences exceeded (0 allowed)". The error appeared in version 7.0.0. The required corrections were introduced (#9679).

Import from XML file: Import of Date/Time without offset didn't work if the time zone offset was set by default and also import of Date/Time with offset. The situation was rectified (#9748).

Import from Text file: Numbers up to 12 inclusively changed places with months when specifying yyyy/mm/dd date format in the wizard (#9676).

Binning: Incorrect generation of binning range captions when using substring from the field caption in Cyrillic characters was corrected. If for the data set in which the field captions were set in the another language in Binning node at the stage of binning components configuration the template of bin captions containing FD key (for example, %FD[5] — bin caption contains the first five characters of the field caption) was set, another substring was at the node output in the bin caption column instead of the expected characters from the field caption. The error appeared in version 7.0.0 (#9719).

Neural network (classification): Attempt to retrain the node in the repeatedly opened package with the configured neural network resulted in the following error message: "The validation data set must contain at least one row". But the validation set size in the wizard was equal to 0. Such problem could be faced when retraining the cloned Neural network (classification) node. Decrease of the validation set size after loading of the neural network configuration was corrected (#9739).

MS Access connection, ODBC connection: Data loss was eliminated. It occurred when exporting the string parameters that contained Unicode characters to the database, this data loss appeared in version 7.0.0 (#9669, #9668).

SOAP service connection: The connections in which WSDL source was specified via HTTP protocol and WSDL file contained imports with the relative location of files didn't work: <xsd:import schemaLocation="path1/xsd0.xsd" namespace="..."/>. The error was corrected (#9768).

Supernode: Access violation error was corrected. It occurred when executing the package using Megaladata Integrator if the metadata didn't match the current package version. For example, the error could occur if the package (Package1.mgp) was published, and then several output ports to which data came were added for the published Supernode, and then the package was saved under another name using "Save as" menu (Package2.mgp). Then Package1.mgp was deleted and Package2.mgp was renamed to Package1.mgp and it was executed using Megaladata Integrator (#9584).

Application: Error in the function of Date/Time to row transformation function was corrected. Only time was displayed in the row for 30.12.1899 after introduced changes, and when transforming to OLE Automation Date, 0 is obtained in the integer part. Changes affected the Quick view, previews in the import components, Calculator and Row filter and also all editors of values for the Date/Time and variable data types (configuration of variables, Row filter, Replace components, etc.) (#9715).

Loop: When executing the loop related to the Group processing kind with many groups that was configured to the node for which statistics was calculated by the input data set (for example, Eliminate outliers ), workflow was locked by the mask with "Loading" text for a long time. The mask remained on the workflow even when the loop completed its operation. The application remained operational. It was possible to go to another section (using the address line or menu). Then locking of workflows disappeared. The problem was solved. When executing the loop, workflow is not blocked (#9796).

Export to Excel file: 12.30.1899 and 12.31.1899 dates were exported to Excel in the form of the number with the integer part equal to 0. In case of further import of this data to Megaladata, both dates were as follows: 12.31.1899. The error was corrected (#9763).

Export to Excel file: Dates were incorrectly displayed in the file that was created in the result of data export from Megaladata to Excel (the order of days and months didn't comply with the format). The problem was observed in the packages prepared in the earlier versions and opened in Megaladata 7.0.1. The error was corrected (#9694).

Export to Excel file: The following error was corrected: "Excel sheet name cannot be longer than 31 characters". It occurred in the export to Excel wizard if the row was longer than 17 characters in the "Sheet name" field (#9665).

Export to XML file: "Specify time zone" setting was not operable. According to the selected configuration option of "Specify time zone" setting, operation was performed as if "Do not specify" option was selected. The situation was rectified (#9758).

Export to database:/ The following error was corrected: "Invalid precision value". It occurred while attempting to export the tables with the text field longer than 255 characters to MS Access (#9500).

Improvements

Megaladata Integrator: Now UnixSocket field is located higher than host field in Megaladata Integrator installer on the setting page. If the set host field value differs from localhost or 127.0.0.1, and if Unix Socket is equal to the default value, Unix Socket field is cleared (#9657).

Megaladata Server: After the Megaladata server stop in the case of Linux OS use with the active connection via web socket, its restart was possible after ~60 s. Required improvements were made to provide restart of the Megaladata service in Linux without prior waiting, including in the automatic mode(#9691).

Workflow: Positions and size of the node comment block are saved in the case of its moving, changing its size, opening and closing, and also saving the package (#335).

Cube visualizer: The Cube reconstruction time was reduced, delay when adding/deleting dimension or measure in "Defer layout update" mode was removed, caching of measures became selective(unused measures are not cached) (#6763, #6816, #7139).

Table visualizer Data loading in the visualizer after pressing on "Cancel" button in the Formatting window was optimized. Previously, after cancellation of actions in the Formatting window, the Table was updated with a delay (#8051).

Visualizers: Large integers (exceeding 2^53-1) are displayed without accuracy loss (#7953).

Encrypted package: If there are problems with decryption of the encrypted package, the user can get the following message: "Invalid password". The message was replaced with the following more informative message: "The current application version cannot decrypt package...: unsupported method of package encryption" (#7834).

Import from text file: Date parsing in ISO 8601 format with time offset is now supported. If only time is specified in the imported data (12.30.1899) with offset specified relative to UTC, offset in the application time zone is applied when transforming to the Date/Time type, and also the history of time conversion is taken into account (#9747).

Import from text file: Support of the multiline values import was added. When importing values from the text file, including rows, they can be put in quotation marks. If the odd number of quotation marks or other line terminators is detected inside the row, the rows will be further read until the matching quotation mark is found, and they will be combined into one. Line break characters inside the quotation marks will be included into the total row. The number of quotation marks in the row is calculated in the way that provides parsing of poorly generated user rows as correctly as possible (as in Excel). If it is not possible to find the matching quotation mark, operation of import from text file will be continued and warning will be recorded in the shared log (#100).

Import from text file: Parsing of the dates in which the month is recorded both in the nominative, and in genetive cases is now supported. Date parsing algorithm was optimized. It had a positive effect on performance. In some cases dates are parsed by 10 times faster (#9627).

Calculator: "Template" of the following functions was simplified: Find, Replace, Round. It is set when they are pasted to the Calculator expression:

• Find: the comma was deleted. It separated the last optional parameterOrder because it was rarely used.
• Round: the comma was deleted. It separated the last optional parameter Argument2 (the number of digits after the comma) because it was infrequently used.
• Replace: 2 commas were deleted, by default Replace_all and Ignore_case parameters were not to be specified.

Calculator: StrZero function became dependent on locale. In the case of the non-standard use of the function with the real numbers (the standard use — with the integer argument), the decimal separator will be taken from the package locale settings or from the Megaladata server.

Calculator: Work of the following functions was optimised: Left, Right, Substr, Stuff (#9724).

SQLite connection: The name of the file for database in memory can contain any characters with the exception of the following one: # ? ; ' ". Previously, in memory DB identifier could consist of the following characters: the Latin alphabeth characters, numbers and characters / _ . (#9688).

XSD Schemas Set Connection: Import of the schemas is supported.schemaLocation and namespace in them are in Cyrillic characters. Previously, parsing was terminated when non ASCII character was detected, and if the whole row was not parsed till the end, it was finished with an error (#9651).

Application: Transformation of some real numbers to rows is performed with accuracy loss. It is connected with peculiarities of the fractional decimal numbers holding in the binary number system. Improvements were made to provide transformation of the real decimal numbers to rows with less accuracy loss. The changes affected the operation of the following components: Export to text file, Export to Excel, Calculator, etc.

Application: When setting the value of the variable or the field related to the variable type (for example, when setting condition in Row filter), it became impossible to select the variable data type for this value, because it was possible to specify data type for the set value explicitly (#9357).

Export to XML file, XML Generation, SOAP request: When transforming Date/Time to the XML row, not fixed time offset is used but the time offset with the historical time amendments taken into account, namely, summer/winter time conversions (#9755).

Filter/Search element: When searching using Filter/Search element, empty cells (the cells that contain only nonbreaking space) are skipped.