Home Validate Volume Table Explorer Mars Image Explorer Resources  
Version 1.0.16 (May 5, 2014)
What's New
  • Updated data dictionary
  • Miscellaneous security fixes
Version 1.0.15 (November 23, 2013)
What's New
  • PDS3 Tools Library - Updated tools library to version 3.3.0
  • Miscellaneous security fixes
Version 1.0.14 (August 21, 2013)
What's New
  • Report Generation - Fixed report generation. (PTOOL-274)
Version 1.0.13 (August 2, 2013)
What's New
  • Security Updates - Miscellaneous security updates. (PTOOL-226, -230, -231, -232, -245)
Version 1.0.12 (July 19, 2013)
What's New
  • Contact Information - Updated contact information. (PTOOL-174)
  • Error Message Typos - Fixed misspellings in error messages. (PTOOL-163)
  • Security Fixes - Miscellaneous updates related to web site security. (PTOOL-208)
  • Updated to Current Dictionary - Dictionary is now on version 1.90. (PTOOL-209)
Version 1.0.11 (July 26, 2012)
What's New
  • Updated to Current Dictionary - Dictionary is now on version 1.87.
  • Security Fixes - Miscellaneous updates related to web site security.
Version 1.0.10 (August 24, 2010)
What's New
  • Re-validation Stacking Errors - Re-validation was not clearing out old errors.
  • Updated to Current Dictionary - Dictionary is now on version 1.80.
  • Updated Contact Info - All questions, concerns and bug reports should be sent to Rich Keller <rich.keller@nasa.gov> until otherwise notified.
Version 1.0.9 (July 15, 2010)
What's New
  • Housekeeping Updates - Reinstated cleaning out of temporary validation data after 24 hours. It should have no impact on users.
  • Fixed Circular Pointer Test - Circular pointer tests gave false positives when including the same file multiple times in a label. For example, including the same format file for multiple table definitions raised a problem and prevented the include. Now the test only checks for the included file being an ancestor of the current file.
Version 1.0.8 (June 28, 2010)
What's New
  • Fixed Set Values In Pointers - Fixed issue where pointer values containing multiple file references with no start position evaluated as only one file.
  • Updated Pointer Naming Convention Test - Updated test for file names of pointed to files. Instead of the file names needing to be the same, the pointed to file may merely start with the label name. For example FOO.LBL containing a pointer ^D0M_JPEG_DOCUMENT = {"FOO_PC1.JPG", FOO_WF2.JPG", "FOO_WF3.JPG"}
  • Misc Stability Fixes - Fixing issues related to database connection stability.
Version 1.0.7 (June 25, 2010)
What's New
  • Misc Stability Fixes - This is a small maintenance release to address stability issues arising from the previous major release.
Version 1.0.6 (June 11, 2010)
What's New
  • Display All Errors In Download - All validation problems are now sent to the server and are available in the textual report. Note that the problem display in the GUI may still be truncated. Also, this represents a non-trivial departure from previous implementations and performance may be improved or somewhat slower depending on volume characteristics.
  • Performance Improvement - Consolidated test logic and batching problem processing should speed up validation and reduce memory overhead. However, there is increased network traffic (with all problems being sent to server) and there may be some situations that database integration causes speed issues. If performance seems to be worse with this release, please contact me with details.
  • Corrected Volume Meta Info - Previous updates to the validator removed unnecessary files from display, however this was not accounted for in file, folder, and space display. This is now corrected when viewing the entire volume. This information is not available for sub-folders and has been removed.
  • False Negative In Argument Serialization - In special cases, some problems would have "null" arguments. The test for whether these arguments could be processed would give a false negative in this case.
  • Updated to Current Dictionary - Dictionary is now on version 1.79.
Version 1.0.5 (April 6, 2010)
What's New
  • Performance Improvement - In large volumes, the file listing in the results view caused browser performance issues or crashes. In some cases, the size of the serialized data caused server issues. In an effort to rectify this, only files with problems or new values are displayed. Note that this may suppress files that DO have issues if the problem was not captured due to overflow. If this represents an issue, a flag for whether to display all files will later be made available.
  • Corrected Exception Display - In certain cases, the display of exceptions could cause an exception itself, thus hiding the original issue.
  • Alias Changes In Dictionary Blocked Result Display - In local data dictionaries that have changes involving aliases, a serialization issue blocked the display of results.
  • Item Offset Support - In tabular data, ITEM_OFFSET was not being honored in column definitions.
  • Display Performance Improvement - Modified cache properties and reduced markup in an effort to reduce the load on the browser.
  • Updated to Current Dictionary - Dictionary is now on version 1.78.
  • Update Label Parser Library to 3.0.3 (in development, head version)
    • False Positive On Bad Line Endings - Fixed rare issue where lines containing only a CRLF preceded by a line ending with only an LF falsely reported a bad line ending.
Version 1.0.4 (March 19, 2010)
What's New
  • Title Updates - Changing the display format for titles and adding or correcting a few specific titles.
  • Validation Applet Cache Issue - Unless your cache was explicitly flushed and browser restarted, the wrong validation applet was often used. This has been updated to include the version in the filename so that cache issues across versions are no longer possible.
  • Moving Validation Statistics - Moved validation statistics into the Validate Volume tab. Credentials still required.
  • Added Improvement By Error Type Statistics - Added a new table to the existing validation statistics that shows what errors are and are not being fixed by type. Note that this is an expensive query and page load times may increase by 2 seconds or more.
  • Improved Numeric Handling in Statistics - Now able to handle numbers that exceed java's int boundary. It does not appear that any current statistics were impacted by this.
Version 1.0.3 (March 15, 2010)
What's New
  • Surfacing Analytics - Doing analysis on collected validation statistics and surfacing in a password protected web interface.
  • Update Label Parser Library to 3.0.2 (in development, head version)
    • Bad Attached Data Start Byte For Binary Data - The parser was sometimes reporting a false attached data start byte due to parser errors and recovery on binary data. The parser is now circumvented when finding the start byte.
    • Long Values Truncated For Display - When a value exceeds its length limit, only the first 64 chars are displayed in the error message rather than the whole string.
    • Fixed Support for Required/Optional Checks on Nested Objects - This change provides support for proper checking of required/optional nested objects when the nested object contains a descriptor in its name. An example is when a label contains an external FILE object and within this object is an image object named LO_IMAGE.
    • Support SPECIFIC_GROUP definitions - Library fixed to recognize SPECIFIC_GROUP definitions.
    • Improved Stability of Dictionary Parsing - Certain situations could cause the dictionary parser to exit abruptly. Errors are now captured as problems and parsing will continue past bad definitions.
Version 1.0.2 (February 24, 2010)
What's New
  • Windows Specific Local DD Issue - When validating a volume containing a local data dictionary on a Windows machine, an error would occur due to file separator conflict.
  • IMG Files Ignored As Possible Attached Labels - IMG files were treated as binary files and not tested as possible attached labels unless elsewhere specified as a label.
  • Index Validation On Cumulative Indexes - A naming convention for indexes included CUMINDEX.LBL. Files listed in the associated tabular file are no longer assumed present.
  • Errors In Parsing Index Tables Unhandled - When a column definition had a start or end byte past the end of a row, an exception was raised to the user but the index processing also exited early. It now reports a more specific error and continues processing.
  • Error Message Updates - Corrected and expanded on a few error messages.
  • Disabled Error Email Form - Due to hosting restrictions, emailing notices of exceptions was not possible. Now requesting user email the info directly. Note that information sent with this form may not have been received.
  • Added Exception Logging - Logging all exceptions to the database and encapsulating in web app functionality to view and manage.
  • Update Label Parser Library to 3.0.2 (in development, head version)
    • Handling for Large Attached Labels - Due to way our chosen parser generator, Antlr, works, large files would often cause memory issues. Attached labels with 50 meg or more of attached data would crash the parser. A custom input stream is now used to only allow Antlr to read to the end of the label portion of the file.
Version 1.0.1 (February 8, 2010)
What's New
  • Surface Table Parse Errors - When parsing the sample row of table data, surfacing errors like start or end byte for a column being past the end of the line.
  • Messaging Updates - Updating attached data start byte messages for clarity. Fixing inaccuracies in date related messages. Improved parse error messages.
  • Updating Validation Complete Redirect - On some systems, users report a page refresh rather than redirect on validation completion. Updated redirect process to use javascript instead of location change from inside the running applet.
  • Added About Page - Added a new page describing the validation process in more detail. This page includes a list of the problems the validator checks for.
  • Sticky Expand/Collapse of Results Sections - You can now expand and collapse Test Results, New Values, and Problems sections. This state is saved across sessions using cookies.
  • Troubleshooting Update - Minor updates to the troubleshooting page.
  • Updated to Current Dictionary - Dictionary is now on version 1.77.
  • Update Label Parser Library to 3.0.2
    • Fixed Attached Data Start Byte Test - Attached labels with multiple internal pointers reported the wrong data start byte and consequently surfaced an error about start bytes not matching.
    • Fixed Explicit Positive Integer Test - Integer values beginning with at "+" caused an error.
    • Updated Date Testing - Fixed an issue where times being out of range (such as 61 minutes) was not being tested. Fixed an issue where year only values were not validating for time fields. Fixed issue where no error was surfaced when numeric only (year) values were != 4 digits. Removed requirement that fractional seconds be 3 digits in length.
    • Fixed Failed Parsing Of Pointers - Malformed pointer statements were capable of causing a runtime exception that prevented the label from finishing parsing.
    • Fixed Pointer/Attribute Identifier Collision - Due to name collisions, some pointers were treated as attribute statements and an exception would occur during validation.
    • Alias Name Collision - Accidental override of dictionary definitions when an alias had the same id as an existing definition.
    • Group Validation Failure - Group definition lookups were failing.
    • SFDU Error - When SFDU headers were present, processing of the next line started at the end of the second SFDU rather than on the beginning of the next line. This cause the parser to think there was a blank line after the SFDU and before the PDS_VERSION_ID. Also, the parser always assumed the line ended with two newline characters. If your line was terminated with only one newline character, the first character of the second line would be ignored.
    • Description Pointers Not Satisfying Required Element - Required element checks have been updated to allow description pointers with the same name to satisfy the condition.
Version 1.0.0 (October 21, 2009)
What's New
  • Use of Label Parser Library - Using joint authored (with JPL) label parser library allowing for consistent validation where other PDS tools duplicate functionality.
  • Use of PDS Utils Library - Beginning of Java library to facilitate tool development.
  • Added Version Info - Added version info for label parser library and dictionary.
  • Removing Requirement for ERRATA.TXT - As per the standards ref, this is an optional file.
  • Reducing Parser Memory Consumption - Extremely large (16k+) labels were causing out of memory errors.
  • Value Cast Error - Checking that values are of the expected type for the definition.
  • Improved Line Length Check - Supporting blank fill after END and line padding for attached labels with fixed length.
  • Attached Label Pointer Checking - Verifying start byte is correct for internal pointers.
  • Object Validation Fix - Validating children even when parent definition not found.
  • Unit Validation - Doing basic unit validation. Currently this means that the value appears in the dictionary. Future development will verify that the unit is valid for the definition and that SI derived units are valid.
  • Improved Problem Messaging - Various improvements to problem messages.
  • Reporting NULL as Placeholder - Warning users that the NULL value is meant as a placeholder.
  • Supporting Multiple Local Data Dictionaries - Merging all ".ful" files in the document folder into the master dictionary.
  • Improved Test Coverage - Improved test coverage to meet 1.0 release standards.
  • Various Parsing Improvements - Numerous changes targeting more graceful handling of malformed statements and values.
  • Validating Non Decimals - Fixing validation of non decimals.
  • Various Improvements - Numerous minor fixes included in the development of the label parser library.
Version 0.2.04 (September 17, 2009)
What's New
  • Reporting Missing END, END_OBJECT, END_GROUP, and end quote - The parser now gracefully handles malformed statements that require a closing indicator. Note that this also addresses an issue where unclosed quotes in pointer statements resulted in an irrecoverable error.
  • Reporting Errors in Malformed Known Labels - The parser now continues to parse files that are known to be labels (by name or context) regardless of whether PDS_VERSION_ID appears at the top of the file.
  • Fixing Group Validation - An error in logic skipped the creation of group definitions from the dictionary and validation of groups was ignored.
  • Updated Index Validation to Allow Alternate Label Path - Now allowing PATH_NAME + FILE_NAME in addition to FILE_SPECIFICATION_NAME to describe the location of a label in an index.
  • Supporting Array Values in Tables - Updated table file validation to allow array values. Also increased granularity of error messages relating to tabular validation.
  • Fixed Real Number Validation - Fixed issue where trailing decimals were considered invalid.
  • Fixed Internal Pointer Issue - Fixed issue where pointers to attached content were looking for exterior files.
  • Updated Dictionary - Updated master dictionary to version 1.75 (released July 20).
Version 0.2.03 (August 31, 2009)
What's New
  • Fixed Report Display - Corrected downloadable report to no longer contain html formatting.
  • Fixed SOFTWARE Folder Validation - Fixed error in validation of required files under the SOFTWARE folder.
  • Allow Mismatched Case In Index - While not recommended, it is allowable for the path in index files to not match the case in the volume. Validator now finds these files and reports a warning that the cases do not match.
  • Fixed Malformed Pointer Issue - Fixed issue where a pointer value that was a set with only one value would cause an exception.
  • Added Dates to Version - Per request, added dates to release info.
Version 0.2.02 (August 6, 2009)
What's New
  • Fixed ASCII Real Number Test - Fixed test for real number format to allow explicitly positive exponents and the use of a lowercase "e". Previously a number like "1.2e+2" would have been considered invalid.
  • Fixed Label Fragment Validation - As included files do not contain PDS version information, they were not being validated against the PDS format. They are now treated as labels without the requirement for header information.
  • Removed Step Information - Due to refactoring, the volatility of number of steps involved in validation and forthcoming control over which validation steps occur, the total number of steps has been removed from progress information.
  • Improved Report Download Name - The file name of downloaded validation reports now matches the volume name.
  • Fixed Alternate Directory Check for Documents - The test for valid description pointers was not correctly retrieving the document directory when looking for an alternate location for the pointed to file.
  • Minor Performance Improvements - Some minor performance improvements were added including file extensions to NOT test for having attached labels. For instance, "zip" files and "jpg" files should not have attached labels or it would render them useless for tools that support those formats.
Version 0.2.01 (June 25, 2009)
What's New
  • Fixed Error Count Filter Issue - If a given type of error exceeded the max number captured (200) and the user selected a folder or file to filter on, then selected the root folder, that error type would only indicate 200 errors rather than the full count.
  • Fixed Dictionary Parsing Collisions - A few definitions had the same keyword but different definition types. For instance "SOFTWARE_PURPOSE" is defined as a Object and Element. This is now supported in the validation tool.
  • Fixed Applet Performance Issue with Zip Files On Windows - On windows, when choosing a volume to validate, any folder that you navigated to or appeared in your dropdown of available folders (such as your desktop) and contained zip files would cause a performance problem. These zip files were unpacked by the OS so they could be interpreted as folders. This functionality has been disabled due to the performance issue it created.
  • More Info During Validation - At user suggestion, the version of the tool and the path to the volume being validated now appear during the validation process.
  • Fixed Date Validation - Some ambiguity in date parsing was causing some dates to be validated against the wrong format. This only occurred with yyyy-DDD type formats.
  • Suppressing Validation of Binary Tables - Binary tables are not currently supported by the validator. Attempts to perform basic validation on this data always reported errors. Tables that are defined as having an interchange format of "BINARY" will now be ignored until binary support can be added.
  • Fixed Value Length Check On Quoted Values - Because of line padding, some quoted values would exceed their maximum length. The validator was previously changed to trim padding when performing length tests. However, newlines are required to be two characters, CRLF, so merely stripping padding did not reflect the original, unwrapped value length. Line endings are now being treated as one whitespace character.
  • Improved Exception Reporting During Validation - Prior to this update, all validation status updates were pushed from the applet to your web page. Furthermore, not all types of errors were reported. Now the page, using javascript, periodically polls the applet for its status and all recoverable exceptions are reported to the user. However, if your java process runs out of memory, it may not surface 100% of the time. Please see the Troubleshooting page if you experience difficulty.
  • Memory Management Improvement - Memory consumption for larger volumes is an issue. A few of the more problematic areas have been addressed, however, you may want to increase the memory allocated to your java plugin if you experience difficulty. See the Troubleshooting page for an example or contact me for details.
  • Repaired Error Report Page - The page that displayed application errors (hopefully you will have never seen this) had a problem with the form to submit a report.
  • Added Troubleshooting - Added a first cut of a troubleshooting document. The intent is to target problems running the validator or issues correctly interpreting results. If there is anything you would like to see added there, please let me know.
  • Updated to Current Dictionary - This actually happened previously in an interim update. It is being reported here merely because this is the next documented release following the update. Dictionary is on version 1.73.
Known Issues
  • Larger volumes may cause out of memory errors that may or may not surface to the user.
  • Has issues doing validation on a volume on a network drive using Windows. Can be fixed by mapping the drive or folder.
Version 0.2.00 (April 16, 2009)
What's New
  • Pointers now honor start byte or start record. Currently this only impacts index.tab and the basic tabular data validation
  • Validation of quoted values now ignores line padding when evaluating length
  • Addition of release notes
  • Added Resources section currently only containing a PDS "Home Page" template and implementation instructions
Known Issues
  • Has issues doing validation on a volume on a network drive using Windows
Version 0.1.00 (December 29, 2008)
What's New
  • Validation of an entire volume - initial release