diff --git a/README b/README index 5e8629c11a7814f8616f0bbf4ee8b3c2b07ddd4f..bdd0179dd965b188353332db8e73fc882f720e64 100644 --- a/README +++ b/README @@ -141,7 +141,7 @@ SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. Change history -------------- -Version 2.23c (11/03/2016) +Version 2.24a (11/16/2016) - Added entry_points to setup.py so that distutils will generate executable scripts for executable .py files (for example, @@ -150,12 +150,20 @@ Version 2.23c (11/03/2016) - Fixed function call signature mismatch in MixedContainer call to export method. Thanks to Lev Israel for catching this and providing the solution. -- Fix to catch duplicate definitions of child elements with the same - name inside a single parent element. This fix does the following: - (1) removes duplicate child; (2) makes the remaining child a - Python list (effectively maxOccurs="unbounded"); (3) prints a - warning message when it finds and removes a duplicate. Thanks to - Pietro Saccardi for catching and reporting this. +- Added "remove duplicate elements" fix to catch duplicate + definitions of child elements with the same name inside a single + parent element. This fix does the following: (1) removes + duplicate child; (2) makes the remaining child a Python list + (effectively maxOccurs="unbounded"); (3) prints a warning message + when it finds and removes a duplicate. Thanks to Pietro Saccardi + for catching and reporting this. +- More fixes for "remove duplicate elements". +- Removed command line option for "remove duplicate elements". This + behavior will now always be performed. +- Added unit test for "remove duplicate elements". +- Added command line option "--no-warnings" to turn off warning + messages. I needed it for the unit test for "remove duplicate + elements". Version 2.23b (09/26/2016) diff --git a/generateDS.html b/generateDS.html index e58fca60076fd676e61676a62f233b81d4922c22..5312fd5c4bf2b9fb84f840d66e6a2e9c022bbfb9 100644 --- a/generateDS.html +++ b/generateDS.html @@ -220,7 +220,7 @@ They are used by updateversion.py. --> <col class="field-name" /> <col class="field-body" /> <tbody valign="top"> -<tr class="field"><th class="field-name">revision:</th><td class="field-body">2.23c</td> +<tr class="field"><th class="field-name">revision:</th><td class="field-body">2.24a</td> </tr> </tbody> </table> @@ -229,7 +229,7 @@ They are used by updateversion.py. --> <col class="field-name" /> <col class="field-body" /> <tbody valign="top"> -<tr class="field"><th class="field-name">date:</th><td class="field-body">October 13, 2016</td> +<tr class="field"><th class="field-name">date:</th><td class="field-body">November 16, 2016</td> </tr> </tbody> </table> @@ -531,7 +531,7 @@ python generateDS.py -o people.py -s peoplesubs.py people.xsd <p>Here is the usage message displayed by <tt class="docutils literal">generateDS.py</tt>:</p> <pre class="literal-block"> Synopsis: - Generate Python classes from XML Schema definition. + Generate Python classes from XML schema definition. Input is read from in_xsd_file or, if "-" (dash) arg, from stdin. Output is written to files named in "-o" and "-s" options. Usage: @@ -561,7 +561,7 @@ Options: top level class; if YY omitted XX is the default. class. Also see section "Recognizing the top level element" in the documentation. - --super="XXX" Super module name in generated in subclass + --super="XXX" Super module name in generated subclass module. Default="???" --validator-bodies=path Path to a directory containing files that provide bodies (implementations) of validator methods. @@ -586,10 +586,10 @@ Options: generated files. This is useful if you want to minimize the amount of (no-operation) changes to the generated python code. - --no-process-includes Do not process included XML Schema files. By + --no-process-includes Do not process included XML schema files. By default, generateDS.py will insert content from files referenced by <include ... /> - elements into the XML Schema to be processed. + elements into the XML schema to be processed. --silence Normally, the code generated with generateDS echoes the information being parsed. To prevent the echo from occurring, use the --silence switch. @@ -630,6 +630,7 @@ Options: Default: "[('[-:.]', '_')]" -q, --no-questions Do not ask questions, for example, force overwrite. + --no-warnings Do not print warning messages. --session=mysession.session Load and use options from session file. You can create session file in generateds_gui.py. Or, @@ -894,6 +895,9 @@ option is omitted and the ouput file exists, then generateDS.py will not ask whether the file should be overwritten. (In this case, when "-q" is used, the "-f" must be used to force the output file to be written.</dd> +<dt>no-warnings</dt> +<dd>While running generateDS.py, do not print warning messages that +would be written to stderr.</dd> <dt>session=mysession.session</dt> <dd>Load and use options from session file. You can create a session file in generateds_gui.py, the graphical front-end for @@ -3180,7 +3184,7 @@ following among others:</p> <div class="footer"> <hr class="footer" /> <a class="reference external" href="generateDS.txt">View document source</a>. -Generated on: 2016-10-13 22:29 UTC. +Generated on: 2016-11-16 20:28 UTC. Generated by <a class="reference external" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference external" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source. </div> diff --git a/generateDS.py b/generateDS.py index 1da2d2f43e333c30377c70e69875e766f41f26ea..465a61c85c3d6d261a4ffb3f5c235b03b85d1cd7 100755 --- a/generateDS.py +++ b/generateDS.py @@ -98,12 +98,9 @@ Options: search pattern and second is a replacement. Example: "[('[-:.]', '_'), ('^__', 'Special')]" Default: "[('[-:.]', '_')]" - --remove-duplicate-child-elements - If a child element with the same name is - defined multiple times under a parent, then - remove it and issue a warning. -q, --no-questions Do not ask questions, for example, force overwrite. + --no-warnings Do not print warning messages. --session=mysession.session Load and use options from session file. You can create session file in generateds_gui.py. Or, @@ -207,7 +204,7 @@ logging.disable(logging.INFO) # Do not modify the following VERSION comments. # Used by updateversion.py. ##VERSION## -VERSION = '2.23c' +VERSION = '2.24a' ##VERSION## if sys.version_info.major == 2: @@ -279,7 +276,7 @@ SingleFileOutput = True OutputDirectory = None ModuleSuffix = "" PreserveCdataTags = False -RemoveDuplicateChildElements = False +NoWarnings = False SchemaToPythonTypeMap = {} @@ -922,12 +919,11 @@ class XschemaElement(XschemaElementBase): self.expandGroupReferences_tree(visited) self.collect_element_dict() self.annotate_find_type() - element_dict = {} - to_be_removed = [] + element_dict = None + to_be_removed = None self.annotate_tree( element_dict=element_dict, to_be_removed=to_be_removed) - self.remove_children(to_be_removed) self.fix_dup_names() self.coerce_attr_types() self.checkMixedBases() @@ -1163,11 +1159,18 @@ class XschemaElement(XschemaElementBase): child.annotate_find_type() def annotate_tree(self, element_dict, to_be_removed): - if RemoveDuplicateChildElements: - name = self.getName() - if name in element_dict: - # If we've already seen this element (name), make it a - # list and throw the previous one away. + name = self.getName() + if element_dict is not None: + dup_element = element_dict.get(name) + if (dup_element is not None and + # The element_dict keys are the element name. + # So, we do not need to compare names. + #dup_element.getName() == name and + not self.complexType and + not dup_element.complexType): + # If we've already seen this element (name), and + # these are both xsd:complexType, then + # make it a list and throw the previous one away. self.maxOccurs = 2 to_be_removed.append(element_dict[name]) err_msg( @@ -6894,7 +6897,8 @@ def capture_cleanup_name_list(option): def err_msg(msg): - sys.stderr.write(msg) + if not NoWarnings: + sys.stderr.write(msg) USAGE_TEXT = __doc__ @@ -6916,7 +6920,7 @@ def main(): FixTypeNames, SingleFileOutput, OutputDirectory, \ ModuleSuffix, UseOldSimpleTypeValidators, \ PreserveCdataTags, CleanupNameList, \ - RemoveDuplicateChildElements + NoWarnings outputText = True args = sys.argv[1:] try: @@ -6934,7 +6938,7 @@ def main(): 'one-file-per-xsd', 'output-directory=', 'module-suffix=', 'use-old-simpletype-validators', 'preserve-cdata-tags', 'cleanup-name-list=', - 'remove-duplicate-child-elements', + 'no-warnings', ]) except getopt.GetoptError: usage() @@ -7119,8 +7123,8 @@ def main(): PreserveCdataTags = True elif option[0] == '--cleanup-name-list': CleanupNameList = capture_cleanup_name_list(option[1]) - elif option[0] == '--remove-duplicate-child-elements': - RemoveDuplicateChildElements = True + elif option[0] == '--no-warnings': + NoWarnings = True if showVersion: print('generateDS.py version %s' % VERSION) sys.exit(0) diff --git a/generateDS.txt b/generateDS.txt index 4169ac8ab46af86a313da4a95b4f64f67cc2897b..09a490ee11f6a8ffb531d73d65651456616cfa33 100644 --- a/generateDS.txt +++ b/generateDS.txt @@ -12,7 +12,7 @@ generateDS -- Generate Data Structures from XML Schema .. version -:revision: 2.23c +:revision: 2.24a .. version @@ -223,7 +223,7 @@ Here is the usage message displayed by ``generateDS.py``:: Synopsis: - Generate Python classes from XML Schema definition. + Generate Python classes from XML schema definition. Input is read from in_xsd_file or, if "-" (dash) arg, from stdin. Output is written to files named in "-o" and "-s" options. Usage: @@ -253,7 +253,7 @@ Here is the usage message displayed by ``generateDS.py``:: top level class; if YY omitted XX is the default. class. Also see section "Recognizing the top level element" in the documentation. - --super="XXX" Super module name in generated in subclass + --super="XXX" Super module name in generated subclass module. Default="???" --validator-bodies=path Path to a directory containing files that provide bodies (implementations) of validator methods. @@ -278,10 +278,10 @@ Here is the usage message displayed by ``generateDS.py``:: generated files. This is useful if you want to minimize the amount of (no-operation) changes to the generated python code. - --no-process-includes Do not process included XML Schema files. By + --no-process-includes Do not process included XML schema files. By default, generateDS.py will insert content from files referenced by <include ... /> - elements into the XML Schema to be processed. + elements into the XML schema to be processed. --silence Normally, the code generated with generateDS echoes the information being parsed. To prevent the echo from occurring, use the --silence switch. @@ -322,6 +322,7 @@ Here is the usage message displayed by ``generateDS.py``:: Default: "[('[-:.]', '_')]" -q, --no-questions Do not ask questions, for example, force overwrite. + --no-warnings Do not print warning messages. --session=mysession.session Load and use options from session file. You can create session file in generateds_gui.py. Or, @@ -344,6 +345,7 @@ Here is the usage message displayed by ``generateDS.py``:: also generates member specifications in each class (in a dictionary). + Command line options ---------------------- @@ -613,6 +615,10 @@ q, no-questions case, when "-q" is used, the "-f" must be used to force the output file to be written. +no-warnings + While running generateDS.py, do not print warning messages that + would be written to stderr. + session=mysession.session Load and use options from session file. You can create a session file in generateds_gui.py, the graphical front-end for diff --git a/generateds_gui_notes.html b/generateds_gui_notes.html index 4906de597c2b4de124006be01a4a12c531e1f462..c3958eb09d0f2cd91ba23b089641a5531aab1638 100644 --- a/generateds_gui_notes.html +++ b/generateds_gui_notes.html @@ -220,7 +220,7 @@ They are used by updateversion.py. --> <col class="field-name" /> <col class="field-body" /> <tbody valign="top"> -<tr class="field"><th class="field-name">revision:</th><td class="field-body">2.23c</td> +<tr class="field"><th class="field-name">revision:</th><td class="field-body">2.24a</td> </tr> </tbody> </table> @@ -229,7 +229,7 @@ They are used by updateversion.py. --> <col class="field-name" /> <col class="field-body" /> <tbody valign="top"> -<tr class="field"><th class="field-name">date:</th><td class="field-body">October 13, 2016</td> +<tr class="field"><th class="field-name">date:</th><td class="field-body">November 16, 2016</td> </tr> </tbody> </table> @@ -401,7 +401,7 @@ $ mv generateds_gui.mo locale/ru/LC_MESSAGES/ <div class="footer"> <hr class="footer" /> <a class="reference external" href="generateds_gui_notes.txt">View document source</a>. -Generated on: 2016-10-13 22:29 UTC. +Generated on: 2016-11-16 20:28 UTC. Generated by <a class="reference external" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference external" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source. </div> diff --git a/generateds_gui_notes.txt b/generateds_gui_notes.txt index 1c11460a20f50bac1793eae2bda0effa593a03ca..0086f3da97ada7ac4745f8eed6a9c7fb99b95346 100644 --- a/generateds_gui_notes.txt +++ b/generateds_gui_notes.txt @@ -12,7 +12,7 @@ GenerateDS GUI Notes .. version -:revision: 2.23c +:revision: 2.24a .. version diff --git a/gui/generateds_gui.py b/gui/generateds_gui.py index 64b8c7c45e3f12a10f2acc800f8f3d128b35b46b..8c99e306b86d0f5b9a52ce461c031e5593c8fd16 100755 --- a/gui/generateds_gui.py +++ b/gui/generateds_gui.py @@ -41,7 +41,7 @@ from libgenerateDS.gui import generateds_gui_session # Do not modify the following VERSION comments. # Used by updateversion.py. ##VERSION## -VERSION = '2.23c' +VERSION = '2.24a' ##VERSION## diff --git a/librarytemplate_howto.html b/librarytemplate_howto.html index 84adafcd42f27ecc31f10897b38851ada8145187..785be91b932e99365a284857a8142d3150a52fc4 100644 --- a/librarytemplate_howto.html +++ b/librarytemplate_howto.html @@ -217,7 +217,7 @@ dkuhlman (at) davekuhlman (dot) org <col class="field-name" /> <col class="field-body" /> <tbody valign="top"> -<tr class="field"><th class="field-name">revision:</th><td class="field-body">2.23c</td> +<tr class="field"><th class="field-name">revision:</th><td class="field-body">2.24a</td> </tr> </tbody> </table> @@ -226,7 +226,7 @@ dkuhlman (at) davekuhlman (dot) org <col class="field-name" /> <col class="field-body" /> <tbody valign="top"> -<tr class="field"><th class="field-name">date:</th><td class="field-body">October 13, 2016</td> +<tr class="field"><th class="field-name">date:</th><td class="field-body">November 16, 2016</td> </tr> </tbody> </table> @@ -380,7 +380,7 @@ this command for your needs. For example, you may need to use <div class="footer"> <hr class="footer" /> <a class="reference external" href="librarytemplate_howto.txt">View document source</a>. -Generated on: 2016-10-13 22:29 UTC. +Generated on: 2016-11-16 20:28 UTC. Generated by <a class="reference external" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference external" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source. </div> diff --git a/librarytemplate_howto.txt b/librarytemplate_howto.txt index 61fa8f85f6d2f1470ab70dc6406e68aef08b10b2..4b30ec105915903c42baacd7e17ebc9057125ab5 100644 --- a/librarytemplate_howto.txt +++ b/librarytemplate_howto.txt @@ -8,7 +8,7 @@ How to package a generateDS.py generated library .. version -:revision: 2.23c +:revision: 2.24a .. version diff --git a/process_includes.py b/process_includes.py index ec67d80a0c5f9b501d10302cc8fc70ea09fdfa34..3325f9dc2dbec721f755aa3b9e5c9628168a8875 100755 --- a/process_includes.py +++ b/process_includes.py @@ -40,7 +40,7 @@ except ImportError: # Do not modify the following VERSION comments. # Used by updateversion.py. ##VERSION## -VERSION = '2.23c' +VERSION = '2.24a' ##VERSION## CatalogDict = {} diff --git a/setup.py b/setup.py index 8866826e926f701d15991b779f692b9bf36d091c..2bf1e4617db42cbabadc7484d2aeed3d0164768e 100644 --- a/setup.py +++ b/setup.py @@ -7,7 +7,7 @@ setup(name="generateDS", # Do not modify the following VERSION comments. # Used by updateversion.py. ##VERSION## - version="2.23c", + version="2.24a", ##VERSION## author="Dave Kuhlman", author_email="dkuhlman@davekuhlman.org", diff --git a/tests/check_results.rb b/tests/check_results.rb index fd60745c4c4d4771c34a9162df4ee7fbe243ac37..bdbec1e12a1bd16c2dceeddc35ca98d8ca4af072 100755 --- a/tests/check_results.rb +++ b/tests/check_results.rb @@ -63,6 +63,8 @@ $commands = [ "diff -u defaults_cases1_sup.py defaults_cases2_sup.py", "diff -u cleanupname1_sub.py cleanupname2_sub.py", "diff -u cleanupname1_sup.py cleanupname2_sup.py", + "diff -u rem_dup_elems1_sub.py rem_dup_elems2_sub.py", + "diff -u rem_dup_elems1_sup.py rem_dup_elems2_sup.py", ] def check diff --git a/tests/copy_all b/tests/copy_all index 62b953ada9ffe2857243fa1860527d3ee1366558..b89ea4351500e0adea45b58526ba0e7bd60aeb0f 100755 --- a/tests/copy_all +++ b/tests/copy_all @@ -55,3 +55,5 @@ cp cleanupname2_sup.py cleanupname1_sup.py cp cleanupname2_sub.py cleanupname1_sub.py cp catalogtest2_sup.py catalogtest1_sup.py cp catalogtest2_sub.py catalogtest1_sub.py +cp rem_dup_elems2_sup.py rem_dup_elems1_sup.py +cp rem_dup_elems2_sub.py rem_dup_elems1_sub.py diff --git a/tests/test.py b/tests/test.py index 7558302bc322507b7db12a877133efbc7057683f..84135e01ca98365b8be4f77acea4d3d1a7732a0b 100755 --- a/tests/test.py +++ b/tests/test.py @@ -797,6 +797,32 @@ class GenTest(unittest.TestCase): self.remove('{}2_sup.py'.format(t_)) self.remove('{}2_sub.py'.format(t_)) + def test_032_rem_dup_elems(self): + cmdTempl = ( + 'python generateDS.py --no-dates --no-versions ' + '--member-specs=list -f -a "xsd:" ' + '-o tests/%s2_sup.py -s tests/%s2_sub.py ' + '--super=%s2_sup --no-warnings ' + 'tests/%s.xsd' + ) + t_ = 'rem_dup_elems' + cmd = cmdTempl % (t_, t_, t_, t_, ) + self.executeClean(cmd, cwd='..') + self.compareFiles( + '{}1_sup.py'.format(t_), + '{}2_sup.py'.format(t_), + ('sys.stdout.write',)) + self.compareFiles('{}1_sub.py'.format(t_), '{}2_sub.py'.format(t_)) + # Need to preserve generated files for next command, cleanup at end + cmdTempl = 'python tests/%s2_sup.py tests/%s.xml > tests/%s2_out.xml' + cmd = cmdTempl % (t_, t_, t_, ) + self.executeClean(cmd, cwd='..') + self.compareFiles('{}1_out.xml'.format(t_), '{}2_out.xml'.format(t_)) + # cleanup generated files + self.remove('{}2_sup.py'.format(t_)) + self.remove('{}2_sub.py'.format(t_)) + self.remove('{}2_out.xml'.format(t_)) + def compareFiles(self, left, right, ignore=None): with open(left) as left_file: with open(right) as right_file: diff --git a/tutorial/generateds_tutorial.html b/tutorial/generateds_tutorial.html index 6e3d4672303120f7d7240d1be64d5025fea08029..212e6af8d1079e19a04b1fe4a257ead414cfee2f 100644 --- a/tutorial/generateds_tutorial.html +++ b/tutorial/generateds_tutorial.html @@ -219,7 +219,7 @@ They are used by updateversion.py. --> <col class="field-name" /> <col class="field-body" /> <tbody valign="top"> -<tr class="field"><th class="field-name">revision:</th><td class="field-body">2.23c</td> +<tr class="field"><th class="field-name">revision:</th><td class="field-body">2.24a</td> </tr> </tbody> </table> @@ -228,7 +228,7 @@ They are used by updateversion.py. --> <col class="field-name" /> <col class="field-body" /> <tbody valign="top"> -<tr class="field"><th class="field-name">date:</th><td class="field-body">October 13, 2016</td> +<tr class="field"><th class="field-name">date:</th><td class="field-body">November 16, 2016</td> </tr> </tbody> </table> @@ -1210,7 +1210,7 @@ named <tt class="docutils literal">garden_api.py</tt>, you can create an instanc <div class="footer"> <hr class="footer" /> <a class="reference external" href="generateds_tutorial.txt">View document source</a>. -Generated on: 2016-10-13 22:29 UTC. +Generated on: 2016-11-16 20:28 UTC. Generated by <a class="reference external" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference external" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source. </div> diff --git a/tutorial/generateds_tutorial.txt b/tutorial/generateds_tutorial.txt index 7c71690bdbb55c89957ceafa6ba54dbcc0347960..e334ec34f71201a689ee1ac3950451da1bdd3467 100644 --- a/tutorial/generateds_tutorial.txt +++ b/tutorial/generateds_tutorial.txt @@ -11,7 +11,7 @@ generateDS -- Introduction and Tutorial .. version -:revision: 2.23c +:revision: 2.24a .. version diff --git a/tutorial/generateds_tutorial.zip b/tutorial/generateds_tutorial.zip index b43776a27c1672ffa3296d877d8bedfd5b46d742..31b243add61d581c179eadee276101e0113a622c 100644 Binary files a/tutorial/generateds_tutorial.zip and b/tutorial/generateds_tutorial.zip differ