Method Community

I'm attempting to pull the InvoiceLine table via a python script that's been working for months but all of a sudden is having issues. 

The zeep client (or underlying lxml pkg) is having issues parsing the soap response:

Traceback (most recent call last):

  File "/home/user/envs/sales/lib/python3.6/site-packages/zeep/wsdl/bindings/soap.py", line 170, in process_reply
    doc = parse_xml(content, self.transport, settings=client.settings)
  File "/home/user/envs/sales/lib/python3.6/site-packages/zeep/loader.py", line 54, in parse_xml
    content=content
zeep.exceptions.XMLSyntaxError: Invalid XML content received (xmlSAX2Characters: huge text node, line 321734, column 43)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/home/user/envs/sales/lib/python3.6/site-packages/zeep/proxy.py", line 42, in __call__
    self._op_name, args, kwargs)
  File "/home/user/envs/sales/lib/python3.6/site-packages/zeep/wsdl/bindings/soap.py", line 132, in send
    return self.process_reply(client, operation_obj, response)
  File "/home/user/envs/sales/lib/python3.6/site-packages/zeep/wsdl/bindings/soap.py", line 176, in process_reply
    content=response.content)
zeep.exceptions.TransportError: Server returned response (200) with invalid XML: Invalid XML content received (xmlSAX2Characters: huge text node, line 321734, column 43).

I'm requesting the following fields from the InvoiceLine table:

'Item', 'Amount', 'Class', 'Desc', 'InvoiceAmount', 'LotNumber', 'Other1', 'Quantity', 'Rate', 'Other2', 'TotalAmount', 'InvoiceRecordID', 'RecordID', 'TxnLineID'

Strange this is if I exclude any of the above fields then the reponse comes back fine. Is there just too much data being returned? 

EDIT: 

I enabled the xml_huge_tree setting in the zeep client and am no longer having this issue.

>xml_huge_tree – disable lxml/libxml2 security restrictions and support very deep trees and very long text content

I'm still not sure what is happening here, if anyone can shed some light on it would be much appreciated.

Hello botanica,

Thanks for reaching out. We havent made any changes to our API in the past few weeks. It is possible that the response was huge. Are you trying to retrieve a specific InvoiceLine record or all InvoiceLine records? 

Thanks for the quick reply.

Yeah we have a few different Method accounts with which we do a lot of the same stuff against the API (like pulling invoices) and only one of our accounts is having this issue. It really seems to be some sort of limit being reached (albeit perhaps no longer an issue with the xml_huge_tree setting in zeep enabled). I am retrieving all invoices and all invoice lines in order to build a data set using python/pandas that I then use to populate a dashboard. Is it not recommended to retrieve all invoice lines at once? 

We do not have any limitations on the size of the response. However, it looks like lxml package has a limitation of 10 MB http://www.xmlsoft.org/ChangeLog.html. Also, I am not sure whether this will help but we do have a column called "LastModifiedDate" which gets updated with Date and time whenever that record gets updated. It can help in limiting the size of the response if necessary.

Got it, thank you! I didn't notice the lxml limit.