In Python, how to set _soapheaders for Zeep using Dictionaries?

Question

In working with a SOAP api, the wsdl spec describes the api key passed in the header in a complex namespaced structure as well as additional non-namespaced XML that relates to a paging mechanism for accessing bulk results successively:

Specification:

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns="https://webservice_address_here">
  <soapenv:Header>
    <ns:apiKey>
      <api_key>***</api_key>
    </ns:apiKey>
    <pager>
      <page>1</page>
      <per_page>100</per_page>
    </pager>
  </soapenv:Header>
</soapenv:Envelope>

API Documentation:

Pagination; this method returns paginated results. To specify pages or results per page, use the pager header:

<soapenv:Header>
  <ns:pager>
    <page>1</page>
    <per_page>100</per_page>
  </ns:pager>
</soapenv:Header> Max per page is 100

Pagination information is returned in a pager header:

<soapenv:Header>
  <ns:pager>
    <page>1</page>
    <per_page>100</per_page>
    <next_page>2</next_page>
    <page_items>100</page_items>
    <total_items>2829</total_items>
    <total_pages>29</total_pages>
  </ns:pager>
</soapenv:Header>

The answer, How to set soap headers in zeep when header has multiple elements, describes a similar scenario, without the namespace "ns" but with "acm." I have not been successful in using this method.

This works, allowing access to the api but without the pager making it mostly useless for any methods that return more than 100 results:

from zeep import Client, xsd

# Generate the header structure
header = xsd.Element(
    '{wsdl}AuthenticateRequest',
    xsd.ComplexType([xsd.Element("{wsdl}api_key", xsd.String())])
)

# Insert values into header placeholders
self._header_value = header(api_key=self.api_key)

This does not work:

from zeep import Client, xsd

# Generate the header structure
header = xsd.Element(
    'Header',
    xsd.ComplexType([
        xsd.Element(
            '{wsdl}AuthenticateRequest',
            xsd.ComplexType([
                xsd.Element('{wsdl}api_key', xsd.String()),
            ])
        ),
        xsd.Element(
            'pager',
            xsd.ComplexType([
                xsd.Element('page', xsd.String()),
                xsd.Element('per_page', xsd.String()),
            ])
        ),
    ])
)

# ERROR HERE: Insert values into header placeholders
self._header_value = header(api_key=self.api_key, pager={'page':1,'per_page':100})

Error: TypeError: ComplexType() got an unexpected keyword argument 'api_key'. Signature: AuthenticateRequest: {api_key: xsd:string}, pager: {page: xsd:string, per_page: xsd:string}

This also does not work:

header = xsd.Element(
    '{wsdl}AuthenticateRequest',
    xsd.ComplexType([xsd.Element("{wsdl}api_key", xsd.String())]),
    xsd.Element(
        'pager',
        xsd.ComplexType([
            xsd.Element('page', xsd.String()),
            xsd.Element('per_page', xsd.String()),
        ])
    )
)

# ERROR HERE: Insert values into header placeholders
self._header_value = header(api_key=self.api_key, pager={"page":1,"per_page":100})

'pager' is not defined in the wsdl but the server expects that it could be there.

TypeError: ComplexType() got an unexpected keyword argument 'pager'. Signature: api_key: xsd:string

Recent Failed (2022) Attempt Using @markoan answer:

def get_pager(self, page: int = 1, per_page: int = 100):
    """ Create header that contains the page and records per page. """

    pager_header = xsd.Element(
        'pager',
        xsd.ComplexType([
            xsd.Element(
                'page', xsd.Integer()
            ),
            xsd.Element(
                'per_page', xsd.Integer()
            )
        ])
    )

    return pager_header(page=page, per_page=per_page)

def call(self, endpoint: str, *args, **kwargs):
    """Allows calling of any client service defined in the WSDL."""

    # get the endpoint
    endpoint = getattr(self.client.service, endpoint)

    # get SOAP authentication header which includes the API key embedded from CFG
    headers = [self.get_header()]

    # add the pager complex element to headers if required in kwargs
    if page := kwargs.get("page"):
        per_page = kwargs.get("per_page") or 100
        headers.append(self.get_pager(page, per_page))

    # call the endpoint with provided unnamed and named parameters if any
    result = endpoint(*args, **kwargs, _soapheaders=headers)

    # serialize and return result
    return self.serialize(result)

What is the simplest way using Zeep to set the namespace api_key and non-namespaced complex pager element?

Answer 1

I find it's easier to work with Zeep if we have a valid and complete WSDL.

A simple API service WSDL that expects an element without namespace would import a schema with no namespace like this:

<?xml version="1.0" encoding="UTF-8"?>
<wsdl:definitions targetNamespace="http://tempuri.org/" xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:tm="http://microsoft.com/wsdl/mime/textMatching/" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:mime="http://schemas.xmlsoap.org/wsdl/mime/" xmlns:tns="http://tempuri.org/" xmlns:s="http://www.w3.org/2001/XMLSchema" xmlns:soap12="http://schemas.xmlsoap.org/wsdl/soap12/" xmlns:http="http://schemas.xmlsoap.org/wsdl/http/" xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/" >
  <wsdl:types>
    <s:schema elementFormDefault="qualified" targetNamespace="http://tempuri.org/">
      <s:import schemaLocation="namespaceLessElement.xsd"/>
      <s:element name="Api" minOccurs="0" maxOccurs="1">
      </s:element>
      <s:element name="ApiResponse">
        <s:complexType>
          <s:sequence>
            <s:element minOccurs="1" maxOccurs="1" name="ApiResult" type="s:int"/>
          </s:sequence>
        </s:complexType>
      </s:element>
      <s:element name="apiKey">
        <s:complexType>
          <s:sequence>
            <s:element name="api_key" type="s:string"></s:element>
          </s:sequence>
        </s:complexType>
      </s:element>
    </s:schema>
  </wsdl:types>
  <wsdl:message name="ApiSoapIn">
    <wsdl:part name="parameters" element="tns:Api"/>
  </wsdl:message>
  <wsdl:message name="ApiSoapOut">
    <wsdl:part name="parameters" element="tns:ApiResponse"/>
  </wsdl:message>
  <wsdl:message name="ApiKeyHeader">
    <wsdl:part name="ApiKeyHeaderParam" element="tns:apiKey"/>
  </wsdl:message>
  <wsdl:message name="PagerHeader">
    <wsdl:part name="PagerHeaderParam" ref="pager"/>
  </wsdl:message>
  <wsdl:portType name="ApiSoap">
    <wsdl:operation name="Api">
      <wsdl:documentation>This is a test WebService. Returns a number</wsdl:documentation>
      <wsdl:input message="tns:ApiSoapIn"/>
      <wsdl:output message="tns:ApiSoapOut"/>
    </wsdl:operation>
  </wsdl:portType>
  <wsdl:binding name="ApiSoap" type="tns:ApiSoap">
    <soap:binding transport="http://schemas.xmlsoap.org/soap/http"/>
    <wsdl:operation name="Api">
      <soap:operation soapAction="http://tempuri.org/Api" style="document"/>
      <wsdl:input>
        <soap:header message="tns:ApiKeyHeader" part="ApiKeyHeaderParam" use="literal"/>
        <soap:header message="tns:PagerHeader" part="PagerHeaderParam" use="literal"/>
        <soap:body use="literal"/>
      </wsdl:input>
      <wsdl:output>
        <soap:body use="literal"/>
      </wsdl:output>
    </wsdl:operation>
  </wsdl:binding>
  <wsdl:service name="ApiTest">
    <wsdl:port name="ApiSoap" binding="tns:ApiSoap">
      <soap:address location="http://superpc:8082/"/>
    </wsdl:port>
  </wsdl:service>
</wsdl:definitions>

With namespaceLessElement.xsd:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<s:schema xmlns:s="http://www.w3.org/2001/XMLSchema"
           elementFormDefault="qualified">
  <s:element name="pager">
    <s:complexType>
      <s:sequence>
        <s:element name="page" type="s:int"></s:element>
        <s:element name="per_page" type="s:int"></s:element>
      </s:sequence>
    </s:complexType>
  </s:element>
</s:schema>

Note how the operation definition that expects header values points to correct messages:

<wsdl:operation name="Api">
  <soap:operation soapAction="http://tempuri.org/Api" style="document"/>
  <wsdl:input>
    <soap:header message="tns:ApiKeyHeader" part="ApiKeyHeaderParam" use="literal"/>
    <soap:header message="tns:PagerHeader" part="PagerHeaderParam" use="literal"/>
    <soap:body use="literal"/>
  </wsdl:input>
  <wsdl:output>
    <soap:body use="literal"/>
  </wsdl:output>
</wsdl:operation>

and these in turn reference correct elements:

<wsdl:message name="ApiKeyHeader">
  <wsdl:part name="ApiKeyHeaderParam" element="tns:apiKey"/>
</wsdl:message>
<wsdl:message name="PagerHeader">
  <wsdl:part name="PagerHeaderParam" ref="pager"/>
</wsdl:message>

You should check in the WSDL of your web service that the operation describes both headers and that it includes a schema definition for both elements. In the example WSDL the service namespace is targetNamespace="http://tempuri.org/" but this should point to your web service URL.

So assuming your WSDL is valid and complete, we need to define the Client pointing to the WSDL and then set the header values using the _soapheaders parameter, similar to the method I used here but building the content reference. Zeep can take care of the different namespaces but I found issues with empty ones:

transport = Transport(cache=SqliteCache())
self.Test = Client(wsdl='http://my-endpoint.com/production.svc?wsdl', transport=transport)

# Header objects
apiKey_header = xsd.Element(
    '{http://tempuri.org/}apiKey',
    xsd.ComplexType([
        xsd.Element(
            'api_key', xsd.String()
        )
    ])
)

pager_header = xsd.Element(
    'pager',
    xsd.ComplexType([
        xsd.Element(
            'page', xsd.Integer()
        ),
        xsd.Element(
            'per_page', xsd.Integer()
        )
    ])
)

apiKey_header_value = apiKey_header( api_key=key)
pager_header_value = pager_header( page=page, per_page=perpage)

# Request
response = self.Test.service.Api( _soapheaders=[apiKey_header_value, pager_header_value] )

logger.debug("Result={1}".format(response))

# Prints: Result=2 (or whatever value the test API sends)

EDIT: Example of generated XML request:

<soap-env:Envelope xmlns:soap-env="http://schemas.xmlsoap.org/soap/envelope/">
   <soap-env:Header>
      <ns0:apiKey xmlns:ns0="http://tempuri.org/">
         <api_key>1230011</api_key>
      </ns0:apiKey>
      <pager>
         <page>2</page>
         <per_page>10</per_page>
      </pager>
   </soap-env:Header>
   <soap-env:Body>
      <ns0:Api xmlns:ns0="http://tempuri.org/"/>
   </soap-env:Body>
</soap-env:Envelope>

Make sure that the header that has a namespace is defined with the correct URL.

If you still have problems it may mean your WSDL does not define all elements or that it's not linking correctly to external XSDs. In those cases one option is to save a local copy os the WSDL and linked XSDs, then edit the files to fix references and then point Zeep to that local file instead.