Calling API function, returns data that does not match the API function description

[Paul Millard]

Good morning.

I'm currently working on developing a software that utilises the Sage Accounting API v3.1.  I downloaded the Swagger file, and generated a C# .NET Class library based on the 'swagger.full.json' file.  I used the Swagger-Codegen utility:

java -jar /projects/swagger-codegen/modules/swagger-codegen-cli/target/swagger-codegen-cli.jar generate -i swagger.full.json -l csharp -o .\CSharp

This was a great start because I successfully wrote the authorisation token code and I'm able to communicate with the API, because it returns the exact same data that I get when calling the 'Products/Get all products' method in Postman:

{
    "$total": 1,
    "$page": 1,
    "$next": null,
    "$back": null,
    "$itemsPerPage": 20,
    "$items": [
        {
            "id": "33095baf0a6647de91c34117753c6a97",
            "displayed_as": "Standard A4 Notebook",
            "$path": "/products/33095baf0a6647de91c34117753c6a97"
        }
    ]
}

The API documentation describes the end-point for 'Products/GET - Returns all products' as returning a JSON response that follows this structure:

[
    {
        "id": "string",
        "displayed_as": "string",
        "$path": "string",
        "created_at": "2019-08-24T14:15:22Z",
        "updated_at": "2019-08-24T14:15:22Z",
        "deleted_at": "2019-08-24T14:15:22Z",
        "deletable": true,
        "deactivatable": true,
        "used_on_recurring_invoice": true,
        "item_code": "string",
        "description": "string",
        "notes": "string",
        "sales_ledger_account": {},
        "sales_tax_rate": {},
        "purchase_ledger_account": {},
        "usual_supplier": {},
        "purchase_tax_rate": {},
        "cost_price": 0,
        "sales_prices": [],
        "source_guid": "string",
        "purchase_description": "string",
        "active": true,
        "catalog_item_type": {}
    }
]

So, the problem is this.  The Swagger file has generated a class in my Class Library, called ProductsApi.  The function I used to test the ability to call the API, is 'GetProducts':

/// <summary>
/// Returns all Products
/// </summary>
/// <remarks>
/// ### Endpoint Availability  * Accounting Plus: 🇨🇦, 🇩🇪, 🇪🇸, 🇫🇷, 🇬🇧, 🇮🇪, 🇺🇸 * Accounting Standard: 🇬🇧, 🇮🇪 * Accounting Start: 🇨🇦, 🇩🇪, 🇪🇸, 🇫🇷, 🇬🇧, 🇮🇪, 🇺🇸  ### Access Control Restrictions  Requires the authenticated user to have any mentioned role in one of the listed areas: * Area: &#x60;Products &amp; Services&#x60;: Read Only, Restricted Access, Full Access * Area: &#x60;Sales&#x60;: Read Only, Restricted Access, Full Access * Area: &#x60;Purchases&#x60;: Read Only, Restricted Access, Full Access
/// </remarks>
/// <exception cref="IO.Swagger.Client.ApiException">Thrown when fails to make API call</exception>
/// <param name="search">Use this to filter by the item code or description. (optional)</param>
/// <param name="updatedOrCreatedSince">Use this to limit the response to Products changed since a given date (format: YYYY-MM-DDT(+|-)hh:mm) or date-time (format: YYYY-MM-DDThh:mm:ss(+|-)hh:mm). Inclusive of the passed timestamp. (optional)</param>
/// <param name="deletedSince">Use this to limit the response to Products deleted since a given date (format: YYYY-MM-DDT(+|-)hh:mm) or date-time (format: YYYY-MM-DDThh:mm:ss(+|-)hh:mm). Not inclusive of the passed timestamp. (optional)</param>
/// <param name="active">Use this to only return active or inactive items (optional)</param>
/// <param name="itemsPerPage">Returns the given number of Products per request. (optional, default to 20)</param>
/// <param name="page">Go to specific page of Products (optional, default to 1)</param>
/// <param name="attributes">Specify the attributes that you want to expose for the Products (expose all attributes with &#39;all&#39;). These are in addition to the base attributes (name, path) (optional)</param>
/// <returns>List&lt;Product&gt;</returns>
List<Product> GetProducts (string search = null, DateTime? updatedOrCreatedSince = null, DateTime? deletedSince = null, bool? active = null, int? itemsPerPage = null, int? page = null, string attributes = null);

As you can see, it returns a List<Product> object.  When I call this function, I get an immediate exception error with the following details:

Quote

Cannot deserialize the current JSON object (e.g. {"name":"value"}) into type 'System.Collections.Generic.List`1[IO.Swagger.Model.Product]' because the type requires a JSON array (e.g. [1,2,3]) to deserialize correctly.
To fix this error either change the JSON to a JSON array (e.g. [1,2,3]) or change the deserialized type so that it is a normal .NET type (e.g. not a primitive type like integer, not a collection type like an array or List<T>) that can be deserialized from a JSON object. JsonObjectAttribute can also be added to the type to force it to deserialize from a JSON object.
Path '$total', line 1, position 10.

Further investigation shows that the API is returning data in the exact format seen in Postman, which you can see further up.  So, the issue is that the function I call, is not matching up with the expected data for the API documentation.  Can anyone explain why this is the case, and is it possible to resolve?

[Paul Millard]

Continuing from my post, I have called the function, passing the parameter for 'attributes' with the value 'all', and clearly get all of the fields that are described in the API documentation.  But the prefix part of the JSON data structure, still causes the function to throw an exception error.

Quote

{"$total":1,"$page":1,"$next":null,"$back":null,"$itemsPerPage":20,"$items":[{"id":"33095baf0a6647de91c34117753c6a97","displayed_as":"Standard A4 Notebook","$path":"/products/33095baf0a6647de91c34117753c6a97","created_at":"2023-06-26T15:40:10Z","updated_at":"2023-06-26T15:40:10Z","deletable":true,"deactivatable":true,"used_on_recurring_invoice":false,"item_code":"BOOK001","description":"Standard A4 Notebook","notes":"","sales_ledger_account":{"id":"fbcf0a5e142e11eea14402b6e5b12293","displayed_as":"Sales - Products (4000)","$path":"/ledger_accounts/fbcf0a5e142e11eea14402b6e5b12293"},"sales_tax_rate":{"id":"GB_STANDARD","displayed_as":"Standard 20.00%","$path":"/tax_rates/GB_STANDARD"},"purchase_ledger_account":{"id":"fbd022f8142e11eea14402b6e5b12293","displayed_as":"Cost of Sales - Goods (5000)","$path":"/ledger_accounts/fbd022f8142e11eea14402b6e5b12293"},"usual_supplier":null,"purchase_tax_rate":{"id":"GB_STANDARD","displayed_as":"Standard 20.00%","$path":"/tax_rates/GB_STANDARD"},"cost_price":"2.45","sales_prices":[{"id":"7afa2ba2b2594c4680a381fd860d8c40","displayed_as":"Sales Price","created_at":"2023-06-26T15:40:10Z","updated_at":"2023-06-26T15:40:10Z","price_name":"Sales Price","price":"2.95","price_includes_tax":false,"product_sales_price_type":{"id":"91e907bad31945e48af124ba2a0a5f81","displayed_as":"Sales Price","$path":"/product_sales_price_types/91e907bad31945e48af124ba2a0a5f81"}},{"id":"b915c2ef63af4648827c68488f7714f2","displayed_as":"Trade","created_at":"2023-06-26T15:40:10Z","updated_at":"2023-06-26T15:40:10Z","price_name":"Trade","price":"0.0","price_includes_tax":false,"product_sales_price_type":{"id":"87d70f76a7b64e36b849b5dcdfcfbcc6","displayed_as":"Trade","$path":"/product_sales_price_types/87d70f76a7b64e36b849b5dcdfcfbcc6"}},{"id":"5cef5da24026454eac71a1b356e49ac3","displayed_as":"Wholesale","created_at":"2023-06-26T15:40:10Z","updated_at":"2023-06-26T15:40:10Z","price_name":"Wholesale","price":"0.0","price_includes_tax":false,"product_sales_price_type":{"id":"05c487e3d88c4ca7a45f6aef832b2a43","displayed_as":"Wholesale","$path":"/product_sales_price_types/05c487e3d88c4ca7a45f6aef832b2a43"}}],"source_guid":null,"purchase_description":"","active":true,"catalog_item_type":{"id":"PRODUCT","displayed_as":"Non-stock","$path":"/catalog_item_types/PRODUCT"}}]}

So the call is clearly returning data, but the function does not interpret the RESPONSE correctly.

[Paul Millard]

This is the function that causes the problem for me.  I will try to modify this function myself, but presently, if the API development team, are able to provide an interim solution, it would be very much appreciated.

/// <summary>
/// Deserialize the JSON string into a proper object.
/// </summary>
/// <param name="response">The HTTP response.</param>
/// <param name="type">Object type.</param>
/// <returns>Object representation of the JSON string.</returns>
public object Deserialize(IRestResponse response, Type type)
{
    IList<Parameter> headers = response.Headers;
    if (type == typeof(byte[])) // return byte array
    {
        return response.RawBytes;
    }

    // TODO: ? if (type.IsAssignableFrom(typeof(Stream)))
    if (type == typeof(Stream))
    {
        if (headers != null)
        {
            var filePath = String.IsNullOrEmpty(Configuration.TempFolderPath)
                ? Path.GetTempPath()
                : Configuration.TempFolderPath;
            var regex = new Regex(@"Content-Disposition=.*filename=['""]?([^'""\s]+)['""]?$");
            foreach (var header in headers)
            {
                var match = regex.Match(header.ToString());
                if (match.Success)
                {
                    string fileName = filePath + SanitizeFilename(match.Groups[1].Value.Replace("\"", "").Replace("'", ""));
                    File.WriteAllBytes(fileName, response.RawBytes);
                    return new FileStream(fileName, FileMode.Open);
                }
            }
        }
        var stream = new MemoryStream(response.RawBytes);
        return stream;
    }

    if (type.Name.StartsWith("System.Nullable`1[[System.DateTime")) // return a datetime object
    {
        return DateTime.Parse(response.Content, null, System.Globalization.DateTimeStyles.RoundtripKind);
    }

    if (type == typeof(String) || type.Name.StartsWith("System.Nullable")) // return primitive type
    {
        return ConvertType(response.Content, type);
    }

    // at this point, it must be a model (json)
    try
    {
        return JsonConvert.DeserializeObject(response.Content, type, serializerSettings);
    }
    catch (Exception e)
    {
        throw new ApiException(500, e.Message);
    }
}

 

[Steel, Mark]

Hi Paul,

Thank you for your question. 

We've seen the same issue with an integrator compiling a java library in the same way. Their solution was to amend the swagger at source to include a page object to describe the pagination meta data for the API's they required. I've sent you an example of the swagger they used in reply to your email.

We have tickets on our backlog to improve the swagger definitions we provide and hope to be able to action these tickets in the near future.

Thanks

Mark

[Paul Millard]

Quote

Hi Paul,

Thank you for your question. 

We've seen the same issue with an integrator compiling a java library in the same way. Their solution was to amend the swagger at source to include a page object to describe the pagination meta data for the API's they required. I've sent you an example of the swagger they used in reply to your email.

We have tickets on our backlog to improve the swagger definitions we provide and hope to be able to action these tickets in the near future.

Thanks

Mark

Hi Mark,

 Many, many thanks for this information!  After examining the Swagger file that you sent to me via email, I was able to modify my Swagger file to accommodate the new Pagination type for the model records that I was testing with.  Regenerating the Swagger CSharp project, I was able to see the new return types.  The GET methods now work without error.  As mentioned in my email, I think it's helpful that the community are aware of this, if they are generating their model/api classes from the Swagger file.  I have attached the swagger file you sent via email.

For the community, you need to look at the "BankAccountPage" and "LedgerAccountPage" types that are defined in the Swagger file.  They inherit the "Page" type and define the array of subsequent "BankAccount" or "LedgerAccount" types.  You will also need to examine the "Get" methods for these types, as they originally returned a list of "BankAccount" objects, but now return a single "BankAccountPage" object which is correctly deserialized from the JSON response message.

Again, many thanks Mark, all the help is much appreciated.

Paul.

swagger.full.pagination-poc.json

Edited July 21, 2023 by Paul Millard
Add Swagger file

[Tony Fletcher]

Thank you @Paul Millard, we have run into the same issue which we have emailed Sage about and your attachment has been very helpful.

We have been dreading integrating with Sage after having previous experience with other Sage API's and so far it seems our fears have been justified, when compared to how easy it is to integrate with Xero, the integration with Sage is already causing problems and we have barely even started.

We were adding a reference to Sage's swagger file to auto generate the code in Visual Studio for calling their API and as you have explained above the responses set out in their swagger file do not match what the API actually returns.

I can see that you have updated the swagger file for "LedgerAccounts", "Products" and "BankAccounts" but I assume we will need to do the same for any API calls that are supposed to return an array of items or is it only those three that are affected by this issue? 

Hopefully Sage can update their swagger file so that it shows the correct responses that their API returns because we don't really want to have to keep manually modifying the swagger file each time they release a new version of their API.