Consuming and providing REST APIs using the Data Exchange library

In addition to reading from and writing data to JSON, CSV, XML and Excel files, the Data Exchange library also supports sending HTTP client requests to an HTTP (REST) service, as well as exposing procedures in your model through a built-in HTTP service component. This makes it the one-stop solution for communicating JSON, CSV, XML and Excel data with external REST APIs, as well as exposing your model through a REST API, accepting JSON, CSV, XML or Excel data.

Consuming REST APIs

The Data Exchange library contains a fully asynchronous HTTP client library, based on the well-known libCurl library (see the libCurl documentation). Each request will be executed in parallel through a fixed number of concurrent connections, and upon each response a user-specified callback will be executed when the AIMMS engine is idle, or whenever the modeler has explicitly requested the Data Exchange library to execute the callbacks for all handled requests. This approach allows massive amounts of requests to be handled in parallel, tremendously decreasing the total time it takes to perform all requests when the service called is set up in a scalable manner.


The functions in the dex::client namespace offer alternative to the httpclient library, fully integrated within the Data Exchange library which will most likely be necessary for API calls anyway to map request bodies and responses to identifiers in the model. Both offer similar functionality, although there are some differences, most notably

  • the httpclient library does automatic proxy discovery, while for dex::client requests proxy discovery must be performed manually via the dex::client::ProxyResolve() function and subsequently set via the curl PROXY option

  • dex::client HTTP requests can make use of all libCurl functionality that is available via libCurl options but not in the httpclient library (e.g. SPNEGO authentication)

  • dex::client HTTP requests only support a fully asynchronous execution model, optimized for massive amounts of HTTP/API requests to be executed in parallel

Although the dex::client HTTP requests more than the httpclient library forces you to e.g. adhere strictly to its asynchronous model, or invoke certain functionality by specifically enabling it through the available options, the tight integration with the Data Exchange mapping capabilities allows for more advanced features on the Data Exchange Roadmap like automatic API client generation from an OpenAPI specification.

To initiate a request, call the function dex::client::NewRequest(), where you specify the URL of the request, the callback to be called for the response, the HTTP method (GET, POST, …), and, if

dex::client::NewRequest("request-1", "", 'SimpleRequestCallback', httpMethod: 'GET', responseFile: "out/request-1.html");

This as an instruction to create a new HTTP request with a unique identifier request-1, which will get the contents of After the Data Exchange library receives the response, it will be store in the file out/request-1.html, after which a callback called SimpleRequestCallback, will be called. The Data Exchange library will not come up with a unique identifier for the request itself, but will leave it up to you to come up with an appropriate scheme to identify your requests yourself.

The request will not be executed yet, however. Prior to this, you first have the opportunity to set additional headers for the request using the function dex::client::AddRequestHeaders(), or add additional Curl options using the function dex::client::AddRequestOptions(), for instance, to specify a proxy that is to be used to connect to the given server, or whether libCurl should follow any redirects it encounters.

To actually execute the request, you should call the function dex::client::PerformRequest()


This will queue the request for execution on one of the concurrent connections maintained by the Data Exchange library for making HTTP requests. You can specify how many concurrent connections you want to be used to execute HTTP requests through the function dex::client::SetParallelConnections(). By default, the Data Exchange library will use up to 16 parallel connections. By increasing this maximum number of connections you may substantially decrease the total amount of time taken to execute a large number of requests, but you should also make sure that the server infrastructure handling these requests is comfortable handling the number of parallel connections you set.

Upon completion of the request, your specified callback function will be called, with three arguments:

  • theRequest, the specific request identifier for which the callback is called.

  • statusCode, the HTTP status code of the response.

  • errorCode, the Curl error code for the response in case the request was not successful.

If there was a libCurl error, the HTTP status code will be 0, and you can use the function dex::client::GetErrorMessage(), to retrieve a description of the Curl error that occurred, based on the errorCode argument.

If the status code is 200 (OK), then you can proceed to request the response headers using the function dex::client::GetResponseHeaders(), request additional info about the request from libCurl using the function dex::client::GetInfoItems() (e.g. the total request time, or the final destination of your request in case of redirects), or can use the function dex::ReadFromFile() to read the response data into identifiers in your model in case of REST call to some REST API.

The Data Exchange library will close a request as soon as the specified callback function has been called, not to leave any resources in use unnecessary. It will, however, not remove any request and/or response files you specified.

The library has been tested to be able to call a very simple HTTP service (i.e., with an empty response) for 100,000 times over 256 parallel connections within 20 or so seconds, so should able to deal with a more realistic number of calls to a non-trivial service as well. Note that in this case, the time taken to deal with the response in the callback (e.g. reading the data in AIMMS identifiers) may substantially add to the overall time to make and handle all requests.

Debugging client requests

When you experience trouble invoking a URL using dex::client requests, a good way to debug this is to install a tool called Fiddler Everywhere. Using this tool you can install a local proxy on your own computer, which can decrypt any HTTPS traffic you send from dex::client requests. As Fiddler uses a local root certificate on your computer without a certificate revocation list, you need to set some additional options on the request, to make libcurl not check the revocation list.

dex::client::ProxyResolve(requestURL, proxyURL);        ! determine proxy URL
if (proxyURL) then
        stringOptions(dex::client::stropt) := { 'PROXY' : proxyURL };                           ! instruct libcurl to use the given proxy
        intOptions(dex::client::intopt) := { 'HTTPPROXYTUNNEL' : 1, 'SSL_OPTIONS' : 2 };        ! use a proxy tunnel, and don't check revocation list
        dex::client::AddRequestOptions(reqId, intOptions, stringOptions);

When you perform the request after setting these options, you can watch the contents of the request in Fiddler, and check what could be the cause of the problem.

Providing REST APIs

The Data Exchange library is also capable of providing a REST API service that exposes procedures in your model, and will form the basis of exposing procedures in published AIMMS apps in our cloud platform in the future.

With each procedure in your model, you can associate a dex::ServiceName annotation, which will expose your procedure under the path /api/v1/tasks/{service-name}, where {service-name} is the value you entered in the dex::ServiceName annotation.

  • /api/v1/tasks/{service-name}

    • POST: accepts any JSON/XML/CSV/Excel/… data as the request body. The REST API Service hander built into the Data Exchange library will queue the request, and call the procedure in your model corresponding to {service-name}. Within the procedure handling the request, the string parameter dex::api::RequestAttribute will provide you with access to the

      • id: the id assigned to the request by the Data Exchange library

      • request-data-path: the file path containing the request body

      • response-data-path: the file path in which to store the final response body

      • status-data-path: the file path in which to store any (regularly updated) intermediate model status you want to communicate to the caller while handling the request, prior to completion

      In addition, you can access the request headers via the string parameter dex::api::RequestHeader, while the string parameter dex::api::RequestParameter will hold any query parameters added to the request.

      A POST request to the URL will either return the status code 403 Forbidden if the service name cannot be found, or 200 OK if the request has been queued. In the latter case, the request will return a status response similar to:


      where status can be any of queued, executing, solving, interrupted or finished.

      The procedure body for handling such a request could look like:

      ! read data from request body
      dex::ReadFromFile(dex::api::RequestAttribute('request-data-path'), "GraphHopperMatrix", 1, 0, 1);
      ! do some manipulation of data
      GraphHopperMatrixResults(restp, from_point, to_point) *= 2;
      ! write response body
      dex::WriteToFile(dex::api::RequestAttribute('response-data-path'), "GraphHopperMatrix");
      ! the application-specific returncode that will be returned via the task status of the job
      return 1;
  • /api/v1/tasks

    • GET: will return 200 OK where the response body will contain a array with the statuses of all submitted jobs, similar to:

  • /api/v1/tasks/{id}

    • GET: will return a 404 Not found if there is no task with the given id, or 200 OK with a response body similar to:

    • PUT: the request will accept a request body similar to:


      where the setstatus field can be either interrupt-execution or interrupt-solve. The request will return 404 Not found when there is no such request, 405 Method not allowed when the setstatus field has an invalid value, or 200 OK with a status response body, with a setstatus field added with a value of interrupt-execution, interrupt-solve or interrupt-processed indicating whether the interrupt is scheduled, or already processed.

    • DELETE: the request will return a status code of 405 Method not allowed if the task is still running, or 200 OK if the task is still queued, interrupted, or already finished. When a task is deleted all associated resources, including all files containing the files contained request, response or intermediate status bodies will be deleted.

  • /api/v1/tasks/{id}/response

    • GET: will return a 404 Not found if there is no taks with the given id, or 200 OK with the final response body stored as stored in the file dex::api::RequestAttribute('response-data-path') by the service handler procedure.

  • /api/v1/tasks/{id}/status

    • GET: will return a 404 Not found if there is no taks with the given id, or 200 OK with an intermediate status response body stored as stored in the file dex::api::RequestAttribute('status-data-path') by the service handler procedure.