http://zorba.io/modules/http-client

View as XML or JSON.

This module provides functions for performing HTTP requests.

A simple GET request using the get#1 convenience function

 import module namespace http="http://zorba.io/modules/http-client";
 http:get("http://www.example.com")
 

This example makes a GET request to example.com and returns the server's response as a JSON object.

 {
   "status" : 200,
   "message" : "OK",
   "headers" : {
     "Content-Length" : "1270",
     "Date" : "Tue, 11 Jun 2013 22:27:10 GMT",
     ...
   },
   "body" : {
     "media-type" : "text/html",
     "content" : "..."
   }
 }
 

Response format

Most functions in this module (all except options#1) return a single JSON item, describing the server's response, as in the previous example. The server status (integer) and message (string) fields are always present. If the server replied sending one or more headers, they are reported in an optional headers object. Each header is represented as a single (string) field.

For non-multipart responses, as in the previous example, the response body, if any, is reported as a body object. This object contains both the (string) media-type returned by the server and its content. The type of the content field is determined by the media-type returned by the server. If the media-type indicates that the body content is textual, then the content has type string, base64Binary otherwise. Specifically, the body content is considered textual only if the MIME-type specified in the media-type is one of:

  • "application/json"
  • "application/x-javascript"
  • "application/xml"
  • "application/xml-external-parsed-entity"
or if the MIME-type starts with "text/" or ends with "+xml".

For multipart responses, multiple bodies are returned, as in the following example:

 {
   "status" : 200,
   "message" : "OK",
   "headers" : {
     "Date" : "Tue, 11 Jun 2013 22:34:13 GMT",
     ...
   },
   "multipart" : {
     "boundary": "--AaB03x",
     "parts": [
       {
         "headers" : {
            "Content Disposition: file",
            ...
         },
         "body": {
           "media-type" : "image/gif",
           "content" : "..."
         }
       },
       {
         "body" : {
           "media-type" : "text/html",
           "content" : "..."
         }
       }
    ]
 }
 

The multipart field contains both the boundary used to separate parts and an array containing all parts. Each part contains its specific headers, if any, and the corresponding body.

Important Notice Regarding Nondeterministic Functions

The following functions in this module - get#1, get-text#1, get-binary#1, send-nondeterministic-request-1, head#1, and options#1 are declared to be nondeterministic, which means that their results will not be cached. However, they are not declared to be sequential, which means that they may be re-ordered during query optimization. According to the HTTP RFC, GET, HEAD an OPTIONS requests should not have any side-effects. However, in practice it is not uncommon, especially for GET requests, to have side-effects. If your application depends on the ordering of side-effects from requests issued through these functions, you should either use the send-request() function (which is declared sequential), or alternatively wrap each call to get() in your own sequential function, to ensure that the requests are not reordered.

$href Arguments to Functions

Several functions in this module accept a URL argument named $href. In all cases, the value passed to $href must be a valid anyURI. However, all functions declare $href to be of type string. This is for convenience, since you can pass a string literal value (that is, a URL in double-quotes spelled out explicitly in your query) to an string parameter.

Relation to the EXPath http-client module

EXPath defines its own http-client module, which is available separately. There are two primary differences between EXPath's http-client and this module:

  1. EXPath defines only the send-request() function, although it does include convenient 1- and 2-argument forms in addition to the full 3-argument form. EXPath does not include the simpler get(), post(), put(), delete(), head(), and options() functions defined by this module.
  2. EXPath uses XML to represent request for its send-request() function, whereas this module uses JSON.
  3. EXPath specifies that all XML content returned by an HTTP server will be parsed and returned as an XML document, whereas all HTML content will be tidied up into valid XML, and then parsed into an element. This module returns any textual content as string and any binary content as base6Binary.
  4. EXPath accepts XML nodes as body in the send-request() function and automatically serializes them into a string. The send-request() function defined in this module only allows string, base64Binary, and hexBinary as body types.

See the full spec of the EXPath http-client module for more information.

Function Summary

delete ($href as string) as object()

This function makes an HTTP DELETE request to a given URL.

get-binary ($href as string) as object()

This function makes a GET request on a given URL.

get-text ($href as string) as object()

This function makes a GET request to a given URL.

get ($href as string) as object()

This function makes a GET request to a given URL.

head ($href as string) as object()

This function makes an HTTP HEAD request on a given URL.

options ($href as string) as string*

This function makes an HTTP OPTIONS request, which asks the server which operations it supports.

post ($href as string, $body as atomic) as object()

This function makes an HTTP POST request to a given URL.

post ($href as string, $body as atomic, $content-type as string) as object()

This function makes an HTTP POST request to a given URL.

put ($href as string, $body as atomic) as object()

This function makes an HTTP PUT request to a given URL.

put ($href as string, $body as atomic, $content-type as string) as object()

This function makes an HTTP PUT request to a given URL.

send-nondeterministic-request ($request as object()) as object()

This function sends an HTTP request and returns the corresponding response.

send-request ($request as object()) as object()

This function sends an HTTP request and returns the corresponding response.

Functions

delete#1

declare  %an:sequential function http:delete($href as string) as object()

This function makes an HTTP DELETE request to a given URL.

Parameters

href as string
The URL to which the request will be made (see note above).

Returns

object()
standard http-client return type.

get-binary#1

declare  %an:nondeterministic function http:get-binary($href as string) as object()

This function makes a GET request on a given URL. All returned bodies are forced to be interpreted as binary data, and will be returned as base64Binary items.

Parameters

href as string
The URL to which the request will be made (see note above).

Returns

object()
standard http-client return type.

get-text#1

declare  %an:nondeterministic function http:get-text($href as string) as object()

This function makes a GET request to a given URL. All returned bodies are forced to be interpreted as textual, with a UTF-8 charset and will be returned as string items.

Parameters

href as string
The URL to which the request will be made (see note above).

Returns

object()
standard http-client return type.

get#1

declare  %an:nondeterministic function http:get($href as string) as object()

This function makes a GET request to a given URL.

Parameters

href as string
The URL to which the request will be made (see note above).

Returns

object()
standard http-client return type.

head#1

declare  %an:nondeterministic function http:head($href as string) as object()

This function makes an HTTP HEAD request on a given URL.

Parameters

href as string
The URL to which the request will be made (see note above).

Returns

object()
standard http-client return type.

options#1

declare  %an:nondeterministic function http:options($href as string) as string*

This function makes an HTTP OPTIONS request, which asks the server which operations it supports.

Parameters

href as string
The URL to which the request will be made (see note above).

Returns

string*
A sequence of string values of the allowed operations.

post#2

declare  %an:sequential function http:post($href as string, $body as atomic) as object()

This function makes an HTTP POST request to a given URL.

The body passed to this function must be either a string, a base64Binary, or an hexBinary. If it is a string, the Content-Type sent to the server will be "text/plain", "application/octet-stream" otherwise.

Parameters

href as string
The URL to which the request will be made (see note above).
body as atomic
The body which will be sent to the server.

Returns

object()
standard http-client return type.

post#3

declare  %an:sequential function http:post($href as string, $body as atomic, $content-type as string) as object()

This function makes an HTTP POST request to a given URL.

The body passed to this function must be either a string, a base64Binary, or an hexBinary. In any case, Content-Type of the request sent to the server will be $content-type.

Parameters

href as string
The URL to which the request will be made (see note above).
body as atomic
The body which will be sent to the server
content-type as string
The content type of the body as described above.

Returns

object()
standard http-client return type.

put#2

declare  %an:sequential function http:put($href as string, $body as atomic) as object()

This function makes an HTTP PUT request to a given URL.

The body passed to this function must be either a string, a base64Binary or an hexBinary. If it is a string, the Content-Type sent to the server will be "text/plain", "application/octet-stream" otherwise.

Parameters

href as string
The URL to which the request will be made (see note above).
body as atomic
The body which will be sent to the server.

Returns

object()
standard http-client return type.

put#3

declare  %an:sequential function http:put($href as string, $body as atomic, $content-type as string) as object()

This function makes an HTTP PUT request to a given URL.

The body passed to this function must be either a string, a base64Binary, or an hexBinary. In any case, Content-Type of the request sent to the server will be $content-type.

Parameters

href as string
The URL to which the request will be made (see note above).
body as atomic
The body which will be sent to the server.
content-type as string
The content type of $body to send to the server.

Returns

object()
standard http-client return type.

send-nondeterministic-request#1

declare  %an:nondeterministic function http:send-nondeterministic-request($request as object()) as object()

This function sends an HTTP request and returns the corresponding response.

This function has the same semantics of send-request-1, but is declared as nondeterministic and thus should only be used when the request has no side-effects.

Parameters

request as object()
see request parameter of send-request#1

Returns

object()
standard http-client return type.

send-request#1

declare  %an:sequential function http:send-request($request as object()) as object()

This function sends an HTTP request and returns the corresponding response.

This function is declared as sequential and should be used whenever the request may have side-effects.

The request parameters are specified in the $request JSON object, which has the following minimal structure:

   {
     "href": "http://www.example.com"
   }
 

This object specifies a GET request of the URI "http://www.example.com"

Additional optional parameters can be specified when issuing a request, using the following structure:

  {
    "method": "POST",
    "href": "http://www.example.com",
    "authentication":
    {
      "username" : "user",
      "password" : "pass",
      "auth-method" : "Basic"
    },
    "options":
    {
      "status-only": true,
      "override-media-type": "text/plain",
      "follow-redirect": false,
      "timeout": 30,
      "user-agent": "Mozilla/5.0"
    },
    "headers":
    {
      "name": "value",
      ...
    },
    "body":
    {
      "media-type": "text/plain",
      "content": "..."
    }
  }

The method field (string) defines the HTTP verb to use in the HTTP request (i.e., GET, HEAD, OPTIONS, POST, PUT, DELETE). If not specified GET will be used. The authentication field can be used to specify the credentials and authentication method used when issuing a request (e.g, Basic). If the authentication field is specified, all its (string) subfields must be specified. If an authentication object is provided, it overrides any Authorization header specified in the request. Additionally, the following options can be specified:

  • status-only. If true, the response body contents are omitted from the response object.
  • override-media-type. Is a MIME type that will override the Content-Type header returned by the server. It affects the type of the result body content.
  • follow-redirect. Control whether an http redirect is automatically followed or not. If it is false, the http redirect is returned as the response. If it is true (the default) the function tries to follow the redirect, by sending the same request to the new address (including body, headers, and authentication credentials.) Maximum one redirect is followed (there is no attempt to follow a redirect in response to following a first redirect).
  • timeout. Is the maximum number of seconds to wait for the server to respond. If no response is received withing this time duration, an error is thrown.
  • user-agent. The user agent sent to the server when issuing the request. If not specified libcurl-agent/1.0 is used.

One or more headers can be sent to the server, specifying them in an optional headers object. Each header is represented as a single (string) field. These headers are overridden if the corresponding option/authentication has been specified in the request.

For non-multipart request a body object can be specified. This object must contain both the desired (string) media-type and its content. The type of the content field must be either string, base64Binary, or hexBinary.

For multipart requests, multipart object can be specified in place of the body object. The multipart object has the following structure:

  "multipart" : {
    "boundary": "--AaB03x",
    "parts": [
      {
        "headers" : {
           "Content Disposition: file",
           ...
        },
        "body": {
          "media-type" : "image/gif",
          "content" : "..."
        }
      },
      {
        "body" : {
          "media-type" : "text/html",
          "content" : "..."
        }
      }
   ]
 }
 

The multipart field contains an optional (string) field which specifies the boundary used to separate each part and an array containing all parts. Each part contains its specific headers, if any, and the corresponding body.

Parameters

request as object()
a JSON http-client request object

Returns

object()
standard http-client return type.