AsyncJob

From Ocean Framework Documentation Wiki
Jump to: navigation, search

The AsyncJob resource has two public URIs. (For information about the difference between Public and Internal URLs and HATEOAS conventions, please see API Structure and Conventions.)

/v1/async_jobs

The second one is used mainly by CronJobs:

/v1/async_jobs/cleanup

Structure

uuid
(String) The UUID of the AsyncJob.
steps
(Array) An array of job steps (see below).
default_step_time
(Integer) The maximum time a job step is assumed to take by default (this can be overridden for each separate job step). Default 30 seconds. If the job should fail for any reason, this is the amount of time it will take before the job is considered again.
default_poison_limit
(Integer) The number of job step restarts after which the whole job is to be regarded as poison and removed from the queue (the creator will be notified). Default: 5. The limit can be overridden per job step, and the count is cleared before each step starts.
max_seconds_in_queue
(Integer) The maximum number of seconds this AsyncJob may exist in the queue. Default: 1 day (86400 seconds).
destroy_at
(String) The datetime after which this AsyncJob may be deleted from the queue. (Sum of created_at and max_seconds_in_queue.) (read only)
credentials
(String) The username and password (Base64 encoded, cf. POST to Authentications) to use in re-authentication. This attribute is required when creating an AsyncJob, but will never appear in any external resource representation.
token
(String or null) The authentication token to use in all HTTP operations. This attribute is optional when creating an AsyncJob. It will never appear in any external resource representation. If not present, the service will use the credentials to obtain one; supplying an initial token relieves the job of doing this.
poison_email
(String) This email address, if present, will receive email notification containing the full AsyncJob (except the credentials) should it turn poison.
created_at
(String) The datetime when this AsyncJob was created. (read only)
updated_at
(String) The datetime when this AsyncJob was last updated. (read only)
started_at
(String) The datetime when this AsyncJob was first started. Not present if not yet started. (read only)
finished_at
(String) The datetime when this AsyncJob was finished. Not present if not yet finished. (read only)
last_completed_step
(Integer) The index of the last completed job step in steps or null if no job step yet has been completed. (read only)
succeeded
(Boolean) True if the job has completed successfully. (read only)
failed
(Boolean) True if the job has failed or been deemed poison. (read only)
poison
(Boolean) True if the job has been deemed poison. (read only)
last_status
(Integer) The most recent HTTP status code received during the execution of job steps. (read only)
last_headers
(Object) The most recent HTTP headers received during the execution of job steps. (read only)
last_body
(Object) The most recent HTTP body received during the execution of job steps. (read only)

Job steps

The steps attribute is an array of zero or more hashes. Each hash is a job step. Each step has zero or more of the following keys, with corresponding values:

Request parameters

name
(String) A name for the step. This will appear in logs.
url
(String) The URL to which to make the request. If this value isn't specified, the job step will not be executed.
method
(String) The HTTP method to use. Should be one of GET (the default), POST, PUT, or DELETE.
headers
(Hash) A hash of extra headers to add to the request, e.g. {"X-Custom": "0afb"}. Defaults to values for Content-Type, Accept and X-API-Token.
body
(String) The body of the request.

Step control

step_time
(Integer) The number of seconds after which this step will become executable by another worker. Typically set to the maximum time the job step is expected to take to complete plus some margin. If the job step takes longer, another worker will try to run it again. If the job step takes a shorter amount of time, the next step will be executed at once. Thus, a high value affects only rescheduling of the same step. Defaults to the value of default_step_time, which defaults to 30 seconds. The maximum value is 43200 seconds (12 hours).
poison_limit
(Integer) The number of times this job step may be retried before the entire job is regarded as poison and fails. Defaults to the value of default_poison_limit, which defaults to 5.

The following attributes collectively control retry intervals:

retry_base
(Number) Defaults to 1.0.
retry_multiplier
(Number) Defaults to 1.0.
retry_exponent
(Number) Defaults to 1.0.

The values are used according to the formula

(retry_base + ((receive_count - 1) * retry_multiplier) ** retry_exponent).ceil

and is thus always an integer. The longest retry interval is 43200 seconds (12 hours), as this is the limit supported by the Amazon Simple Queue Service.

The default settings will create a backoff starting with 1 second, then 2 seconds, then 3, 4, 5 and so on.

Logging

Some data is added to each job step during execution:

log
(Array of Strings) Job events pertaining to this step are logged here, in the order they occur. This includes things like restarts, redirects, exceptions, successes and failures.
receive_count
(Integer) The number of times this step has been attempted, starting with 1.

Examples

{"url": "https://api.example.com/v1/emails/314", "method": "DELETE"}
{"url": "https://api.example.com/v1/emails/314", "retry_exponent": 2.7}
{"url": "https://api.example.com/v1/users/11023/mail_new_password", "method": "PUT",
 "step_time": 60, "poison_limit": 3}
{"url": "https://api.example.com/v1/baskets", "method": "POST",
 "body": "<JSON string>"}
{"url": "https://api.example.com/v1/emails/314", "method": "DELETE",
 "poison_limit": 10,
 "receive_count": 4,
 "log": ["Faraday::Error::ConnectionFailed: Connection refused - connect(2)",
         "Faraday::Error::ConnectionFailed: Connection refused - connect(2)",
         "Remote server error: 503",
         "Succeeded: 200"]
}

Hyperlinks

The following hyperlink names are used in the following subsections. To retrieve the URL and Content-Type to use, use the _links hash. Note that the URI may change, and thus you should make no inference on how to construct similar URIs from any previous URI. For more information, refer to Hyperlink URIs are Opaque. You may store hyperlink URIs in permanent storage for later use.

The hyperlinks for an AsyncJob resource are:

self
as always, the self hyperlink can be used to perform CRUD actions on the AsyncJob resource.
creator
the ApiUser who created this AsyncJob.
updater
the ApiUser who last updated this AsyncJob.


GET self

Polls an AsyncJob resource for changes. For scalability, GET requests should have an If-None-Match HTTP header (cf. Conditional GETs).

200
The operation was successful.

DELETE self

Deletes an AsyncJob resource. Jobs may only be cancelled while they are waiting to start, or after they have completed.

204
The delete operation was successful.
422
The AsyncJob is in progress and may not be deleted.

POST /v1/async_jobs

Creates a new AsyncJob resource.

  • Required attributes:
    • credentials
    • steps
  • Optional attributes:
    • token
    • default_step_time
    • default_poison_limit
    • max_seconds_in_queue
201
The operation was successful and a new AsyncJob resource has been created. The response headers will, as is customary, include a standard Location header giving the URI of the newly created resource. To save you the extra request Ocean also returns the new AsyncJob resource in the response body.

PUT /v1/async_jobs/cleanup

Purges outdated AsyncJobs. The operation returns a statistics object in the response body, e.g.:

{"purged": 39, "remaining": 41207}
200
The operation was successful. The body contains a JSON structure describing the results.