Using Tidoop REST API
Content:
GET /tidoop/v1/version
POST /tidoop/v1/user/{userId}/jobs
GET /tidoop/v1/user/{userId}/jobs
GET /tidoop/v1/user/{userId}/jobs/{jobId}
DELETE /tidoop/v1/user/{userId}/jobs/{jobId}
NOTE: A X-Auth-Token
header has been included in all the requests assuming the API is protected by means of some kind of token-based authentication mechanism, such as OAUth2.
GET /tidoop/v1/version
Gets the running version of cosmos-tidoop.
Request example:
GET http://<tidoop_host>:<tidoop_port>/tidoop/v1/version HTTP/1.1
X-Auth-Token: 3bzH35FFLdapMgVCOdpot23534fa8a
Response example:
HTTP/1.1 200 OK
{
"success": "true",
"version": "0.2.0-next"
}
POST /tidoop/v1/user/{userId}/jobs
Runs a MapReduce job given the following parameters:
- Java jar containing the desired MapReduce application.
- The name of the MapReduce application.
- Arguments as a Json array (i.e. arguments separated by white spaces, enclosed by
[...]
). Leave it empty (""
) if no arguments are required apart from the input and the output directories.
Please observe if any of the arguments refers to a HDFS path (e.g. input or output directory), the absolute path including the HDFS endpoint is required.
Request example:
POST http://computing.cosmos.lab.fiware.org:12000/tidoop/v1/user/frb/jobs HTTP/1.1
Content-Type: application/json
X-Auth-Token: 3bzH35FFLdapMgVCOdpot23534fa8a
{
"jar": "/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar",
"class_name": "wordcount",
"args": ["hdfs://storage.cosmos.lab.fiware.org/user/frb/input","hdfs://storage.cosmos.lab.fiware.org/user/frb/output"]
}
Response example:
HTTP/1.1 200 OK
{
"success": "true",
"job_id": "job_1460639183882_0005"
}
GET /tidoop/v1/user/{userId}/jobs
Gets the details for all the MapReduce jobs run by the given user ID.
Request example:
GET http://computing.cosmos.lab.fiware.org:12000/tidoop/v1/user/frb/jobs HTTP/1.1
X-Auth-Token: 3bzH35FFLdapMgVCOdpot23534fa8a
Response example:
HTTP/1.1 200 OK
{
"success": "true",
"jobs": [{
"job_id": "job_1460639183882_0005",
"state": "SUCCEEDED",
"start_time": "1460963556383",
"user_id": "frb"
}, {
"job_id": "job_1460639183882_0004",
"state": "SUCCEEDED",
"start_time": "1460959583838",
"user_id": "frb"
}]
}
GET /tidoop/v1/user/{userId}/jobs/{jobId}
Gets the details for the given MapReduce job run by the given user ID.
Request example:
GET http://computing.cosmos.lab.fiware.org:12000/tidoop/v1/user/frb/jobs/job_1460639183882_0005 HTTP/1.1
X-Auth-Token: 3bzH35FFLdapMgVCOdpot23534fa8a
Response example:
HTTP/1.1 200 OK
{
"success": "true",
"job": {
"job_id": "job_1460639183882_0005",
"state": "SUCCEEDED",
"start_time": "1460963556383",
"user_id": "frb",
"stderr": "...",
"stdout": "..."
}
}
Please observe when getting details about a specific job, the stderr
and stdout
traces are returned.
DELETE /tidoop/v1/user/{userId}/jobs/{jobId}
Deletes the given MapReduce job run by the given user ID.
Request example:
DELETE http://computing.cosmos.lab.fiware.org:12000/tidoop/v1/user/frb/jobs/job_1460639183882_0005 HTTP/1.1
X-Auth-Token: 3bzH35FFLdapMgVCOdpot23534fa8a
Response example:
HTTP/1.1 200 OK
{
"success": "true"
}