Worker-Service

Experiments  ¶

The /experiments command is used to retrieve all running experiments for the platform.

The response:

Content-Type: application/json

{
  "experiments": [
    {
      {
        "id" : 1,
        "title" : "example title",
        "description": "example description"
      },
      {
        "id" : 3,
        "title" : "another experiment",
        "description": "with another description"
      }
    }
  ]
}

Next  ¶

The /next command is crucial for the worker-service. It instructs the requestor what to display next and consequently what the worker should be working on. Every response has a type which describes the result. There are 5 different types: FINISHED, ANSWER, RATING, CALIBRATION and EMAIL. FINISHED means there are no more assignments left, the worker can be redirected to the crowdworking-platform. ANSWER represents an Creative-Task, the client is expected to present the experiment to the worker and let the worker create one or more answers to it. RATING should present the experiment and display the answers of other workers. The worker can now rate these answers. CALIBRATION expects the client to present the returned questions and let him choose the answer from pre-defined fields. EMAIL represents the need of an email-address from a worker. Additional query-parameters may be required for certain platforms.

The response:

The worker-service supports multiple sessions. Depending on the time-limit of the platform and platform-specific properties, the worker can work on an experiment, finish and later start again.

The protobuf definition of the resource can be viewed here;

Full details about the types in the next segments:

Content-Type: application/json

{
  "authorization" : "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiYWRtaW4iOnRydWV9.TJVA95OrM7E2cBab30RMHrHDcEfxjoYZgeFONFh7HgQ",
  "type": "CALIBRATION",
  "calibrations": [
    {
      "question": "did you attend to an university?",
      "answerOptions": [
        {
          "option" : "yes",
          "id" : 23
        },
        {
          "option" : "no",
          "id" : 24
        }
      ],
      "id": 4,
    }
  ]
}

Example: Next EMAIL  ¶

Scenario: the platform needs an email from his workers and it is the first time the worker is working on our framework. Therefore the example-platform will not find us in the database and the worker-service proceeds to respond with the EMAIL type. The query-parameter exampledependent=13 represents an platform-dependent query-parameter, that will be passed further into the system.

Expected Behavior: Ask the Worker for his email-address and submit it. Then call /next with the jwt obtained through the submit email request.

Content-Type: application/json

{
  "type": "EMAIL"
}

Example: Next CALIBRATION  ¶

Scenario: the example-platform has the displaying of calibrations activated, can identify the worker from the passed, platform-dependent query parameter and the worker has already worker with our framework. The example-platform now finds the matching worker-id in the database and returns it to the worker-service. The worker-service now notices that the worker has not answered all the calibrations, so it returns the type CALIBRATION and an array of calibrations as calibrations. The query-parameter exampledependent=15 represents an platform-dependent query-parameter, that will be passed further into the system.

Expected Behavior: Let the worker choose his answers for all the calibrations and submit them with /calibrations, then call /next with the worker-id as an parameter. Calling /next without submitting all the calibrations will result in the type CALIBRATION, where the field calibrations holds all the remaining calibrations.

Content-Type: application/json

{
  "workerId" : 3,
  "type": "CALIBRATION",
  "calibrations": [
    {
      "question": "did you attend to an university?",
      "answerOptions": [
        {
          "option" : "yes",
          "id" : 23
        },
        {
          "option" : "no",
          "id" : 24
        }
      ],
      "id": 4,
      }
  ]
}

Example: Next ANSWER  ¶

Scenario: The worker has completed all calibrations and the worker-service decides that he can work on an creative-Task. So it returns the type ANSWER and all the relevant information about the experiment(title, description). The experiment has some pictures and some constraints, so additionally it also adds these. The worker-service also returns answer_reservations, which specifies how many creative-answers for the worker are left and the reservation id for each answer. The worker-service also passes an answerType.

Expected Behavior: The client is expected to render the title, description and the pictures. Additionally the worker has to be warned about the constraints. The worker has now the chance to create up to maxAnswersToGive answers and the client should submit them via /answers. After submitting the client should call /next. When the worker-service passes an answerType, the answer to the creative task is a url, pointing to a resource with the mime-type of answerType. It is expected that the client verifies that the answer is indeed a valid url, but it is not expected that it checks whether the mime-type matches the resource the url is pointing to.

Content-Type: application/json

{
  "type": "ANSWER",
  "title": "Find a specific Picture.",
  "description": "This is the description of the example-task. It specifies what the worker has to do.",
  "answer_reservations" : [13, 66, 67, 68],
  "answerType" : "images"
  "pictures": [
      {
        "url": "http://example.picture.jpg",
        "urlLicense": "http://www.source.de"
      }
  ],
  "constraints" : [
    {
      "id" : 5,
      "name" : "The tweet must not be racist."
    }
  ]
}

Example: Next RATING  ¶

Scenario: The example-platform does not render calibrations and the worker-service decides that the worker should do a rating-task. Therefore the type is RATING and all the relevant information about the experiment(title, description) is set. The experiment has also some constraints, so the worker-service passes them too via the field constraints. The worker-service also returns answersToRate and ratingOptions. The field pictures may be set if the experiment provides pictures, but in this example is not. Please compare Next Answer for an example where the field pictures is set.

Expected Behavior: The client is expected to render the title and the description of the experiment. Additionally the calibrations have to be placed prominently. The worker has now the chance to rate the passed answers (answersToRate). For each answer he can choose one of the rating-options (ratingOptions) and specify which constraints (if any) got violated. One rating-option is a description/value pair. The description should be rendered for the worker and the value represents the chosen rating. The worker should also be encouraged to submit a feedback containing a critique of the answer. The client should submit them via /ratings. After submitting the client should call /next.

Content-Type: application/json

{
  "type": "RATING",
  "title": "Come up with a joke!",
  "description": "Come up with a joke about the following situation....",
  "answers": [
    {
      "id": 12,
      "answersID": 15,
      "answer": "Lame joke"
    },
    {
      "id": 18,
      "answersID": 87,
      "answer": "This is another lame joke"
    }
  ],
  "constraints": [
    {
      "id": 55,
      "name": "the joke must not be racist."
    }
  ],
  "ratingOptions": [
    {
      "value": 0,
      "description": "bad"
    },
    {
      "value": 10,
      "description": "good"
    }
  ]
}

Example: Next FINISHED  ¶

Scenario: The worker-service requested 5 ratings from the worker, but the worker only worked on 3. In this situation the query-parameter rating=skip can be passed, to signal that the worker wants to skip further ratings. Otherwise every new /next call would result in the type RATING with 2 answers to rate. The worker-service now detects that there is nothing else to do, so he returns the type FINISHED.

Expected Behavior: The client is expected to notice the worker that his work is finished and redirects him to the crowdworking-platform.

Content-Type: application/json

{
  "type": "FINISHED"
}

Preview  ¶

this call returns a preview of the experiment.

The protobuf definition of the resource can be viewed here;

Please look at the /next command to get a better understanding of the /preview command.

The response:

Content-Type: application/json

{
  "type": "ANSWER",
  "title": "Find a specific Picture.",
  "description": "This is the description of the example-task. It specifies what the worker has to do.",
  "answerType" : "images"
  "pictures": [
      {
        "url": "http://example.picture.jpg",
        "urlLicense": "http://www.source.de"
      }
  ],
  "constraints" : [
    {
      "id" : 5,
      "name" : "The tweet must not be racist."
    }
  ]
}

Email  ¶

Some crowd-platforms require the email-address of the worker. Most of the time this is required because the platform has no native payment system and has to default to the built-in email based payment system in CrowdControl. If a worker wishes to not disclose his email-address, pass an empty string. This has the consequence that CrowdControl is unable to pay the worker. The response is a json-object containing the jwt.

The protobuf definition of the resource can be viewed here;

The protobuf definition of the answer can be viewed here;

The message:

Content-Type: application/json

{
  "email": "example.address@example.org"
  "platform_parameters" : [
    "key" : "exampleKey",
    "value" : ["examplevalue1", "examplevalue2"]
  ]
}

Content-Type: application/json

{
  "authorization": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiYWRtaW4iOnRydWV9.TJVA95OrM7E2cBab30RMHrHDcEfxjoYZgeFONFh7HgQ"
}

Content-Type: application/json

{
  "email": ""
}

Content-Type: application/json

{
  "authorization": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiYWRtaW4iOnRydWV9.TJVA95OrM7E2cBab30RMHrHDcEfxjoYZgeFONFh7HgQ"
}

Answer  ¶

This command is used to submit a creative answer. A creative answer is the work on the /next with the type ANSWER.

The protobuf definition of the answer can be viewed here;

The message:

Content-Type: application/json

{
  "answer": "http://www.example.org/image.jpg",
  "experiment": 33,
  "reservation": 44
}

Content-Type: application/json

{
  "answer": "The picture contains 3 red circles.",
  "experiment": 18,
  "reservation": 45
}

RATING  ¶

This command is used to submit a rating. A rating is the work on the /next with the type RATING.

The protobuf definition of the rating can be viewed here;

The message:

Content-Type: application/json

{
  "rating": 0,
  "experiment": 15,
  "answerId": 22244,
  "feedback": "this is racist!",
  "constraints": [
    13,
    18
  ]
}

Content-Type: application/json

{
  "rating": 30,
  "experiment": 15,
  "answerId": 22244
}

Calibration  ¶

This command is used to submit a calibration. A calibration is the work on the /next with the type CALIBRATION.

The protobuf definition of the calibration can be viewed here;

The message:

Content-Type: application/json

{
  "answerOption": 29
}

Content-Type: application/json

{
  "answerOption": 88
}