Cloud API

Use Cases

To get you started, here are some common use cases for the Image Intelligence API.

Scenario 1: Find any person in an image

A small business owner wishes to be alerted whenever someone is on premise after hours. To avoid any false alarms and unnecessary call-outs, they want to have high confidence that a person appears within an image.

In this scenario the /detect endpoint may be used, which has a setting called human-in-the-loop or HITL. hitl dictates the accuracy of results by determining whether AI is used exclusively, or if human verification should be involved in the outcome. hitl may be set to NEVER, AUTO or ALWAYS. If set to AUTO, AI is used primarily with an additional human verification step triggered if the results fall below a certain threshold.

In this example the following request may be sent to the API, where person is included as a class with hitl set to AUTO.

curl -X POST \
  https://api.imageintelligence.com/v2/detect \
  -d '{
  "images": [
    {
      "url": "https://publicly.available.domain.net/image-001.jpg"
    },
    {
      "url": "https://publicly.available.domain.net/image-002.jpg"
    }
  ],
  "classes": [
    {
      "class": "person",
      "hitl": "AUTO"
    }
  ]
}'

If the API determined that a person was present in the images, jobResults in the API response will return the image chosen as containing a person.

Here is a simplified response from the API.

{
  "id": "d07878f9-f1d4-4c9e-8925-34cb887fc1e3",
  "createdAt": 1512960528023,
  "status": "COMPLETED_SUCCESSFULLY",
  "jobResults": [
    {
      "class": "person",
      "image": {
        "url": "https://publicly.available.domain.net/image-001.jpg"
      },
      "hitl": true
    }
  ],
  "imageResults": [
    {
      "id": "07dcfa6d-cc23-437d-994f-bc6a13e2db7c",
      "url": "https://publicly.available.domain.net/image-001.jpg",
      "objects": [
        {
          "class": "person",
          "confidence": 0.9995893836021423
        }
      ],
      "status": "COMPLETED_SUCCESSFULLY"
    },
    {
      "id": "650bf93a-23ad-41a4-81c2-2c4558b95b89",
      "url": "https://publicly.available.domain.net/image-002.jpg",
      "objects": [
        {
          "class": "person",
          "confidence": 0.32517221570014954
        }
      ],
      "status": "COMPLETED_SUCCESSFULLY"
    }
  ]
}

Scenario 2: Find a specific person in an image

A parent at work wishes to be alerted whenever their child arrives home from school.

In this scenario, the /match endpoint may be used whereby a set of images is sent along with up to three target images of the child to match against. Note that this endpoint always employs human-in-the-loop or HITL to ensure the results are verified by a person.

curl -X POST \
  https://api.imageintelligence.com/v2/match \
  -d '{
  "images": [
    {
      "url": "https://publicly.available.domain.net/front-door-001.jpg"
    },
    {
      "url": "https://publicly.available.domain.net/front-door-002.jpg"
    }
  ],
  "target": {
    "class": "person",
    "images": [
      "https://publicly.available.domain.net/child-001.jpg",
      "https://publicly.available.domain.net/child-002.jpg"
    ]
  }
}'

If the API determined that the child was present in the images, jobResults in the API response will return the image chosen as containing the child.

Here is a simplified response from the API.

{
  "id": "5d689c71-e68f-46c3-ac71-4053806e71de",
  "createdAt": 1487648348,
  "status": "COMPLETED_SUCCESSFULLY",
  "jobResults": [
    {
      "images": [
        {
          "url": "https://publicly.available.domain.net/front-door-002.jpg"
        }
      ],
      "target": {
        "class": "person",
        "images": [
          "https://publicly.available.domain.net/child-001.jpg",
          "https://publicly.available.domain.net/child-002.jpg"
        ]
      },
      "hitl": true
    }
  ]
}

Scenario 3: Find when an unknown car is in a driveway

A homeowner wishes to be alerted whenever an unknown car is parked in their driveway.

In this scenario, the /ask endpoint may be used where the question Is there an unknown car in my driveway? is sent along with a set of images. To avoid confusion, example images of the owners own car may also be included.

curl -X POST \
  https://api.imageintelligence.com/v2/ask \
  -d '{
  "question": "Is there an unknown car in my driveway?",
  "images": [
    {
      "url": "https://publicly.available.domain.net/driveway-001.jpg"
    },
    {
      "url": "https://publicly.available.domain.net/driveway-002.jpg"
    }
  ],
  "examples": {
    "yes": [
      "https://publicly.available.domain.net/unknown-car-in-driveway.jpg",
    ],
    "no": [
      "https://publicly.available.domain.net/my-car.jpg",
      "https://publicly.available.domain.net/my-car-in-driveway.jpg",
    ]
  }
}'

If the API determined that the answer to the question is true, jobResults in the API response will return the image chosen.

Here is a simplified response from the API.

{
  "id": "5d689c71-e68f-46c3-ac71-4053806e71de",
  "createdAt": 1487648348,
  "status": "COMPLETED_SUCCESSFULLY",
  "question": "Is there an unknown car in my driveway?",
  "jobResults": {
    "image": {
      "url": "https://publicly.available.domain.net/driveway-002.jpg"
    }
  }
}

Scenario 4: Detecting and recognizing faces

A small business owner has security footage of shoplifters and want to be alerted if the same shoplifter enters their establishment. This can be achieved through our recognition API.

curl -X POST \
  https://api.imageintelligence.com/v2/recognize \
  -d '{
  "images": [
    {
      "url": "https://publicly.available.domain.net/image-001.jpg"
    },
    {
      "url": "https://publicly.available.domain.net/image-002.jpg"
    }
  ]
}'

Each image in the images field represents a frame in the security footage where the face of the shoplifter is clearly visible.

{
  "id": "5d689c71-e68f-46c3-ac71-4053806e71de",
  "createdAt": 1487648348,
  "status": "COMPLETED_SUCCESSFULLY",
  "customId": "6c78df0a-67b6-4d5f-93cf-5820cfee501c",
  "feedId": "Camera_123456",
  "webhookUrl": "https://publicly.available.domain.net/90f8754aa4e287bbb2c07ec99ade0de72ddbd4b6Et",
  "images": [
    {
      "id": "3534852e-9d44-48d1-b42a-9d14f3652032",
      "url": "https://images.example.com/5673605764397232.jpg",
      "proxyUrl": "https://proxy.images.example.com/5673605764397232.jpg",
      "customId": "a7e0bc2d-3226-46da-b242-9bb40666b14a",
      "objects": [
        {
          "boundingBox": {
            "xMin": 0.23873,
            "yMin": 0.23873,
            "xMax": 0.75087,
            "yMax": 0.96537
          }
        },
        {
          "boundingBox": {
            "xMin": 0.23873,
            "yMin": 0.23873,
            "xMax": 0.75087,
            "yMax": 0.96537
          }
        }
      ]
    }
  ]
}

If faces were found in the supplied images, the imageId and a boundingBox representing the location of the detected face will be returned for each face found. Using the imageIds, you can create an identity to represent the shoplifter.

curl -X POST \
  https://api.imageintelligence.com/v2/identities \
  -d '{
  "name": "Shoplifter 1",
  "imageIds": [
    "ced2d464-0567-480d-9616-092d2895cceb",
    "c3c61384-1861-4026-be2e-4d70646231c0"
  ]
}'

Now that an identity has been created, subsequent calls to /recognize will return an identityId and name if we think it's the same person. objects in the response might look like:

{
  "objects": [
    {
      "identityId": "81e45841-4996-433d-9138-3383214176d1",
      "name": "Shoplifter 1",
      "confidence": 0.410791,
      "boundingBox": {
        "xMin": 0.23873,
        "yMin": 0.23873,
        "xMax": 0.75087,
        "yMax": 0.96537
      }
    }
  ]
}