Overview
Smilart Web API is a set of several API services which provide necessary functions to use Smilart recognition algorithms when you create your own Web application.
List of services included into Smilart API:
-
Person Service — person management service implements basic operations on persons, such as adding, retrieving, removing and updating person in Platform database.
-
Camera Service — camera management service implements basic operations on cameras in Platform.
-
Video Content Analytics Service (VCA Service) — provides access to some events generated by Platform services, such as frame streams from cameras, face detection results and identification result.
-
Photo Booth Service — selects best frames from the camera stream, in order to put them into Platform database to achieve best identification results.
-
Verification Service — implements verification case.
-
Instant Photo Analytics Service — the service for instant photo analysis.
-
Adaptive Verification Service — provides opportunities to manage Adaptive Verification (AV) service.
Platform receives frames from connected cameras. Each camera can be either physical device or software emulation, and must implement one of the supported streaming protocols.
All services share common base of persons that used for face recognition.
Common protocol description
Smilart Web API uses HTTP and WebSockets as a transport layer for communication and JSON as a payload.
WebSockets heartbeat
After the handshake, the server will periodically send ping messages to the client.
When the ping is received, the recipient must send back a pong as soon as possible.
The server use this mechanism to make sure that the client is still alive. It is RECOMMENDED that client also send pings for checking a server.
A ping or pong is just a regular frame, but it is a control frame.
Pings have an opcode of 0x9
, and pongs have an opcode of 0xA
.
When you get a ping, send back a pong with the exact same Payload Data as the ping (for pings and pongs, the max payload length is 125).
You might also get a pong without ever sending a ping; ignore this if it happens.
Most WebSockets libraries already have PING/PONG mechanism.
Root Endpoint
By default root endpoint for Smilart Web API is located at /api
. The default port is 9999,
The default URL for Person Service, Camera Service, IPA Service: http://<yourInstance>:9999/api
and ws://<yourInstance>:9999/api
for other services.
Any Smilart Web API Resource endpoints are relative to these URLs, unless otherwise noted.
Resources Summary
Person Collection
For Person Collection Resource details, see the resource representation.
Method |
HTTP Request |
Description |
|
Lists all persons in base. |
|
|
Deletes all persons from base. |
Person
For Person Resource details, see the resource representation.
Method |
HTTP Request |
Description |
|
Adds new person into base with auto-generated identifier. |
|
|
Adds new person into base with identifier specified by client. |
|
|
Retrieves the person from base. |
|
|
Removes the person from base. |
Person Photo Collection
For Person Photo Collection Resource details, see the resource representation.
Method |
HTTP Request |
Description |
|
Lists all photos for the person. |
|
|
Removes all photos associated with the person. |
Person Photo
For Person Photo Resource details, see the resource representation.
Method |
HTTP Request |
Description |
|
Adds photo to the person. |
|
|
Adds photo with specified identifier to the person. |
|
|
Receives photo from the person. |
|
|
Receives binary JPEG photo from the person. |
|
|
Removes photo from the person. |
Camera Collection
For Cameras Resource details, see the resource representation.
Method |
HTTP Request |
Description |
|
Lists all cameras. |
Camera Info
For Camera Info Resource details, see the resource representation.
Method |
HTTP Request |
Description |
|
Receives the camera information from the system. |
Camera MJPEG
For Camera MJPEG Resource details, see the resource representation.
Method |
HTTP Request |
Description |
|
Starts subscription on the camera MJPEG stream. |
VCA Service
For VCA Service description, see the details.
Method |
HTTP Request |
Description |
|
Subscribes to the events of VCA Service. |
Photo Booth Service
For Photo Booth Service description, see the details.
Method |
HTTP Request |
Description |
|
Starts frames selection from the camera. |
Verification Service
For Verification Service description, see the details.
Method |
HTTP Request |
Description |
|
Starts verification process from the camera. |
IPA Service
For IPA Service description, see the details.
Method |
HTTP Request |
Description |
|
Correlates image with base. |
Adaptive Verification Service
For Adaptive Verification Service description, see the details.
Method |
HTTP Request |
Description |
|
Gets the service configuration. |
|
|
Sets the service configuration. |
|
|
Removes all sampled photos. |
|
|
Removes all sampled photos of the person. |
|
|
Removes all sampled photos from the camera for every person. |
Resources Reference
Person Collection
Overview
Represents collection of persons in base.
Resource Representation
Content-Type: application/vnd.com.smilart.helios.persons+json
JSON array of string identifiers of persons in base.
Methods
ListPersons
Lists all persons in base.
Request
GET /persons
Parameter Name |
Value Type |
Default Value |
Description |
|
string |
not provided |
Option for filtering the list of persons by substring in the person identifier. Case insensetive. |
|
integer |
not provided |
Option to limit the number of identifiers. |
|
integer |
not provided |
Option to offset over the specified number of identifiers. |
Request Body
Do not supply a request body with this method.
Response
- Persons listed
-
Status code:
200 OK
.
Headers:X-Smilart-TotalPersons
Body: representation of the resource.
Property Name |
Value Type |
Description |
|
integer |
Number of all or filtered persons in the base. |
DeletePersons
Deletes all persons in base.
Request
DELETE /persons
Request Body
Do not supply a request body with this method.
Response
- Persons deleted
-
Status code:
204 No Content
.
Person
Overview
Represents person. Person has a (possibly empty) collection of processed photos.
Resource Representation
Content-Type: application/vnd.com.smilart.helios.person+json
{
"id":string,
"creationTime":integer,
"modificationTime":integer
}
Property Name |
Value Type |
Description |
|
string |
Identifier of the person. |
|
integer |
Person creation time. |
|
integer |
Last person modification time in ms. It is updated on every modification of any person field, include initial person creation. |
Methods
AddPerson
Adds new person into base with auto-generated identifier.
Request
POST /persons
Request Body
Do not supply a request body with this method.
Response
- Person added
-
Status code:
201 Created
.
Body: representation of the resource.
AddPersonWithId
Adds new person into base with specified identifier.
Request
PUT /persons/{personId}
Parameter Name |
Value Type |
Description |
|
string |
Client-generated identifier of new person. Non empty string. Max lenght is 50. ASCII symbols with codes [32, 126]. Should be unique for the person. |
Request Body
Do not supply a request body with this method.
Response
- Person added
-
Status code:
201 Created
.
Body: representation of the resource. - Person with specified identifier already exists
-
Status code:
409 Conflict
. - PersonId validation error
-
Status code:
400 Bad Request
.
GetPerson
Gets person from base.
Request
GET /persons/{personId}
Parameter Name |
Value Type |
Description |
|
string |
Identifier of the person. |
Request Body
Do not supply a request body with this method.
Response
- Person provided
-
Status code:
200 OK
.
Body: representation of the resource. - Person not found
-
Status code:
404 Not Found
.
DeletePerson
Deletes person from base.
Request
DELETE /persons/{personId}
Parameter Name |
Value Type |
Description |
|
string |
Identifier of the person. |
Request Body
Do not supply a request body with this method.
Response
- Person deleted
-
Status code:
204 No Content
. - Person not found
-
Status code:
404 Not Found
.
Person Photo Collection
Overview
Represents collection of photos for the person in base.
Resource Representation
Content-Type: application/vnd.com.smilart.helios.photos+json
JSON array of string identifiers of photos for the person in base.
Methods
ListPersonPhotos
Lists photos of the person.
Request
GET /persons/{personId}/photos
Parameter Name |
Value Type |
Description |
|
string |
Identifier of the person. |
Request Body
Do not supply a request body with this method.
Response
- Photos listed
-
Status code:
200 OK
.
Body: representation of the resource. - Person not found
-
Status code:
404 Not Found
.
DeletePersonPhotos
Deletes all photos of the person.
Request
DELETE /persons/{personId}/photos
Parameter Name |
Value Type |
Description |
|
string |
Identifier of the person. |
Request Body
Do not supply a request body with this method.
Response
- Photos deleted
-
Status code:
204 No Content
. - Person not found
-
Status code:
404 Not Found
.
Person Photo
Overview
Represents photo of the person in base.
Resource Representations
JPEG photo of the person as multipart/form-data
Content-Type: multipart/form-data; boundary={your boundary}
{your boundary}\r\n
Content-Type: image/jpeg\r\n\r\n
{binary jpeg}
{your boundary}\r\n
Meta information about person photo
Content-Type: application/vnd.com.smilart.helios.photo+json
{
"id":string,
"creationTime":integer,
"autoSampled":boolean
}
Property Name |
Value Type |
Description |
|
string |
Identifier of the person’s photo. |
|
integer |
Photo creation time in ms. |
|
boolean |
True if this photo was sampled by Adaptive Verification service during self-learning process, other false. |
Methods
AddPersonPhoto
Adds binary representation of a photo to the person.
Only JPEG images are supported.
Request
POST /persons/{personId}/photos
Parameter Name |
Value Type |
Description |
|
string |
Identifier of the person. |
Request Body
- Content-Type
-
multipart/form-data; boundary={your boundary}
- Payload
-
Representation of the resource
Response
- Photo added
-
Status code:
201 Created
.
Content-Type:application/vnd.com.smilart.helios.photo+json
Body: representation of the resource. - Face not found in the photo
-
Status code:
422 Unprocessable Entity
.
Body: No face. - Payload validation error
-
Status code:
400 Bad Request
. Body: reason. - Person not found
-
Status code:
404 Not Found
.
AddPersonPhotoWithId
Adds binary representation of a photo to the person with specified identifier.
Only JPEG images are supported.
Request
PUT /persons/{personId}/photos/{photoId}
Parameter Name |
Value Type |
Description |
|
string |
Identifier of the person. |
|
string |
Client-generated identifier of new photo. Non empty string. Max lenght is 50. ASCII symbols with codes [32, 126]. Should be unique for the photo for each person. |
Request Body
- Content-Type
-
multipart/form-data; boundary={your boundary}
- Payload
-
Representation of the resource
Response
- Photo added
-
Status code:
201 Created
.
Content-Type:application/vnd.com.smilart.helios.photo+json
Body: representation of the resource. - Photo with specified identifier already exists
-
Status code:
409 Conflict
. - Face not found in the photo
-
Status code:
422 Unprocessable Entity
.
Body: No face. - Payload validation error
-
Status code:
400 Bad Request
. Body: reason. - Person not found
-
Status code:
404 Not Found
.
GetPersonPhoto
Receives information about photo with specified identifier.
Request
GET /persons/{personId}/photos/{photoId}
Parameter Name |
Value Type |
Description |
|
string |
Identifier of the person. |
|
string |
Identifier of the photo. |
Request Body
Do not supply a request body with this method.
Response
- Photo received
-
Status code:
200 OK
.
Content-Type:application/vnd.com.smilart.helios.photo+json
Body: representation of the resource. - Photo or person not found
-
Status code:
404 Not Found
.
GetPersonPhotoJpeg
Receives binary JPEG photo from the person.
Request
GET /persons/{personId}/photos/{photoId}/jpeg
Parameter Name |
Value Type |
Description |
|
string |
Identifier of the person. |
|
string |
Identifier of the photo. |
Request Body
Do not supply a request body with this method.
Response
- Photo received
-
Status code:
200 OK
.
Content-Type:image/jpeg
.
Body: binary representation of the photo for the person. - Photo or person not found
-
Status code:
404 Not Found
.
DeletePersonPhoto
Deletes photo of the person with specified identifier.
Request
DELETE /persons/{personId}/photos/{photoId}
Parameter Name |
Value Type |
Description |
|
string |
Identifier of the person. |
|
string |
Identifier of the photo. |
Request Body
Do not supply a request body with this method.
Response
- Photo deleted
-
Status code:
204 No Content
. - Photo or person not found
-
Status code:
404 Not Found
.
Camera Collection
Overview
Represents collection of cameras in system.
Resource Representation
Cameras
Content-Type: application/vnd.com.smilart.helios.cameras+json
JSON array with string identifiers of cameras in system.
CamerasInfo
Content-Type: application/vnd.com.smilart.helios.cameras_info+json
JSON array with camera representation:
[
{
"id":string,
"active":boolean,
"running":boolean,
},
...
]
Methods
ListCameras
Lists all cameras in system.
Request
GET /cameras
Header Name |
Value Type |
Description |
|
string |
Media type(s) that is/are acceptable for the response. Acceptable types are:
If not provided application/vnd.com.smilart.helios.cameras+json is used. |
Request Body
Do not supply a request body with this method.
Response
- Cameras listed
-
Status code:
200 OK
.
Body: representation of the resource. - Not Acceptable
-
Status code:
406 Not Acceptable
Camera Info
Overview
Represents camera info.
Resource Representation
Content-Type: application/vnd.com.smilart.helios.camera_info+json
{
"id":string,
"active":boolean,
"running":boolean,
}
Property Name |
Value Type |
Description |
|
string |
Identifier of the camera. |
|
boolean |
Whether the camera was activated (started) in the system, typically intentionally by the system administrator. |
|
boolean |
True if camera frames are available for processing, otherwise false. |
Methods
GetCameraInfo
Gets the camera information from the system.
Request
GET /cameras/{cameraPid}
Parameter Name |
Value Type |
Description |
|
string |
Identifier of the camera. |
Request Body
Do not supply a request body with this method.
Response
- Camera Info provided
-
Status code:
200 OK
.
Body: representation of the resource. - Camera not found
-
Status code:
404 Not Found
.
Camera MJPEG
Overview
Represents MJPEG Stream from cameras.
Methods
GetCameraMjpegStream
Gets a stream from the camera.
Request
GET /cameras/{cameraPid}/mjpeg
Parameter Name |
Value Type |
Description |
|
string |
Identifier of the camera. |
Parameter Name |
Value Type |
Description |
|
enumeration |
Resolutions of the frames. Acceptable types are: small, medium, large, original. If not provided original is used. |
|
integer |
Rate limit for incoming messages per seconds. |
Request Body
Do not supply a request body with this method.
Response
- Camera MJPEG Stream provided
-
Status code:
200 OK
.
Headers: multipart/x-mixed-replace; boundary=<some boundary>
Body: representation of the resource. - Incorrect request parameters
-
Status code:
400 Bad Request
. - Camera not found
-
Status code:
404 Not Found
.
VCA Service
Overview
VCA Service provides access to events generated by Platform in the process of handling frames.
Subscription to video analytics events is implemented via WebSockets.
To subscribe client should open WebSocket with header Sec-WebSocket-Protocol: vca
.
Open WebSocket will receive messages with the result of the analysis of the frame from the selected camera.
The message is an aggregated event (frame + detection + correlation + identification) in JSON.
There is an aggregation timeout for event and if the system does not collect all the information during this time some information in the aggregated event may be missing. |
A subscription for more then one type of event will add a delay (about the aggregation timeout value) to the sending. |
In the subscription request you can specify what type of information with analysis of the frame you want to receive and how often you want to receive messages.
If an error occurs during validation of specified options in the beginning or during the process, WebSocket would be closed by the server with the corresponding code.
The server will only close connection if the client terminated subscription by closing WebSocket. |
Correlations are primarily debugging information that can be completely correct interpreted only by the vendor’s specialists and reflects the features of the currently used face recognition algorithm that may change in the future. Therefore, you SHOULD NOT make any conclusions based on the received coefficients, except for getting the top of the most similar persons in the database according to the current face recognition algorithm. |
Request
GET /cameras/{cameraPid}/vca
Parameter Name |
Value Type |
Description |
|
string |
Identifier of the camera. |
Parameter Name |
Value Type |
Default Value |
Description |
|
string |
|
Defines what types of events should be sent.
|
|
string |
not provided |
Defines what types of events are required and should always be presented in aggregated message. These set of types should be subset of
|
|
string |
|
Size of frames from camera.
|
|
integer |
not provided |
Rate limit for incoming messages per seconds. |
|
string |
|
Type of images to send.
|
|
string |
|
Type of images to send.
|
|
string |
|
Type of images to send.
|
|
integer |
10 |
Maximum number of matches in correlation event. Positive integer. |
|
integer |
not provided |
Minimum time interval in seconds between identification messages of the same person. Positive integer. |
Server Send Message Representation
Below is a scheme of an aggregated event in JSON.
Depending on the settings and system performance, some objects may be missing.
{
"camera": string,
"sequence": integer,
"timestamp": integer,
"frame": {
"meta": {
"width": integer,
"height": integer,
},
"image": {
"contentType": "image/jpeg",
"data": string
}
},
"detects": {
"<detect id>": {
"id": <detect id>,
"top": integer,
"left": integer,
"right": integer,
"bottom": integer,
"leftEyeTop": integer,
"leftEyeLeft": integer,
"rightEyeTop": integer,
"rightEyeLeft": integer,
"detectFace": {
"contentType": "image/jpeg",
"data": string
}
}
},
"correlations": {
"<detect id>": {
"id":<detect id>,
"matches": [
{
"correlation": float,
"personId": string,
"photoId": string,
"databaseFace": {
"contentType": "image/jpeg",
"data": string
}
}
]
}
},
"identifications": {
<detect id>: {
"id": <detect id>,
"correlation": float,
"threshold": float,
"personId": string,
"photoId": string,
"databaseFace": {
"contentType": "image/jpeg",
"data": string
}
}
}
}
Property Name |
Value Type |
Description |
|
string |
Pid of camera. |
|
integer |
Sequential number of the message. |
|
integer |
Server-side timestamp (with ms resolution) of the message. |
|
nested object |
Frame info if requested by client. |
|
nested object |
Frame meta. |
|
integer |
Width of the image of frame. |
|
integer |
Height of the image of frame. |
|
nested object |
Image of the frame. |
|
string |
Content type of payload. Only |
|
string |
Base64 encoded binary content of the image. |
|
nested object |
Information about detects. Present if requested by client. |
|
nested object |
Information about detect with identifier |
|
string |
Detect id = |
|
integer |
Coordinate of top horizontal line of face rectangle. |
|
integer |
Coordinate of bottom horizontal line of face rectangle. |
|
integer |
Coordinate of left vertical line of face rectangle. |
|
integer |
Coordinate of right vertical line of face rectangle. |
|
integer |
Top coordinate of face left eye. May be missing. |
|
integer |
Left coordinate of face left eye. May be missing. |
|
integer |
Top coordinate of face right eye. May be missing. |
|
integer |
Left coordinate of face right eye. May be missing. |
|
nested object |
Face from frame. Present if requested by client. |
|
string |
Content type of payload. Only |
|
string |
Base64 encoded binary content of the image. |
|
nested object |
Information about correlations. Present if requested by client. |
|
nested object |
Information about correlation for detect with identifier |
|
string |
Detect identifier = |
|
array |
List of matches for correlation. |
|
float |
Correlation coefficient between detected face and photo from base. |
|
string |
Person id correlated with. |
|
string |
Photo id from base correlated with. |
|
nested object |
Face image from base for correlated photo. Present if requested by client. |
|
string |
Content type of payload. Only |
|
string |
Base64 encoded binary content of the image. |
|
nested object |
Information about identifications. Present if requested by client. |
|
nested object |
Information about identification for detect with identifier |
|
string |
Detect id = |
|
float |
Correlation coefficient between detected face and photo from base. |
|
float |
Current system threshold for correlation coefficient. Person is identified if the correlation is not less than the threshold. |
|
string |
Person id identified with. |
|
string |
Photo id from base identified with. |
|
nested object |
Face image from base for identified photo. Present if requested by client. |
|
string |
Content type of payload. Only |
|
string |
Base64 encoded binary content of the image. |
Errors Handling
The table below shows error codes and descriptions that are returned if the emergency shutdown of the WebSocket on server side occurs.
Note that closing message is limited by 125 characters.
Code |
Full-text Reason Description |
Description |
Proposed client’s actions |
1001 |
|
Indicates that an endpoint is "going away", such as a server going down for some internal reason. |
Contact tech support. |
1011 |
|
Indicates that a server is terminating the connection because it encountered an unexpected condition that prevented it from fulfilling the request. |
Contact to tech support. |
4000 |
|
Optional query parameter name |
Check the parameter’s compliance with the service protocol. |
4001 |
|
Process couldn’t be started due to absence of camera with identifier |
Check availability of the specified camera. |
Photo Booth Service
Overview
Service provides the way to get optimal set of faces for person who stands in front of the camera to enroll the person using these photos.
To start process client should open WebSocket with header Sec-WebSocket-Protocol: photobooth
.
There are several steps to get the set of faces:
-
The person to enroll should stands in front of the camera.
-
The process of building the set of faces starts via WebSocket request.
-
The person by moving his or her head to different poses provides to the service different images of his or her face.
-
The service accepts the images depending on its sampler scheme.
-
When all necessary images have been collected or operation timeout occurs, the service stops the sampling process and creates the optimal set of faces, which will be returned from the service.
Sampling process is finished when area of interest is full.
The area of interest consists of one or several groups, each of which indicates a certain head pose or face position and is called named position or named group.
The area of interest is considered full, when each named group included in it is full.
A group of head pose is considered full when it collects enough detects where face relates to the group.
Implementation of the service reserves the right to define strategy of group of head pose progress estimation.
Client will receive JSON messages about the start, progress, detects and result of the process for the selected camera through the opened WebSocket.
There is a timeout to receive and convert an image for detect message and if the system does not succeed in the specified time, the message will not be sent. |
In the subscription request, you can specify what information you want to receive and how often to receive messages.
There is a timeout for sampling process (20 seconds by default) after which partial completed result will be send.
Clients can interfere with each other if they try to run the process from the same camera at the same time. |
If an error occurs during validation of options, start or during process, the WebSocket is closed by the server with the corresponding code.
The subscription is terminated when the client closes the WebSocket (the server does not close the WebSocket unless an error occurs even after the sampling process finished). |
Request
GET /cameras/{cameraPid}/photobooth
Parameter Name |
Value Type |
Description |
|
string |
Identifier of the camera. |
Parameter Name |
Value Type |
Default Value |
Description |
|
string |
|
Defines what types of events should be sent besides sampling result.
To receive messages only about sampling result use empty string as a parameter value. |
|
string |
|
Type of images to send.
|
Detected Face Named Positions
Named position indicates a certain head pose or face position.
-
"leftwardTurn"
: person looks to the left. -
"leftwardUpwardTurn"
: person looks to the left and up. -
"leftwardDownwardTurn"
: person looks to the left and down. -
"forwardTurn"
: person looks straight forward. -
"forwardUpwardTurn"
: person looks up. -
"forwardDownwardTurn"
: person looks down. -
"rightwardTurn"
: person looks to the right. -
"rightwardUpwardTurn"
: person looks to the right and up. -
"rightwardDownwardTurn"
: person looks to the right and down. -
"outsideLeftwardTurn"
: person looks to the left too far from forward position. -
"outsideRightwardTurn"
: person looks to the right too far from forward position. -
"outsideUpwardTurn"
: person looks up too far from forward position. -
"outsideDownwardTurn"
: person looks down too far from forward position.
Areas Of Interest
Set of face positions taken into account during the sampling process:
-
"cross"
: 5 positions ("forwardTurn"
,"leftwardTurn"
,"rightwardTurn"
,"forwardUpwardTurn"
,"forwardDownwardTurn"
). -
"horizontal"
: 3 positions ("forwardTurn"
,"leftwardTurn"
,"rightwardTurn"
). -
"forwardTurn"
: only forward position ("forwardTurn"
). -
"allInnerPoses"
: all 9 positions.
Server Send Message Representation
All messages from the service can be divided into 3 types:
-
initial message
-
info message
-
final message
Initial message
{
"camera": string,
"sequence": integer,
"timestamp": integer,
"startOptions": {
"timeLimitSeconds": integer,
"scheme": {
"grid3x3": {
"areaOfInterest": string
}
}
}
}
Info messages
{
"camera": string,
"sequence": integer,
"timestamp": integer,
"detects": {
"<detect id>": {
"id": <detect id>,
"top": integer,
"left": integer,
"right": integer,
"bottom": integer,
"leftEyeTop": integer,
"leftEyeLeft": integer,
"rightEyeTop": integer,
"rightEyeLeft": integer,
"headPose": string,
"detectFace": {
"contentType": "image/jpeg",
"data": string
}
}
}
}
{
"camera": string,
"sequence": integer,
"timestamp": integer,
"progress": {
"progressPercentage": integer,
"perPoseStatistics":
[{
"namedPosition": string,
"collectedPercentage": integer
}]
}
}
Final message
{
"camera": string,
"sequence": integer,
"timestamp": integer,
"done": {
"progressPercentage": integer,
"photos": [{
"contentType": string,
"data": string
}],
"description": string
}
}
Properties of messages
Property Name |
Value Type |
Description |
|
string |
Pid of camera. |
|
integer |
Sequential number of the message. |
|
integer |
Server-side timestamp (with ms resolution) of the message. |
|
nested object |
Initial information about sampling process. Sent only once in the beginning. |
|
integer |
Duration of sampling process in seconds. |
|
nested object |
Sampling scheme. |
|
nested object |
Type of scheme. Only |
|
string |
Type of area where a face should be detected. One of area types. |
|
nested object |
Information about detects. Present if requested by client and the system succeed to aggregate detect information. |
|
nested object |
Information about detect with identifier |
|
string |
Detect id = |
|
integer |
Coordinate of top horizontal line of face rectangle. |
|
integer |
Coordinate of bottom horizontal line of face rectangle. |
|
integer |
Coordinate of left vertical line of face rectangle. |
|
integer |
Coordinate of right vertical line of face rectangle. |
|
integer |
Top coordinate of face left eye. May be missing. |
|
integer |
Left coordinate of face left eye. May be missing. |
|
integer |
Top coordinate of face right eye. May be missing. |
|
integer |
Left coordinate of face right eye. May be missing. |
|
string |
Detected head pose. One of named positions. |
|
nested object |
Face from frame. Present if requested by client. |
|
string |
Content type of payload. Only |
|
string |
Base64 encoded binary content of the image. |
|
nested object |
Information about progress. Present if requested by client. |
|
integer |
Information about progress. It grows from 0 to 100 and corresponds to the proportion of frames with a face received from the camera and taken for sampling, from the number of frames with a face required for sampling process. |
|
array |
Progress of filling the pose with frames. |
|
string |
Named head pose. One of named positions. |
|
integer |
Information about progress for the head pose. It grows from 0 to 100 and corresponds to the proportion of frames with a face received from the camera and taken for sampling, from the number of frames with a face required for sampling process in the current head pose. |
|
nested object |
Information about sampling process results. Send only once on finish. |
|
integer |
Information about progress. It grows from 0 to 100 and corresponds to the proportion of frames with a face received from the camera and taken for sampling, from the number of frames with a face required for sampling process. |
|
array |
Optimal list of faces selected during training. |
|
string |
Content type of payload. Only |
|
string |
Base64 encoded binary content of the image. |
|
string |
Describes result status. One of:
|
Errors Handling
The table below shows error codes and descriptions that are returned while emergency shutdown of the WebSockets by server side.
Note that closing message is limited by 125 characters.
Code |
Full-text Reason Description |
Description |
Proposed client’s actions |
1001 |
|
Indicates that an endpoint is "going away", such as a server going down for some internal reason. |
Contact to tech support. |
1011 |
|
Indicates that a server is terminating the connection because it encountered an unexpected condition that prevented it from fulfilling the request. |
Contact to tech support. |
4000 |
|
Optional query parameter name |
Check the parameter’s compliance with the service protocol. |
4001 |
|
Process couldn’t be started due to absence of camera with identifier |
Check availability of the specified camera. |
4002 |
|
Process couldn’t be started due to sampling process already started for camera with identifier |
Terminate last sampling process or wait for some time. |
Verification Service
Overview
Service implements verification scenario: is the person who stands in front of the camera a person from base?
Subscription to verification events is carried out via WebSockets.
To subscribe client should open WebSocket with header Sec-WebSocket-Protocol: verification
.
The opened WebSocket will receive messages about the result of the analysis of the frame from the selected camera.
The message is an aggregated event (frame + detection + correlation + successfull verification) in JSON.
There is an aggregation timeout for event and if the system does not collect an information during the specified time, then some information in the aggregated event may be missing. |
A subscription for more then only successfull verification event will add a delay (about the aggregation timeout value) to the sending. |
In the subscription request, you can specify what information you want to receive, how often to receive messages.
Clients can interfere with each other if they try to run the process from the same camera at the same time. |
If an error occurs during validation of options, start or during process, the WebSocket is closed by the server with the corresponding code.
The subscription is terminated when the client closes the WebSocket (the server does not close the WebSocket on successful workflow even after the verification finished). |
When starting verification, the client does not specify in advance the maximum verification time. The verification process continues for as long as the client keeps the connection open (except as described above when the process is stopped by the server for other reasons). |
Correlations are primarily debugging information that can be completely correct interpreted only by the vendor’s specialists and reflects the features of the currently used face recognition algorithm that may change in the future. Therefore, you SHOULD NOT make any conclusions based on the received coefficients, except for getting the top of the most similar photos in the database according to the current face recognition algorithm. |
Request
GET /cameras/{cameraPid}/verify
Parameter Name |
Value Type |
Description |
|
string |
Identifier of the camera. |
Parameter Name |
Value Type |
Description |
|
string |
Identifier of the person to match. |
Parameter Name |
Value Type |
Default Value |
Description |
|
string |
|
Defines what types of events should be sent besides successfull verification event. Represents composition of types joined by "+".
To receive messages only about successfull verification use empty string as a parameter value. |
|
string |
|
Size of frames from camera.
|
|
integer |
not provided |
Rate limit for incoming messages per seconds. |
|
string |
|
Type of images to send.
|
|
string |
|
Type of images to send.
|
|
string |
not provided |
Name of the predefined verification threshold in the vendor implementation. Case insensetive. If not set — verification process will be started with threshold selected by implementation. List of available names should be provided by the vendor implementation. |
Server Send Message Representation
Below is a scheme of an aggregated event in JSON.
Depending on the settings and system performance, some objects may be missing.
{
"camera": string,
"sequence": integer,
"timestamp": integer,
"frame": {
"meta": {
"width": integer,
"height": integer
},
"image": {
"contentType": "image/jpeg",
"data": string
}
},
"detects": {
"<detect id>": {
"id": <detect id>,
"top": integer,
"left": integer,
"right": integer,
"bottom": integer,
"leftEyeTop": integer,
"leftEyeLeft": integer,
"rightEyeTop": integer,
"rightEyeLeft": integer,
"detectFace": {
"contentType": "image/jpeg",
"data": string
}
}
},
"correlations": {
"<detect id>": {
"id": <detect id>,
"matches": [
{
"correlation": float,
"personId": string,
"photoId": string,
"databaseFace": {
"contentType": "image/jpeg",
"data": string
}
}
]
}
},
"terminated": {},
"verified": {
"id": <detect id>,
"correlation": float,
"threshold": float,
"personId": float,
"photoId": string
}
}
Property Name |
Value Type |
Description |
|
string |
Pid of camera. |
|
integer |
Sequential number of the message. |
|
integer |
Server-side timestamp (with ms resolution) of the message. |
|
nested object |
Frame info if requested by client. |
|
nested object |
Frame meta. |
|
integer |
Width of the image of frame. |
|
integer |
Height of the image of frame. |
|
nested object |
Image of the frame. |
|
string |
Content type of payload. Only |
|
string |
Base64 encoded binary content of the image. |
|
nested object |
Information about detects. Presents if requested by client. |
|
nested object |
Information about detect with identifier |
|
string |
Detect id = |
|
integer |
Coordinate of top horizontal line of face rectangle. |
|
integer |
Coordinate of bottom horizontal line of face rectangle. |
|
integer |
Coordinate of left vertical line of face rectangle. |
|
integer |
Coordinate of right vertical line of face rectangle. |
|
integer |
Top coordinate of face left eye. May be missing. |
|
integer |
Left coordinate of face left eye. May be missing. |
|
integer |
Top coordinate of face right eye. May be missing. |
|
integer |
Left coordinate of face right eye. May be missing. |
|
nested object |
Face from frame. Presents if requested by client. |
|
string |
Content type of payload. Only |
|
string |
Base64 encoded binary content of the image. |
|
nested object |
Information about correlations. Presents if requested by client. |
|
nested object |
Information about correlation for detect with identifier |
|
string |
Detect identifier = |
|
array |
List of matches for correlation. |
|
float |
Correlation coefficient between detected face and photo from base. |
|
string |
Person id correlated with. |
|
string |
Photo id from base correlated with. |
|
nested object |
Face image from base for correlated photo. Presents if requested by client. |
|
string |
Content type of payload. Only |
|
string |
Base64 encoded binary content of the image. |
|
nested object |
Indicates that the process was terminated by another start request with the same camera pid. Send only once. |
|
nested object |
Information about verification. Send only once on finish. |
|
string |
Detect identifier = |
|
float |
Correlation coefficient between detected face and photo from base. |
|
float |
Current system threshold for correlation coefficient. Person is verified if the correlation is not less than the threshold. |
|
string |
Person id identified with. |
|
string |
Photo id from base identified with. |
Errors Handling
The table below shows error codes and descriptions that are returned if the emergency shutdown of the WebSocket on server side occurs.
Note that there is a limitation on the size of the message when closing WebSockets (125 characters).
Code |
Full-text Reason Description |
Description |
Proposed client’s actions |
1001 |
|
Indicates that an endpoint is "going away", such as a server going down for some internal reason. |
Contact to tech support. |
1011 |
|
Indicates that a server is terminating the connection because it encountered an unexpected condition that prevented it from fulfilling the request. |
Contact to tech support. |
4000 |
|
Optional query parameter name |
Check the parameter’s compliance with the service protocol. |
4001 |
|
Process couldn’t be started due to absence of camera with identifier |
Check availability of the specified camera. |
4002 |
|
Process couldn’t be started due to absence of person with identifier |
Check existence of the specified person in base. |
4003 |
|
Process couldn’t be started due to absence of threshold named |
Check the correctness of the threshold name. |
IPA Service
Overview
Instant Photo Analytics (IPA) Service provides capability to analyze an image and generate analysis report.
Current supported content type of binary representation of a photo is "image/jpeg"
only.
Methods
PhotoAnalysis
Correlates image with base and generate analysis report.
Request
POST /photo_analysis
Parameter Name |
Value Type |
Default Value |
Description |
|
string |
|
Defines what types of events should be sent.
|
|
string |
|
Type of images to send.
|
|
string |
|
Type of images to send.
|
|
string |
|
Type of images to send.
|
|
integer |
10 |
Maximum number of matches in correlation event. Positive integer. |
|
integer |
not provided |
Maximum number of faces in detect event. Positive integer. |
Request Body
- Content-Type
-
multipart/form-data; boundary={your boundary}
- Payload
-
Binary
Success Response
- Status code
-
200 OK
- Content-Type
-
application/vnd.com.smilart.helios.ipa.result+json
- Body
{
"faces":[{
"detect":{
"face":{
"top": integer,
"left": integer,
"right": integer,
"bottom": integer,
"leftEyeTop": integer,
"leftEyeLeft": integer,
"rightEyeTop": integer,
"rightEyeLeft": integer
},
"cutting": {
"top": integer,
"left": integer,
"right": integer,
"bottom": integer,
"leftEyeTop": integer,
"leftEyeLeft": integer,
"rightEyeTop": integer,
"rightEyeLeft": integer,
"detectedFace": {
"contentType": "image/jpeg",
"data": string
}
}
},
"correlations":[{
"correlation": float,
"personId": string,
"photoId": string,
"databaseFace": {
"contentType": "image/jpeg",
"data": string
}
}],
"identification":{
"threshold": float,
"correlation": float,
"personId": string,
"photoId": string,
"databaseFace": {
"contentType": "image/jpeg",
"data": string
}
}
}]
}
Property Name |
Value Type |
Description |
|
array |
List of information on found faces sorted by face size in descending order. |
|
nested object |
Information about face detect. Presents when query parameter |
|
nested object |
Information about a face at the original image. |
|
integer |
Coordinate of top horizontal line of face rectangle. |
|
integer |
Coordinate of bottom horizontal line of face rectangle. |
|
integer |
Coordinate of left vertical line of face rectangle. |
|
integer |
Coordinate of right vertical line of face rectangle. |
|
integer |
Top coordinate of face left eye. May be missing. |
|
integer |
Left coordinate of face left eye. May be missing. |
|
integer |
Top coordinate of face right eye. May be missing. |
|
integer |
Left coordinate of face right eye. May be missing. |
|
nested object |
Information about cut face. Presents when query parameter |
|
integer |
Coordinate of top horizontal line of cut face rectangle. |
|
integer |
Coordinate of bottom horizontal line of cut face rectangle. |
|
integer |
Coordinate of left vertical line of cut face rectangle. |
|
integer |
Coordinate of right vertical line of cut face rectangle. |
|
integer |
Top coordinate of cut face left eye. May be missing. |
|
integer |
Left coordinate of cut face left eye. May be missing. |
|
integer |
Top coordinate of cut face right eye. May be missing. |
|
integer |
Left coordinate of cut face right eye. May be missing. |
|
nested object |
Information about cut face image. |
|
string |
Content type of payload. Only |
|
string |
Base64 encoded binary content of the image. |
|
array |
Information about correlations. Presents when query parameter |
|
float |
Correlation coefficient between detected face and photo from base. |
|
string |
Person identifier correlated with. |
|
string |
Photo identifier from base correlated with. |
|
nested object |
Face image from base for correlated photo. Presents when query parameter |
|
string |
Content type of payload. Only |
|
string |
Base64 encoded binary content of the image. |
|
nested object |
Information about identification. Presents when query parameter |
|
float |
Current system threshold for correlation coefficient. Person is identified if the correlation is not less than the threshold. Always presents. |
|
float |
Correlation coefficient between detected face and photo from base. Presents when a person was identified. |
|
string |
Person identifier identified with. Presents when a person was identified. |
|
string |
Photo identifier from base identified with. Presents when a person was identified. |
|
nested object |
Face image from base for identified photo. Presents when a person was identified and query parameter |
|
string |
Content type of payload. Only |
|
string |
Base64 encoded binary content of the image. |
Error Responses
- Incorrect request parameters
-
Status code:
400 Bad Request
. - Request timeout
-
Status code:
408 Request Timeout
. - Image file is too large
-
Status code:
413 Payload Too Large
. - Unsupported image type
-
Status code:
422 Unprocessable Entity
. - Service is overloaded
-
Status code:
429 Too Many Requests
.
Adaptive Verification Service
Overview
Adaptive Verification (AV) provides the way to improve the user experience in verification process by populating person base by sampled photos during successful verification from cameras and adjustment of verification thresholds.
Side effects on other services
Impact on Person Management service
Being activated, AV service can modify list of person photos (add and delete photos), but it can delete only those photos which were added by this service (sampled photos). This service will not delete any person’s photos, added by another service (e.g. Person Management service).
All sampled photos will be accessible in Person Management service with special flag indicating whether the photo was added (sampled) by AV service or not.
Being deactivated this service does not delete sampled photos. Client can get rid of sampled photos via explicit remove requests at any moment.
Impact on Verification service
Being activated AV service will change thresholds for verification requests and provide additional person photos for verification.
Impact on other services
Other services will not take into account sampled photos and will not change their behavior because of the activity of AV service.
Methods
GetConfig
Gets the service configuration.
Request
GET /av/config
Request Body
Do not supply a request body with this method.
Success Response
- Status code
-
200 OK
- Content-Type
-
application/vnd.com.smilart.helios.av.config+json
- Body
{
"active": boolean
}
Property Name |
Value Type |
Description |
|
boolean |
Service activity status. |
Error Responses
- Request timeout
-
Status code:
408 Request Timeout
. - Service is overloaded
-
Status code:
429 Too Many Requests
.
SetConfig
Sets the service configuration.
Request
POST /av/config
Request Body
- Content-Type
-
application/vnd.com.smilart.helios.av.config+json
- Payload
-
JSON
{
"active": boolean
}
Property Name |
Required |
Value Type |
Description |
|
true |
boolean |
Desired service activity status.True if service should sample person photos and adapt verification thresholds, otherwise false. |
Success Response
- Status code
-
200 OK
- Content-Type
-
application/vnd.com.smilart.helios.av.config+json
- Body
{
"active": boolean
}
Property Name |
Value Type |
Description |
|
boolean |
Service activity status. |
Error Responses
- Incorrect request parameters
-
Status code:
400 Bad Request
. - Request timeout
-
Status code:
408 Request Timeout
. - Payload is too large
-
Status code:
413 Payload Too Large
. - Unsupported payload type
-
Status code:
422 Unprocessable Entity
. - Service is overloaded
-
Status code:
429 Too Many Requests
.
RemoveAllPhotos
Removes all sampled photos.
Request
DELETE /av/photos
Request Body
Do not supply a request body with this method.
Success Response
- Sampled photos deleted
-
Status code: 204 No Content.
Error Responses
- Request timeout
-
Status code:
408 Request Timeout
. - Service is overloaded
-
Status code:
429 Too Many Requests
.
RemovePhotosByPerson
Removes all sampled photos of the person.
Request
DELETE /av/photos/persons/{personId}
Parameter Name |
Value Type |
Description |
|
string |
Identifier of the person. |
Request Body
Do not supply a request body with this method.
Success Response
- Sampled photos deleted
-
Status code: 204 No Content.
Error Responses
- Request timeout
-
Status code:
408 Request Timeout
. - Service is overloaded
-
Status code:
429 Too Many Requests
.
RemovePhotosByCamera
Removes all sampled photos from the camera for every person.
Request
DELETE /av/photos/cameras/{cameraPid}
Parameter Name |
Value Type |
Description |
|
string |
Identifier of the camera. |
Request Body
Do not supply a request body with this method.
Success Response
- Sampled photos deleted
-
Status code: 204 No Content.
Error Responses
- Request timeout
-
Status code:
408 Request Timeout
. - Service is overloaded
-
Status code:
429 Too Many Requests
.