At Swipe iX we recently built and deployed Swipe Facial Recognition, a facial recognition sign-in system that uses AWS Rekognition. The product was originally conceptualised to eliminate the need for manual sign in processes, i.e. paper logbooks, and, depending on the nature of the business, receptionists.

The world over visitors are asked to sign in at front desks when they arrive to visit an employee of a business. This process can be cumbersome at best and frustrating at worst resulting in mounds of paper log books that need to be stored for multiple years.

Following our internal ideation and continuous development process the Cloud Innovation team built a faster and more efficient cloud-based solution that uses facial recognition to identify new and returning customers and in so doing eliminate the need for legacy paper-based log books altogether.

From a user’s perspective, the sign in process is fast, intuitive and user-friendly. Powered by a native app installed on a device such as an Android tablet, the visitor presents him/herself to the screen upon arrival. Harnessing the onboard camera and edge machine learning, the app recognises that there is a face in view and initiates the recognition process to the cloud. If the user is a first-time visitor he/she only needs to complete the full sign in process once.

Once all the necessary fields (customisable) are completed, the user has the option to email him or herself a copy of the NDA upon acceptance. Once complete, he/she can select the name of the employee they are there to see and the employee will be notified that the visitor is at the front desk.

The next time the person returns to the premises, he/she only has to approach the tablet, wait for the “confirmation of recognition” screen (+-1 second), which includes a personal welcome, and then select who they are there to see. The whole process takes less than 5 seconds and requires no human intervention.

Most importantly, the user’s data is secure and the user can ask to be “forgotten”. In so doing the system complies with data privacy regulations.

How we built it

Architecture

• The goal of the architecture was to keep the cost low and manageable. To reach these goals, the platform was built using a serverless architecture from the ground up and leveraging SaaS services offered by AWS wherever possible.
• API Gateway serves as the REST API for both the mobile and web applications. Authentication is done with two authorisers, one custom authoriser for all resources used by the mobile app using JWT tokens, and one authoriser against Cognito for the web portal. API Gateway integrates with Lambda as the compute platform to process all the incoming requests and the Lambda functions are written in Golang.
  
• ‍Usage plans are leveraged on API Gateway to control access rate limits and quotas per application. Each application is assigned an API key which in turn is linked to a usage plan. API Gateway is also deployed on the edge to speed up latency to the API endpoints.
• ‍Data is stored using DynamoDB and access to DynamoDB is controlled with fine-grained IAM policies to allow each Lambda function only access to the tables it requires, and the tasks it needs to perform on a specific table. All dynamoDB tables are also encrypted at rest.
• ‍ is used to index and recognise faces from images sent from the mobile app. Images are sent directly to Rekognition tied with a unique user ID during training and recognition. The image data is not stored and is thrown away after training and/or recognition.
• ‍RekognitionThe infrastructure and Lambda function deployments are managed using Terraform and is deployed in a continuous delivery fashion. Application logs and infrastructure monitoring is done using CloudWatch.

App

• ‍Swipe iX set out to build an application that is fast, user-friendly, cost-effective and white label friendly.
• ‍The initial application is focused on Android tablets for their cost-effectiveness and is written as a native Android application. The application does not store any user information, however API access tokens are stored using Realm where encryption provided by Realm and the device keystore is leveraged to keep the tokens safe on the device.
• ‍The most prominent feature of the app is the camera interface. The camera interface is designed to always be active and there is no need for the user to initiate an image to be taken for recognition. Machine learning is used on the device with a video pipeline to continuously monitor the camera feed for any faces entering the feed. This is done on the device to save bandwidth cost to process the video feed in the cloud, and for responsiveness. Once a face has been detected in the feed, an image is automatically taken and then sent to the platform to process. The recognition response is then used to navigate to the correct section of the sign-in process on the device depending on if the user was recognised or not.

Web portal

The goal for the web portal was to be lightweight, to be served with no web servers and to be white label friendly. Polymer was chosen as the framework for the portal and was written as a progressive web app. The web components architecture offered by Polymer were leveraged to provide a flexible interface that is easily customise-able and style-able. The application is hosted on S3 behind a CloudFront distribution.

The portal uses Cognito for account and access management. Cognito groups are further used for access permissions on the portal.

Services used:

• ‍API Gateway
• ‍Lambda - Go runtime
• ‍DynamoDB
• ‍IAM roles and policies
• ‍CloudWatch - application logs + metrics
• ‍S3
• ‍CloudFront
• ‍AWS CLI - Terraform and deployments
• Cognito

If you want to find out more about what Swipe Facial Recognition can do for your business, contact us today.