Mercury scanner for a safe waste separation process
The brief
OPEN Foundation is working with Dutch waste processors to make the e-waste sector more circular. When separating different types of monitors, improperly separating an LCD monitor can lead to safety risks for employees due to the presence of the hazardous mercury (Hg). Moreover, this separation process is time-consuming and highly dependent on the knowledge and experience of each individual employee.
SevenLab has been tasked with developing a scanning app that indicates whether a monitor contains mercury. This scanner uses machine learning to recognise labels on monitors and identify which type of monitor it is.
The challenge
In practice, an experienced worker can quickly recognise the type of monitor, and separate it correctly. This is thanks to subtle differences such as monitor thickness or colour. As a result, these workers can recognise some monitor types from a distance, even without having seen a label.
However, this process requires the training of several employees who, in many cases, are only employed by a waste processor for a short time. Therefore, the current process is time-consuming, and error-prone with inexperienced workers. And so there was a need for a digital solution in the form of a mercury scanner.
To make the use of the application as effective and user-friendly as possible, it was necessary to come up with a tailor-made solution.
Using AI to recognise a label
All monitors contain mandatory labels on the back that contain various characteristics to identify mercury. These features are expressed as logos or codes. We deployed 'optical character recognition' (OCR) to train our model to recognise specific words or letter combinations. In addition to OCR, we also developed a model that uses object recognition to identify various icons. By combining these two techniques, the app can recognise the type of screen the monitor contains from the label, and then display this information to the user.
This information then allows the user to correctly place the monitor in the correct waste stream.
A fast scanner
To minimise the waiting time between scanning and displaying a result, we carefully considered the choice of appropriate software and hardware, and how we would train the machine learning model. In some cases, this process took several seconds longer, leading to additional waiting time for the worker. This is not desirable, given that up to hundreds of monitors are processed daily per waste processor.
User-friendly interface that matches the working environment
When designing the interface, our goal was to minimise the number of actions users have to perform to run a scan. This was important to ensure that using the application takes little time and does not interfere with the employee's work. We created an interface with only two pages, which can be operated with a single physical button to activate the scanner. After scanning, the result is displayed; this page closes automatically after a set time, allowing the user to start a new scan immediately.
Users
When developing our software, we always consider the end user. For this application, there were several interesting requirements that we had to pay specific attention to.
Navigation and gloves
As waste handling takes place in an environment with many sharp objects, workers wear thick gloves. These gloves hamper the use of digital buttons on the screen, as the finger is not recognised due to the thickness of the glove. To circumvent this problem, we considered two possible solutions.
The first solution involved a buttonless scanner, where the photo camera continuously tries to recognise labels without user action. We did not choose this option because the technique consumes too much data.
The second solution, which was our preferred one, uses the physical (volume) buttons on the side of the smartphone. These buttons can be pressed by workers wearing gloves, though, which means we do not have to resort to data-intensive techniques.
By activating the scanner via the volume buttons, users can operate the application even with gloves on, without having to take off their gloves for each scan.
Command of the Dutch language
Employees often come from different backgrounds; for example, there are immigrants who do not speak the Dutch language. To take this into account, it was essential for the application to be understandable without textual support, and to use recognisable words such as 'LED' and 'LCD' as much as possible.
Working visit
To gain these insights, we visited several waste processors to observe how workers currently do their work and assess how the application can complement their working methods. During the visit, we paid extra attention to different working conditions, such as: who is working, is the use of gloves mandatory, can we use audio or is there a lot of loud noise, is there enough light available to perform a scan and how likely is the smartphone to be damaged on the shop floor.
