Optical sorting of particles containing tar and bitumen from mineral material streams with high proportions of black mineral matter

1 Introduction

Tar-based binders have been used in road construction for many decades. Due to their high content of polycyclic aromatic hydrocarbons (PAHs), they are considered a hazard to human health and the environment. Despite the ban that has been in force since 1984, significant quantities of tar-based materials remain in the transport infrastructure. Estimates suggest that there are around 1 billion tonnes of tar-containing asphalt in German roads.

In addition to road rubble, numerous structures and topsoil are also contaminated with tar particles. Historically, tar-containing road rubble was sometimes disposed of improperly or used as backfill material. Through agricultural use and soil cultivation, these contaminated particles have spread over large areas. This problem exists in many countries across Europe. The result is a low proportion (approx. 5 %) of contaminated asphalt particles in topsoil and mineral material flows. This leads to high disposal costs during civil engineering works and when disposing of the resulting excavated material.

To solve this problem, Zwisler GmbH of Tettnang funded the “AsphaCycle” development project. Marco Haas managed the project on behalf of Zwisler. The scientific investigations into optical detection were carried out in a preliminary study at the MFPA in Weimar. The technical development of the sorting technology was carried out in collaboration with Binder+Co AG from Austria. Over a period of four years (Table 1), the “AsphaCycle” development project aimed to automatically separate tar- and bitumen-containing asphalt particles from mineral material streams. Through the targeted separation of contaminated components, contaminated material streams can be significantly reduced and uncontaminated mineral raw materials returned to the building materials cycle.

The results of AsphaCycle I show that sensor-based separation of tar-containing asphalt particles is technically feasible and economically viable. This development contributes to the conservation of resources, the reduction of landfill volumes and transport distances, and the strengthening of the circular economy in the construction materials sector.

A key challenge lay in reliably distinguishing between asphalt (bitumen and tar) and natural dark mineral material. Investigations by the MFPA Weimar showed that modern optical sensor systems are also capable of distinguishing damp, dark mineral material from asphalt. Using samples provided by Zwisler, the MFPA Weimar carried out a preliminary study, which is discussed in detail below.

2 Preliminary study on the image-analytical differentia­tion of contaminants such as tar and bitumen (asphalt) particles, in natural aggregates

2.1 Test materials and image datasets

Using an image dataset specifically compiled at the MFPA Weimar for the materials to be sorted – gravel, asphalt (bitumen & tar) – various optical recognition tasks were investigated (Fig: 2):

1. Two-class problem: gravel (dark) (Class 1) and bitumen and tar (Class 2)

2. Three-class problem: gravel (dark) (Class 1), bitumen (Class 2) and tar (Class 3)

For all samples, at least 1000 objects per class with a wet surface were imaged under specular illumination using a high-resoluting industrial colour camera. The resulting dataset was then used to train machine learning algorithms. These were employed to determine characteristic image features such as texture, colour, shape and grey-scale values from the images. Table 2 provides an overview of the datasets created. Table 3 shows example images of segmented shots of the classes under consideration.

2.2 Classification based on image features

For each image object, a total of 200 features (texture, colour, shape and grey-scale features) were calculated using the relevant characteristics and summarized in a feature vector. The selected features were used to train a classifier. The feature vector thus constitutes the input for the classifiers. The next important step in the process was classifier training. To this end, the existing dataset was divided into training, validation and test datasets. Three classical machine learning methods – Support Vector Machine (SVM), Multilayer Perceptron (MLP) and k-Nearest Neighbour (kNN) – were used as classifiers. In addition, two innovative deep learning classifiers (DL_enhanced and DL_resnet50) were tested for comparison. Fig. 3 and Fig. 4 below illustrate the recognition rates achieved by the tested classifiers, firstly for the 2-class problem and secondly for the 3-class problem. The recognition rate represents the proportion of correctly recognised objects in the test dataset.

The results in Fig. 3 show that the particles to be sorted out (bitumen + tar) could be very clearly distinguished from the gravel particles, with a recognition rate of over 98 %. All classifiers achieved similarly good results. The best result, with a recognition rate of 98.4 %, was achieved using the SVM classifier.

Distinguishing between all three classes presented a greater challenge, and the classifiers produced varying results (Fig. 4). The SVM classifier also achieved the best result here, with a recognition rate of 95.6 %. A similarly good result was achieved using the MLP. The kNN classifier delivered lower recognition rates, at 91.10 %. Most misclassifications occur between bitumen and tar.

The pre-trained artificial neural networks delivered slightly better results than the classical classifiers, with an average recognition rate of 98.4 % in the 2-class problem and 96.0 % in the 3-class problem; however, the computational effort required was significantly higher.

Finally, a further dataset was created containing only dark particles from the gravel fraction. The following figures show a summary of the recognition rates achieved for both the two-class and the three-class problems (Fig. 5 and Fig. 6).

Here, too, it can be summarized that SVM proved to be a suitable classifier for both the two-class and the three-class problems. The highest recognition rate among the classical classifiers was 98.1 % for the first case and 94.2 % for the second. The deep learning classifiers delivered a marginally better result than the classical methods. Using the pre-trained resnet50 neural network, recognition rates of 98.5 % and 95.2 % were achieved for the two-class and three-class problems respectively.

2.3 Summary

The preliminary and feasibility studies show that it is possible to distinguish both the dark asphalt particles (tar + bitumen) from the gravel and, individually, the tar, bitumen and gravel particles with wet surfaces. The highest recognition rates achieved in each case were 98.4 % (SWM and DL_resnet50) when bitumen and tar were considered together. When tar, bitumen and gravel were considered separately, the maximum recognition rates were 95.6 % (SVM) and 96.0 % (DL_enhanced) respectively. In addition, a separate analysis of only “dark” gravel particles and bitumen/tar particles was investigated. Here, too, it was confirmed that differentiation based on image features was very well possible. The average recognition rate achieved here was also very high at 98.5 % (DL_enhanced). Fig. 7 provides a summary of all the results obtained.

It is important to note that all images were captured under laboratory conditions (high-resolution images) and that a deterioration in recognition performance is to be expected when implementing the system in an industrial setting. The effects of sorting using compressed-air nozzles must also be taken into account in practical applications. The results were subsequently implemented in collaboration between the two companies, Hermann Zwisler Besitz- und Verwaltungs-GmbH und Co. KG and Binder+Co AG, and are described in more detail below.

3 Implementation in a practical application

Armed with the findings from the preliminary studies, Zwisler GmbH approached Binder+Co AG in 2022. The two companies had previously been in contact regarding the optical sorting of mineral materials. Binder+Co has long possessed extensive expertise in the processing of bulk materials, particularly in the field of construction raw materials, where Binder+Co’s sorting and screening technology is internationally renowned.

At that time, Binder+Co had already initiated a development project on “AI-supported optical sorting” and had implemented this in sorting machines to test a wide variety of applications.

The promising results from the preliminary investigations under laboratory conditions were an initial indicator of the industrial feasibility of optical sorting. However, it should be noted that mere detection does not yet constitute separation. In real-world applications, there are a large number of interfering factors that can impair sorting. These primarily include the following factors:

Inhomogeneity of material classes

Inconsistent surface properties (dirt, water, ...)

Abrasion of the particles

Short exposure times

Different illumination geometry

Challenges in detecting dark/black particles

The need for high throughputs and, consequently, sufficient computing capacity

Mechanical inaccuracy in sorting

All these factors require an industrial machine to be equipped with a high-performance optical system, as well as a mechanical engineering solution that combines robustness, throughput and mechanical performance. Binder+Co addresses the needs of the industry and is able to successfully overcome the challenges listed above thanks to its many years of experience in the sorting of minerals and other bulk materials.

3.1 Description of Binder+Co’s technology

AI classifier

In the neural network used by Binder+Co, the objects to be sorted are assigned to a predefined optical material class; this is referred to hereafter as “classification”. With the algorithm used, classification takes place in fractions of a second and can handle a large number of particles simultaneously, thereby meeting the basic requirement for industrial-scale application.

Optical sorter

In an optical sorting machine, a free-flowing, narrowly graded bulk material is fed onto the sorting machine in a single layer via a vibrating conveyor chute. On the sorting machine, the bulk material is further separated in the direction of conveyance by the acceleration device (belt or chute) and thus presented to the sensor system (in this case, a camera operating in the visible light spectrum). Immediately after the image data is captured by the sensor, the classifier makes the sorting decision to activate the corresponding air nozzles. As a result, the parts to be sorted out are deflected from the ejection trajectory at the end of the belt and directed past the separation edge into the reject product. In the case of two-product separation, this works across the entire width of the sorting machine. Consequently, a large quantity of material can be reliably sorted within a short period of time.

3.2 Initial investigations through to confirmation of feasibility

To verify the suitability of the neural network used by Binder+Co for the task at hand, small samples (~50 parts per feature class) from two particle size fractions were first imaged and used to train the neural network. The resulting classifier was then tested using a previously untrained portion of the sample to validate the system. Fig. 8 shows the feature classes used for the sorting trials.

Once the initial investigations had proved promising, the samples from the material classes were imaged in the next step using the camera installed in the sorting machine, in order to confirm the feasibility for the existing sorting system used for sorting.

3.3 Sorting trials and results

For the subsequent sorting trials, larger quantities of training samples (~1000 parts per feature class) were used to account for the inhomogeneities within the feature classes.

The sorting tests were carried out on a CLARITY AI belt sorter (Fig. 9). Due to its flexibility with regard to different feed materials and particle sizes, this proved to be the optimal test system for various tasks that can be solved using the AI classifier. The separation efficiencies of the three particle size fractions tested are shown in Table 4.

During the experiments, basic sorting was observed for all the above-mentioned fractions. However, as the particle size decreased steadily, the system reached its limits in several respects. On the one hand, throughput was increasingly restricted for finer particle sizes due to the lower average height of the monolayer; on the other hand, as the particle size decreased, there were less and less image informations available for classification. Misclassifications and over-sorting due to high valve activity lead to both increased disposal costs and higher compressed air consumption. High sorting accuracy is therefore an essential prerequisite for the cost-effectiveness of sensor-based sorting processes. The results shown in Table 5 indicate that, as particle size increases, material throughput rises whilst specific compressed air consumption decreases.

3.4 Developments and future prospects for AI-supported optical sorting.

Since then, the results have been confirmed for the 8 – 16 mm and 16 – 32 mm fractions in terms of asphalt yield and classification accuracy. The first industrial machines have also already been commissioned. Furthermore, the AI classifier has since been applied to other sorting tasks involving various types of construction waste. This now makes it possible to separate a wide variety of different contaminants and recoverable materials from different fractions. A key advantage is that only a single sensor (RGB camera) is now required, rather than the complex integration of data from various sensors as was previously the case. This is possible, on the one hand, due to the use of neural networks to classify more complex image information and, on the other hand, due to the image features exhibited by the majority of secondary raw material streams. Some materials with usable image features are shown in Fig. 10.

The material samples shown in Fig. 10 exhibit “optical fingerprints” that can be used for classification. This concept can also be applied to other sorting tasks involving anthropogenic material streams.

4 Conclusion Haas Environmental Technology

The results of the AsphaCycle I development project demonstrate that optical sorting holds great potential for the future processing of mineral material streams. Advances in sensor technology, image processing and AI-supported data analysis are continuously improving separation accuracy and broadening the technology’s range of applications. At the same time, modern optical sorting systems can be flexibly adapted to different material streams and tasks in construction material recycling.

The comparatively high investment and operating costs are offset by high separation efficiency. Particularly in the case of contaminated mineral material streams, the targeted separation of contaminated components can significantly reduce the volume of material requiring costly disposal. This results in both economic benefits and a significant reduction in the burden on landfill capacity. At the same time, high-quality mineral raw materials are retained within the material cycle and can be reused.

Optical sorting technology will have a lasting impact on the circular economy in the construction materials sector in the coming years. It offers considerable economic and resource-conserving potential, particularly for regional material streams, the processing of contaminated materials and the decommissioning of existing landfill sites. With the ongoing development of high-performance sensor technology and artificial intelligence, it is expected that optical sorting systems will become a key technology in building materials recycling in the future.