Fraunhofer Institute for Production Technology IPTResearch Project "KOMBI"
(off.: Nutzbarmachung subjektiver Qualitätskriterien durch Kombination von Smart Devices und Machine Learning)
In many small and medium-sized enterprises (SMEs) today, optical visual inspections are often still purely subjective tasks of quality assurance that cannot be reproduced in detail because they are based on the experience of the individual employees. In the FQS-funded research project "KOMBI", the Fraunhofer IPT used smart devices and deep learning algorithms to develop a new form of quality assurance that uses objective scales for evaluation. This increases the reliability of quality assurance, relieves the workload of employees and allows for flexible use.
Variant diversity requires new concepts for quality control
The industrial environment is strongly characterized by the growing demands for flexibility and adaptability: 41 percent of all companies in the mechanical and plant engineering sector are already planning to manufacture products in quantities of one [1]. Due to the variability of the products, the requirements for quality characteristics often change, so that their quality can no longer be assessed in a reproducible manner. The increasing individualization of products further increases the already high number of manual visual inspections. However, the employees' assessments are always subjective. As a result, the reliability of the quality assessment depends on the experience of the employee, and trained employees can detect errors more reliably. Objectivity and reliability of the measurement are therefore not given. In addition, SMEs in particular have great difficulties in filling appropriate positions in quality assurance. Their goal is therefore to deploy skilled personnel as efficiently as possible in quality assurance and - if possible - to enable unskilled employees to perform certain tasks of this kind.
Smart devices assist in quality assurance
The aim of the "KOMBI" project was to objectify subjective quality criteria in order to support SMEs in quality assurance. This was achieved by providing worker assistance in the form of smart devices such as tablets or smart glasses. The worker assistance serves the objective quality evaluation and visualizes the classification features to be evaluated to the user. In addition, the data collected from the quality assessments performed is used to automate the visual inspection. Based on the worker assistance, deep learning algorithms are used.
With the results of the project "KOMBI", wrong decisions due to subjective influences can be avoided at the point of origin and the quality of the testing process can be improved. Thus the reliability of the test, which is a central requirement for a measuring method, can be secured. Unskilled employees will be enabled to perform manual visual inspections, so that the lack of skilled personnel will have less impact. The use of new technologies and the efficient use of data and information allows a cost-effective automation of quality assurance.
Deep learning algorithms for the automation of visual inspections
For manual visual inspection, experts define their subjective assessment using measurement scales. These are transformed into an objective quality concept. On the basis of this concept, a decision logic is built up which translates subjective assessments of the employee into an objective evaluation of the quality feature. The decision logic offers the employees in the visual inspection department a selection of options, which they can use to judge whether an existing component resembles a reference image on the measuring scale. This "pairwise comparison" is repeated until the component quality has been successfully classified.
Based on the results of this objectification, the decision logic is implemented in an application for smart devices in order to make it available to the employee regardless of location. By integrating the application into the employee's workflow right from the start, the component quality can be continuously evaluated during the manual visual inspection. A camera integrated in the Smart Device and aligned to the component records image data and the employee can classify the component in this way.
Both the image data and the classifications based on the measurement scales serve as a basis for a machine learning model. A supervised learning algorithm recognizes patterns and correlations in the recorded training data. The resulting models can then classify the components automatically after training.[2]
[1] Goschy, W. and Rohrbach, T. (2017). German Industry 4.0 Index 2017. STAUFEN.AG (Ed.)
[2] Mende, Hendrik; Peters, Alexander; Ibrahim, Faruk; Schmitt, Robert H. (2022). Integrating deep learning and rule-based systems into a smart devices decision support system for visual inspection in production. In: Procedia CIRP 109, p. 305–310.
Tabbed contents
Project supporting committee
The Fraunhofer IPT is performing the research in the "KOMBI" project. The following companies are part of the project-accompanying committee:
- adQuas human resources GmbH
- AiSight GmbH
- Aurubis Stolberg GmbH & Co. KG
- Calsitherm Verwaltungs GmbH
- CSP GmbH & Co. KG
- e.GO Mobile AG
- ELABO GmbH
- Güntert Präzisionstechnik GmbH
- Hammerwerk Fridingen GmbH
- IconPro GmbH
- ISRA VISION AG
- J. Schlenter Production GmbH
- LBBZ-NRW GmbH
- oculavis GmbH
- Pham & Schild GmbH
- Rhein-Getriebe GmbH
- Schoeller Werk GmbH & Co. KG
Funding information
The IGF project "KOMBI - Nutzbarmachung subjektiver Qualitätskriterien durch Kombination von Smart Devices und Machine Learning (IGF 21181N) of the research association FQS Forschungsgemeinschaft Qualität e. V. was funded via the AiF within the framework of the program for the promotion of joint industrial research (IGF) by the Federal Ministry for Economic Affairs and Climate Action based on a resolution of the German Bundestag.