Machine vision (MV), which utilising the optical tools and computer technology, could be applied to collect object data for processing and comprehension as well as to simulate human visual function.
It has been widely used in automobile, transportation, pharmaceutical, aerospace, military, and many other fields. Nowadays, it has attracted much attention in the field of intelligent manufacturing.
MV as an interdisciplinary technique, encourages traditional manufacturing processes to become more intelligent. It requires expertise in a variety of fields, which including optical imaging, signal processing, image processing, machine learning, and deep learning.
Academics have researched the digital image processing in considerable depth since the 1950s, when the concept of MV was originally proposed. From the 1970s to the 1990s, a range of disciplinary procedures was combined with MV to enrich and implement its theoretical structure.
In the 21st century, with the rapid development of artificial intelligence, machine learning and deep learning technology, MV can achieve higher accuracy and more applications. It has been commonly utilised in furniture workshops as the application scope of MV has gradually expanded.
The application of MV in furniture manufacturing has hastened the transition of modern furniture manufacturing workshops from manual to intelligence manufacture.
The advantages of MV include e its high detection accuracy, quick speed, and strong coordination. It can increase the intelligence of wooden products manufacturing workshops, accelerate production, improve detection accuracy and reduce accident risk.
The method of system literature review (SLR) was applied to retrieve, sort out and analyse the relevant literature, and finally summarised the application status and technical realisation methods of MV in the furniture manufacturing process. SLR includes the following steps.
Step 1 is to define the boundaries of the concept. Considering the objectives and research questions of the review, conceptual boundaries are defined.
Consistent with our goal of exploring the application of MV in the furniture manufacturing industry, we have defined neither the MV used in this paper nor any restrictions on the furniture manufacturing process that applies these technologies.
Our research therefore includes papers on MV applied to other industries, both independently and in combination with other advanced manufacturing technologies.
Step 2 is to determine the retrieval criteria.
Search database: the web of science core collection was used. Search time range: 2011–2022. Search keywords: 10 keywords related to MV and 20 keywords related to the furniture manufacturing industry were selected for combination, to determine the papers that may be included in the review.
MV keywords included the key equipment and key technology. The keywords related to the furniture manufacturing industry were determined according to the actual demand of furniture manufacturing workshops.
We searched the keywords related to MV and the furniture manufacturing industry in the title of the literature, and a total of 2601 literatures were retrieved.
Step 3 is to select and include relevant literature. By selecting the categories of web of science, 219 literatures related to these categories were further screened.
After reading the abstracts of these 219 literatures, some literatures that are not relevant to the content of this study were excluded, such as medical diagnosis, business prediction model and camera calibration, and 128 literatures were retained at last.
Step 4 is to extract, analyse and synthesize key information such as application model and key technologies in the literature. According to the extracted application model, the MV application was summarised into four aspects, such as information collecting, quality detecting, positioning and automatic sorting, and intelligent monitoring.
According to the findings of the collation and analysis, the application aspects with little literatures should be searched separately, and the time and scope restrictions should be relaxed. The practical application technology of these four aspects was described in details as follows.
Information Collecting
The initial stage of furniture production processing is information gathering. MV can immediately locate the coding position in coding information collection line during the furniture manufacturing.
It has the advantages of shorter positioning time, smaller difficulty of coding recognition, higher accuracy, which adapt to real-time online detection.
MV was employed by Lin and Xu to batch recognise encoded information in high-speed motion, and both the detection accuracy and efficiency were higher than manual recognition.
Barcodes with slight defects can also be recognized by MV, and the recognition accuracy could reach to about 100 percent. The encoded information on the furniture components and parts, such as present position, time, and forthcoming operation data, could be captured by MV and uploaded to guide the master database for workpiece’s further processing or troubleshooting.
The gathered data can also be used to analyse the inventory status and compile statistics, enabling automatic storage information management, and assist to control the production process.
Quality Detecting
The quality detecting during furniture manufacturing process is to ensure the quality of furniture components and parts and prevent the outflow of defective products.
For the panel furniture, the objects that need to be examined include dimensional defect, dimensional accuracy of hole processing, surface defect, and colour difference. At present, manual detection is still the main method in furniture workshops.
Colour and surface defects are identified by employees’ experience, and the dimension and position of the hole are mostly measured by tape. The detection efficiency is low, the error rate of the check process is high, and there is also a risk of scratching the panel during manual detection.
The application of MV inspection can make up for the deficiencies of manual inspection, achieving higher efficiency and lower error rate, and making the measurement data more precise during the non-contact inspection process.
In the beginning of quality detecting technology based on MV, three steps of image acquisition, image processing, and image segmentation are needed to make preliminary preparation.
Image Acquisition
Image acquisition is an important link in the realisation of MV quality inspection technology. Acquiring high-quality images can reduce the difficulty of subsequent image processing.
In the field of traditional machine vision, two visible light sensors, either charge-coupled devices (CCD) and complementary metal-oxide-semiconductor sensors (CMOS), are often used. Their photoelectric conversion principle is the same.
Compared to CCD, CMOS has the advantages of low power consumption, but it produces more noise. CCD sensors have high sensitivity and low read noise, but consume high power.
Infrared photoelectric sensors can capture thermal data and play an important role in the measurement of temperature. Compared with optical sensors, it is more suitable for all-weather work, and it is more effective in detecting long-distance targets.
In recent years, sensors that incorporate 2D materials (metals, semi-metals, semiconductors and insulators, etc.) have become strong competitors to traditional sensors due to their thinness and wide detection band. In the future, they may become the main sensors for the next generation of machine vision.
Image Preprocessing
Image preprocessing includes image denoising, image enhancement and other operations. Common denoising methods include median filtering, mean filtering, Gaussian filtering, bilateral filtering, etc., which can effectively reduce image noise.
The commonly used image enhancement methods are grey transformation, colour space change, etc., which can effectively reduce the interference of unimportant information, such as image background on subsequent image processing.
In recent years, deep learning has been applied to image noise reduction techniques and performed well in practical applications. Chen, Zhang, and Cruz apply convolutional neural network (CNN) to denoise images, and the effect is good in practical denoising applications, which is better than traditional methods.
Image Segmentation
Image segmentation is an important basis for feature extraction. In the actual application process, according to different usage scenarios, selecting an appropriate image segmentation algorithm can effectively improve the quality of the segmented image and be better applied to subsequent operations.
Traditional image segmentation methods mainly include threshold-based segmentation methods, edge-based segmentation methods, region-based segmentation methods, and specific theory-based segmentation methods.
In recent years, image segmentation based on deep learning has become the mainstream of research. Shervin summarised the application of deep learning in image segmentation in detail, introduced image segmentation models based on neural network, Encoder-Decoder, Multiscale and Pyramid Network, Dilated Convolutional, etc., and evaluated the segmentation performance.
Then, different algorithms are selected for practical applications. At present, the quality detection of furniture parts mainly includes dimensional defect detection, hole processing quality detection, surface defect identification and colour difference detection.
Dimensional defect detection is used to determine the length, width, and thickness. Hole processing quality detection can identify the hole’s location, number, dimension, and other characteristics on the products components and parts.
MV can also automatically identify and classify surface defects in wooden products, such as scratches, potholes, dead knots, slip knots, and wormhole, etc.
Colour is one of the key parameters, which affects the products appearance quality, so large colour difference is not allowed. MV could be applied for colour sorting, and the same colour level of product will be put into the same category.
Dimensional Defect Detection
The dimensional defect detection technology based on MV can accomplish the detection work with high real-time, high accuracy, high efficiency, low error, and non-contact.
The most common dimensional defect detection method involves pre-processing the image, using the Hough transform (HT. Its basic idea is to convert the detection problem from the image space into the parameter space, complete inspection tasks using straightforward statistics accumulated in the parameter space, and describe the edge of the image by some form of parameter that satisfies most of the edge points.) to identify the workpiece’s outline information, finding the edge line and transforming it, rotating the image of workpiece’s edge to the horizontal position in accordance with the slope of the line, and then calculating the actual physical distance between the pixels using camera calibration data.
The measurement data is uploaded to the database, and compared with the database data. If the error is within the range, the workpiece will be retained and entered the next process. Otherwise, it will be eliminated.
However, the Hough transform method has a large amount of calculation and is seriously affected by noise, which requires high-quality collected pictures.
Only applying the Hough change cannot meet the variety of detection needs due to the complexity of workpiece and the enhancement of dimensional defect detection requirements.
In recent years, many scholars have made more and more in-depth research on dimensional defect detection by MV. Edge detection algorithms (such as the Canny operator, and Sobel operator) and particle swarm algorithms have been proposed and applied to the field of dimensional defect detection. The detection range extends from length and width to angle, radian, diagonal length, and even thickness.
Hole Processing Detection
Hole information, which detected by MV, can greatly reduce the time requirement, improve detection accuracy, and optimise the detection process.
The commonly used detection method of hole characteristic parameters is similar to the dimensional defect detection. Firstly, the region of interest (ROI) will be extracted. Then, using HT to detect the edge information of the hole on the ROI area. At last, the ellipse fitting will be used.
The fitted circle presents the hole in products parts. The hole information will be compared with the database data, and the reasonable error is considered qualified.
Although, the detection method based on Hough change is traditional, it still has great application value. Scholars improved the basis of HT and proposed probabilistic HT, randomised HT, fuzzy HT, curvature aided HT, etc., which can be better applied to circle detection.
Li used a semi-supervised learning algorithm for circular hole detection. This method can meet the circular hole detection requirements of plates with complex textures. The measurement accuracy can reach to 0.03 mm.
However, the existing literature mainly focused on the detection of hole diameter information, and seldom concerned the depth information of hole.
3D point cloud technology solves the problem of hole depth measurement from the 3D level. The edge feature points of the hole are retrieved and refined based on the 3D point cloud data of the hole. To acquire data on hole distance and aperture, the feature points are parameterised by a space circle in the final step.
Surface Defect Identification
Surface defect identification by MV includes three steps of image pre-processing, defect area localization, and defect feature extraction. Firstly, image pre-processing includes eliminating noise and other interference information, enhancing image contrast, and making the features in the image salient.
Secondly, different methods of the model comparison method and image segmentation method, will be used to locate the defect areas for the different objects. Thirdly, some relevant algorithms would be selected to analyse the location area, extract defect features, and identify and classify defect types.
At present, the surface defect identification methods based on MV are mainly divided into three categories: defect detection based on image processing, defect detection based on machine learning, defect detection based on deep learning.
Surface defect detection, which based on image processing, applies an image segmentation algorithm to extract defect surface features. Canny and Sobel operators are the more commonly used algorithms to extract defect edges. And the feature extraction or template matching is applied to identify defects.
This defect recognition method is simple, widely used, and has good robustness. But it is difficult to deal with some complex changes in the defect and has no autonomous learning ability. It is only suitable for a single fixed detection target. In recent years, the convolution neural network (CNN), deep belief network (DBN), and residual neural network model (ResNet) that have deep learning capabilities are gradually applied for defect detection.
Compared with image processing and machine learning, these deep learning methods have higher detection efficiency and the ability to learn. However, their calculation is complicated, many datasets are needed to be trained.
Kamal utilised grey level co-occurrence matrix and laws texture energy measures as texture feature extractors, and chose feed-forward back-propagation neural network as classifier to identify wood defects. The results showed that that method had an accuracy of 84.3 percent.
Colour Difference Detection
Based on MV, fast, efficient, and low-error colour difference detection can be realized. The colour difference detection based on MV is different from the above three aspects. It needs to use a colour camera to collect colour images.
Commonly used detection methods need to pre-select multiple groups of boards with different colour grades, extract the values of the colour space (The colour space models mainly include HSV, RGB, HSI, CHL, LAB, CMY, etc.), and perform training, testing, and verification.
At the beginning of the application, image acquisition and pre-processing are executed. Then, locate the workpiece area, remove the interference of the background colour to the detection, and extract the colour features. At last, the classifier algorithm will be applied for colour classification.
Colour difference detection involves deep learning algorithms and requires many data sets for training, testing, and verification. The more sufficient the data set, the more significant the grading effect will be.
The colour information the wood board, which captured by colour camera, will be affected by the ambient light condition. In the factory actual application, the change of ambient light greatly interferes with the results of the colour difference inspection for furniture.
Placing a hood around the camera and lights can effectively reduce the impact of ambient light condition on the colour information collection. In the actual application process, the combination of algorithms such as extreme learning machine (ELM), multi-layer perceptron (MLP), and grey wolf algorithm is more competent to achieve the optimal recognition effect.
Positioning & Automatic Sorting
The MV positioning technology can realise the accurate determination of the workpiece position, which is convenient to analyse and use the workpiece position information for subsequent processing operations.
For example, automatic sorting of furniture parts, precise spraying of paint spray gun, etc. are all need MV technology to assist in their operations. After the quality detecting process, the products will be sorted and put into storage.
The manual sorting process requires workers to move the workpiece to the specified position after validating the information obtained by scanning the board’s bar code or electronic label.
A 250-person furniture manufacturing workshop can produce about 20,000 parts per shift (8 hr). If there are 10 sorting employees, each sorting employee needs to sort an average of 2000 boards per shift. It is labour-intensive, inefficient, and costly, and cannot accommodate the current trend of mass customization.
MV can calculate the three-dimensional coordinates of objects in the real world, and guide the manipulator to locate the plate. MV can significantly improve sorting efficiency, reduce the error rate, and reduce labour costs effectively. Wu combined robot technology with MV to identify and sort the sorting objects at a high speed, which improved the automation and intelligence level of flexible manufacturing production lines.
Intelligent Monitoring
The application of MV in monitoring workshops is to use computer equipment and learning algorithms to quickly analyse the specific content information in the video.
It is primarily used to monitor three aspects, including employee behaviour, storage status, and device status.
Furniture manufacturing workshops include workers and equipment. Workers have a certain risk of injury during some operations, and if these operators do not follow conventional operating procedures, a serious mishap would be quickly occur.
Nowadays, some smart monitoring devices are often used to monitor and evaluate employee behaviour, such as monitoring whether the employees wear safety helmets and whether the employees have dangerous behaviours.
The data of workers’ long-term actions are gathered and examined by MV, and these data would be used as a reference for guiding the workers’ behaviour.
Currently, intelligent monitoring technology mostly uses the YOLOv4 algorithm to locate and evaluate targets and to send out early warning signals whenever a situation deviates from previously established standards (such as workers are not wearing safety helmets, smoking, not in the key positions).
MV also could be applied for storage status monitoring in the workshop. The monitoring of the inventory can reflect the inventory status in real-time, and alarm in the case of inventory backlog or shortage, so that the staff can timely supply the warehouse.
The state of equipment affects the normal operation of production, which could be monitored by visual monitoring technology, such as monitoring equipment tool wear, equipment conveyor belts. Real-time understanding of device status could effectively reduce the downtime of machine failure.
Limitations
Site Environment
Most of the development and research of MV system is completed in a specific environment, which has poor adaptability and versatility.
Once the environment changes (such as light source, conveyor belt colour, etc.), the speed and accuracy of the detection algorithm will be affected.
In the future, MV technology should be improved by hardware and software optimization, to achieve stronger adaptability and versatility.
Device Performance
MV is also limited by equipment performance, which includes two aspects of camera performance and computer computing power.
The limited observation range of the camera, the distortion produced by the lens, the quality of the sensor, etc., affect the quality collected image.
The performance of the computer graphics card has a great influence on the speed of image processing.
Insufficient performance of the computer graphics card will affect the running speed of the image processing algorithm, resulting in a short-term stagnation of the entire workshop production line, making it difficult to improve industrial production efficiency.
Detection Diversity
The needs in the production process of the workshop are diverse, and the single system is difficult to meet multiple detection requirements.
Therefore, most of the relevant research focuses on a single detection requirement. Under the same detection requirement, there are various objects to be detected.
In surface defects identification process, there are various types of defects, which is difficult for MV systems to extract all defect features. For the ambiguous defect characteristics, the processing capability of the MV system is still insufficient.
Scarcity Of Talents
Most workers in the manufacturing workshop do not understand the operating principle of the MV system and have no ability to independently deal with equipment failures.
They cannot adjust the relevant parameters according to the needs of the workshop. With the popularisation of education and the development of the MV industry, the number of MV talents is gradually increasing, and the impact of this problem is gradually weakening.
Directions For Future Work
3D MV Technology
With the increase of inspection requirements, 2D MV has been difficult to meet the needs of workshops. Structured light cameras, binocular and multi-eye vision, and other technologies that can obtain 3D information of objects are gradually applied in factories.
3D machine vision can break the bottleneck of the detection range of 2D machine vision, and meet the needs of industrial applications. It has gradually become the future development trend of the MV. In the future, using 3D MV combined with robots, AI and other new technologies can create an intelligent workshop.
Deep Learning
Traditional MV often requires the assistance of machine learning (Support Vector Machine, Decision Tree, etc.) and basic image processing algorithms (OSTU, Hough transform, Canny, etc.).
Sometimes, it is not competent for changing detection objects. The MV based on deep learning is more adaptable to change and provide greater versatility in practical applications, but it requires a large amount of labelled data for training.
The combination of traditional methods and deep learning can make up for each other’s shortcomings, reduce the difficulty of algorithm processing, and improve the running speed and performance.
MV is applied to the monitoring of the workshop to ensure personnel safety and equipment efficiency, information gathering at each manufacturing link to ensure manufacturing process transparency, check product quality to guarantee that no defective items are released, and upload alarm information for repair when the problems are discovered to prevent recurrence.
Automatic sorting of furniture parts could accomplish products classification and storage. The application of MV significantly increases the production management condition in furniture manufacturing workshops, as well as the degree of automation and production efficiency.
In the future, 3D MV could combine with other advanced manufacturing technologies to create a safe, intelligent, and efficient manufacturing workshop.
