ML-TN-003 — AI at the edge: visual inspection of assembled PCBs for defect detection — Part 3

Info Box Applies to Machine Learning Applies to Machine Learning Technical Notes

History

Version	Date	Notes
1.0.0	March 2021	First public release

Building the dataset

Defects generation and acquisition

Anomalies, generation process and numerosity

Anomaly	P&P	Serigraphy	Manual	Numerosity
Missing	✓			21
Manhattan			✓	4
Shift x-axis	✓			51
Shift y-axis	✓			58
Shift&Rotation (shift x+z-axes)	✓			57
Rotation (z-axis)	✓			55
Under soldering		✓		All
Over soldering		✓		All

Class subdivision and labelling

In order to build a dataset for training a ML model for a classification application, a set of classes has to be defined. By looking at the standard IPC-A-610E-2010 developed by IPC for the Acceptability of Electronic Assemblies, and at the features of collected images, the classes identified are the following ones:

• Acceptable: the AOI machine signals the component as a possible anomaly because the component image doesn’t respect the color constraints specified by the software of the machine but in truth, there is no defect. Generally, this occurs when the component is correctly soldered on both pads, but the amount of red and green color is too high with respect to the amount of blue. In this case, quality is not the target one but still is acceptable.
• Missing: the component is not in place, hence only the pads with applied solder are visible.
• Tombstoning: the component is lifted from a pad of the PCB; this class also comprehends all the cases for which the component is lifted and rotated by a certain amount.
• Under soldering: the component is soldered on both pads, but the amount of solder is too low. By looking at picture of the reported component, this is clearly visible when on a pad or both there is a higher amount of red with respect to the blue one.

To simplify the problem, all the classes into which the images are divided are mutually exclusive. manhattan and over soldering defect typologies are no longer included among the possible classes because their generation is too difficult hence the quantity of examples obtained is too low for training a model.

Examples of four types of defects

For labelling the images makesense−ai tool was used.

Acceptable	Missing	Tombstoning	Under soldering
17	408	376	24

It is evident that this dataset is unbalanced because it has a relatively low number of acceptable and under soldering defect images. Nevertheless, taking into account the overall results achieved, this first attempt can be considered a successful one.

Data augmentation with image synthesis

Progressive GAN implementation

ProGAN: training progression

ProGAN: growing progression of the model during training

Results validation

Class	Synth image	Resolution (pixel)	Google Colab (min)	AWS SageMaker (min)
missing	full	512 × 512	~480	~410
	upper/lower	256 × 256	~435	~310
tombstoning	full	512 × 512	~460	~390
	upper/lower	256 × 256	~420	~300

Synthesized images for missing class

Synthesized images for tombstoning class

T-SNE algorithm results for missing class synthesized images

512 × 512 resolution full images

256 × 256 resolution full images

256 × 256 resolution upper and lower region images

T-SNE algorithm results for tombstoning class synthesized images

512 × 512 resolution full images

256 × 256 resolution full images

256 × 256 resolution upper and lower region images

Useful links

• Tero Karras, Timo Aila, Samuli Laine, Jaakko Lehtinen, Progressive Growing of GANs for Improved Quality, Stability, and Variation, February 2018.
• Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, Aaron Courville, Improved Training of Wasserstein GANs, December 2017.
• IPC-A-610E: Acceptability of Electronic Assemblies, a standard developed by IPC, April 2010.