Using the Super Resolution method for 3d scanning or is it possible to jump above your head
How to achieve maximum resolution in 3d scanning, what equipment characteristics are required for this. If you get away from advertisements and dry numbers in tables, how to get a really decent result.
First of all, it is necessary to clarify that accuracy and resolution (detail) are two different characteristics, sometimes related to each other, but generally not. If we use simple words, then accuracy is how accurate the dimensions and distances are in millimeters, and detail is how small objects are clearly distinguishable on the results of a 3d scan. For example, to scan the microtext engraved on the shell of a chicken egg, it is precisely good detail that is needed, while the distortion of the shape of the egg itself is not so important, the main thing is to clearly see the microtext. And to analyze the shape of an egg, geometric accuracy is needed, while the very presence of microtext is not necessary or even interferes. Both indicators (accuracy and detail) have the same dimension – millimeters, but reflect different characteristics. With modern 3D scanning equipment, the Accuracy value is usually better (a number less) than the Detail value, but there are exceptions.
This article is about getting maximum detail on available equipment.
As possible technologies were considered – manual 3d scanning and scanning using structured illumination. Both technologies almost always use the principle of stereo vision, i.e. calculation of 3d coordinates through the processing of multi-angle images of an object, usually illuminated either by laser lines, or by special static patterns, or by coded sequences. If we consider the best performance, then, as a rule, hand-held scanners lose in detail to stationary scanners, because:
use a large scan area
are forced to receive 3d data from each image (stereo pair), while stationary scanners receive data as a result of processing a series of images
use lower resolution cameras, opting for higher frame rates instead
Some handheld scanners claim detail as low as 0.035mm. At the time of writing, this is an excellent indicator. We will achieve even better results using an inexpensive stationary scanner. Among stationary scanners, to achieve maximum detail, you need to pay attention to the following characteristics.
scan area size. The smaller the size of the scan area, the smaller the area of the object per pixel of the camera and, accordingly, the higher the detail
camera resolution. The higher the resolution, the higher the detail
blue light technology. A light filter is wound onto the camera lens, which passes only a narrow band of frequencies corresponding to blue light. The rest of the colors are blocked by a light filter. Working with a single frequency of light (monochromatic mode) eliminates the rainbow effect (chromatic aberration) in the lens and significantly improves its ability to create a sharp, focused image
a black-and-white or color matrix is used in the camera. A color matrix is needed to obtain color images. Initially, a photosensitive material does not distinguish colors, it absorbs photons and in return “creates” electrons that carry no color information. To obtain a color image, the pixels on the matrix itself are complemented by micro-color filters (usually green, red and blue) in a chess-like order (Bayer filter), this is done during the production of the matrix, usually each block of 2×2 pixels is painted in this way. Accordingly, such a matrix uses 4 pixels to obtain one color value. There are more complex algorithms (“debayerization algorithms”) that use a larger number of neighboring pixels. The effect of using color matrices in 3d scanning is as follows – when using ordinary white light, the image is “blurred” due to the averaging of pixels by 2×2 blocks, i.e. the effective resolution of such a matrix drops by about 4 times. The use of “blue light” technology with color matrices does not make practical sense, because. matrix pixels with green and red filters will not see anything. As a result, only a black and white matrix is suitable for obtaining high resolution 3d scanning. Some manufacturers of fairly well-known scanners are cunning, indicating the full resolution of a color camera as equivalent to the resolution of a black and white
quality lens is used. To obtain a clear image, it is necessary to use a lens with a resolution corresponding to the pixel size of the camera matrix. If the pixel sizes are too small, the lens will not be able to create a clear image on the sensor. It is also necessary to take into account that high-quality lenses do not show their properties to the maximum under all shooting conditions. For example, with a very closed aperture, the depth of field increases, but the clarity of the image decreases, which means that the detail also decreases.
the projector can focus well on the scanned area. When working with small scan areas, it is necessary that the projector that performs structured illumination can focus well, and its illumination area practically coincides with the scan area
the software contains a sufficient number of settings. The settings for changing the number of frames during 3d scanning, for changing the “bandwidth” of the backlight, for changing the brightness of the backlight are important. These parameters are possible when using the “Phase Shifting” highlighting method consisting of sinusoids, as opposed to line scanning. Also important is scanning in automatic mode on the turntable and the ability to select a large number of positions per revolution. The algorithm for constructing the final grid of triangles from the original scans must be able to “pull” information from a large number of scans. Those. as the number of scans increases, the noise should decrease, and the detail, at least, should not deteriorate. Whether the detail will increase is the research objective for this article.
structural rigidity and calibration quality. These characteristics are often overlooked, although they are the basis for stereo vision measurements. It is difficult to catch microns when the camera is fixed through plastic parts, the aluminum profile is not rigid enough, or the calibration is implemented “for the convenience of the user” by one frame of the calibration field
Separately, I would like to dwell on the accuracy of the calibration. To implement the Super Resolution method, it is not enough just to take a large number of scans from different angles. It is necessary to bring these scans into one coordinate system with the highest possible accuracy. For this to be possible, the accuracy of the 3d scan must be very high. Ideally, an order of magnitude higher than the resolution that is planned to be obtained. Those. if it is planned to obtain a detail of 0.01 mm (10 microns), the divergence of scans due to calibration errors should not exceed 0.001-0.002 mm (1-2 microns).
Among the equipment available to me, the most suitable scanner for research turned out to be VT MINI with 6.3 Mpix cameras, operating in single-camera mode. This scanner is suitable for scanning objects in a very wide range of sizes, from jewelry to cars and more. We will be interested in the following mode of its operation:
full resolution black and white camera 6.3 Mpix
scanning area 50 mm. Working in Single Camera Mode
scanning mode Phase Shifting, the number of frames with sinusoids – 60
highlight period width – 10 pixels
uses blue light technology
scanning mode – on the turntable
scanning at a reduced projector brightness to sharpen the focus of the camera lens
The selected scan area has a height of approximately 50 mm, which for a camera resolution of 3088×2064 gives a single pixel size of 50 / 2064 = 0.0242 mm. Those. the original resolution of one scan, with some rounding, approximately corresponds to 25 microns.
assembling the scanner to work with an area of 50 mm
chamber warm-up and calibration
scanning of selected objects in automatic mode on a turntable, 180 positions per full revolution
processing of received groups of scans (180 pieces each):
global alignment of scans by geometry
cutting off excess, only the central part of 30×30 mm remains
re-registration of scans on the remaining geometry, to achieve maximum alignment accuracy
construction of the resulting surface with a resolution of 0.025 mm and 0.01 mm
comparison of the obtained results with each other and with the original single scan
Dried gadfly and a drill with a diameter of 0.5 mm were chosen as experimental objects. The drill had to be matted with a titanium oxide spray. The gadfly did not matte.
Figure 2 shows a geometric feature on the left wing of the gadfly.
single scan with a resolution of 0.025 mm, taken from an oblique angle
the result, built with a resolution of 0.025 mm from 180 scans, shows a significant reduction in noise due to averaging
result built with a resolution of 0.01 mm from 180 scans, you can see an increase in resolution compared to a single scan
Figure 3 shows a 0.5 mm drill, from left to right:
single drill scan
processing result of 180 scans, built with a resolution of 0.025 mm
processing result built with a resolution of 0.01 mm. You can see a slight improvement in detail, especially at the tip. But it is not so noticeable due to the layer of matting and increased noise level.
image of a drill from a scanner camera
If you really want to squeeze better performance out of the equipment, the Super Resolution method may well help. Best of all, the effect of increasing the resolution is seen not on the technical object “drill”, but on the organic object “gadfly”, because it contains very narrow width elements and has not been matted.
If you choose the right equipment for this, you can jump noticeably higher than your head, but for this you will have to try – make a large number of jumps and process the results using Super Resolution methods.