Jump to content

PhyCV

Add links
From Wikipedia, the free encyclopedia

This is an old revision of this page, as edited by Eejit43 (talk | contribs) at 03:57, 30 October 2022 (Adding short description: "Computer vision library"). The present address (URL) is a permanent link to this revision, which may differ significantly from the current revision.

File:Realtime phycv pst.gif
Real-time edge detection at 35 frames per second using PhyCV on a 4K video (frame size 3840x2160).
File:Retina phycv pst.png
Retina vessel detection using PhyCV.
File:Sunflower phycv page.png
Directional edge detection of a sunflower using PhyCV.

PhyCV is the first computer vision library which utilizes algorithms directly derived from the equations of physics governing physical phenomena. The algorithms appearing in the first release emulate the propagation of light through a physical medium with natural and engineered diffractive properties followed by coherent detection. Unlike traditional algorithms that are a sequence of hand-crafted empirical rules, physics-inspired algorithms leverage physical laws of nature as blueprints. In addition, these algorithms can, in principle, be implemented in real physical devices for fast and efficient computation in the form of analog computing.[1] Currently PhyCV has two algorithms, Phase-Stretch Transform (PST) and Phase-Stretch Adaptive Gradient-Field Extractor (PAGE). Both algorithms have CPU and GPU versions. PhyCV is now available on GitHub and can be installed from pip.

History

Phase-Stretch Transform (PST) and Phase-Stretch Adaptive Gradient-field Extractor (PAGE) were inspired by the physics of the photonic time stretch[2][3] (a hardware technique for ultrafast and single-shot data acquisition). The edge detection algorithm PST was open-sourced in 2016 and as of mid 2022 has 800+ stars and 200+ forks on GitHub. The directional edge detection algorithm PAGE was open-sourced in February, 2022. PhyCV was originally developed and open-sourced by Jalali-Lab @ UCLA in May 2022. In PhyCV, the original open-sourced code of PST and PAGE is significantly refactored and improved to be modular, more efficient, GPU-accelerated and object-oriented.

Background

Phase-Stretch Transform (PST)

Phase-Stretch Transform (PST) is a computationally efficient edge and texture detection algorithm with exceptional performance in visually impaired images.[4][5][6] The algorithm transforms the image by emulating propagation of light through a device with engineered diffractive property followed by coherent detection. It has been applied in improving the resolution of MRI image,[7] extracting blood vessels in retina images,[8] dolphin identification,[9] and waste water treatment,[10] single molecule biological imaging,[11] and classification of UAV using micro Doppler imaging.[12]

Phase-Stretch Adaptive Gradient-Field Extractor (PAGE)

Phase-Stretch Adaptive Gradient-field Extractor (PAGE) is a physics-inspired algorithm for detecting edges and their orientations in digital images at various scales.[13][14] The algorithm is based on the diffraction equations of optics. Metaphorically speaking, PAGE emulates the physics of birefringent (orientation-dependent) diffractive propagation through a physical device with a specific diffractive structure. The propagation converts a real-valued image into a complex function. Related information is contained in the real and imaginary components of the output. The output represents the phase of the complex function.

Highlights

Modular Code Architecture

Modular code architecture of PhyCV algorithms.

The code in PhyCV has a modular design which faithfully follows the physical process from which the algorithm was originated. Both PST and PAGE modules in the PhyCV library emulate the propagation of the input signal (original digital image) through a device with engineered diffractive property followed by coherent (phase) detection. The dispersive propagation applies a phase kernel to the frequency domain of the original image. This process has three steps in general, loading the image, initializing the kernel and applying the kernel. In the implementation of PhyCV, each algorithm is represented as a class in Python and each class has methods that simulate the steps described above. The modular code architecture follows the physics behind the algorithm. Please refer to the source code on GitHub for more details.

GPU Acceleration

PhyCV supports GPU acceleration. The GPU versions of PST and PAGE are built on PyTorch accelerated by the CUDA toolkit. The acceleration is beneficial for applying the algorithms in real-time image video processing and other deep learning tasks. The running time PhyCV algorithms on CPU and GPU for videos at different resolutions are shown below.

Running Time (per frame) of PhyCV Edge Detection on Videos
CPU (Intel i9-9900K) GPU (NVIDIA TITAN RTX)
1080p (1920x1080) 550 ms 6.5 ms
2K (2560 x 1440) 1000 ms 12 ms
4K (3840 x 2160) 2290 ms 28.5 ms
Running Time (per frame) of PhyCV Directional Edge Detection on Videos
CPU (Intel i9-9900K) GPU (NVIDIA TITAN RTX)
1080p (1920x1080) 2800 ms 55 ms
2K (2560 x 1440) 5000 ms 95 ms
4K (3840 x 2160) 11660 ms 219 ms

Installation and Examples

Please refer to the GitHub README file for a detailed technical documentation.

Current Limitations

I/O (Input/Output) Bottleneck for Real-time Video Processing

When dealing with real-time video streams from cameras, the frames are captured and buffered in CPU and have to be moved to GPU to run the GPU-accelerated PhyCV algorithms. This process is time-consuming and it is a common bottleneck for real-time video-processing algorithms.

Lack of Parameter Adaptivity for Different Images

Currently the parameters of PhyCV algorithms have to be manually tuned for differently images. Although a set of pre-selected parameters work relatively well for a wide-range of images, the lack of parameter adaptivity for different images remains a limitation for now.

See Also

Reference

  1. ^ Physics-based Feature Engineering. Jalali et al. Optics, Photonics and Laser Technology, 2019
  2. ^ Time-stretched analogue-to-digital conversion. Bhushan et al. Electronic Letters techniques, 1998
  3. ^ Time stretch and its applications. Mahjoubfar et al. Nature Photonics, 2017
  4. ^ Physics-inspired image edge detection. Asghari et al. IEEE Global Signal and Information Processing Symposium, 2014
  5. ^ Edge detection in digital images using dispersive phase stretch. Asghari et al. International Journal of Biomedical Imaging, 2015
  6. ^ Feature Enhancement in Visually Impaired Images. Suthar et al. IEEE Access, 2018
  7. ^ Fast Super-Resolution in MRI Images Using Phase Stretch Transform, Anchored Point Regression and Zero-Data Learning. He et al. IEEE International Conference on Image Processing, 2019
  8. ^ A local flow phase stretch transform for robust retinal vessel detection. Challoob et al. In International Conference on Advanced Concepts for Intelligent Vision Systems, 2020
  9. ^ Dolphin Identification Method Based on Improved PST. Wang et al. In 2021 IEEE/ACIS 6th International Conference on Big Data, Cloud Computing, and Data Science (BCD), 2021
  10. ^ Image segmentation of activated sludge phase contrast images using phase stretch transform. Ang et al. Microscopy, 2019
  11. ^ Phase stretch transform for super-resolution localization microscopy. Ilovitsh et al. In Biomedical Optics Express, 2016
  12. ^ Classification of Drones Using Edge-Enhanced Micro-Doppler Image Based on CNN. Singh et al. Traitement du Signal, 2021
  13. ^ Phase-stretch adaptive gradient-field extractor (page). Suthar et al. Coding Theory, 2020
  14. ^ Phase-Stretch Adaptive Gradient-Field Extractor (PAGE). MacPhee et al. arXiv preprint arXiv:2202.03570, 2022
Retrieved from "https://en.wikipedia.org/w/index.php?title=PhyCV&oldid=1118988523"