To install click the Add extension button. That's it.

The source code for the WIKI 2 extension is being checked by specialists of the Mozilla Foundation, Google, and Apple. You could also do it yourself at any point in time.

4,5
Kelly Slayton
Congratulations on this excellent venture… what a great idea!
Alexander Grigorievskiy
I use WIKI 2 every day and almost forgot how the original Wikipedia looks like.
What we do. Every page goes through several hundred of perfecting techniques; in live mode. Quite the same Wikipedia. Just better.
.
Leo
Newton
Brights
Milds

Computer vision

From Wikipedia, the free encyclopedia

Computer vision is an interdisciplinary scientific field that deals with how computers can be made to gain high-level understanding from digital images or videos. From the perspective of engineering, it seeks to automate tasks that the human visual system can do.[1][2][3]

Computer vision tasks include methods for acquiring, processing, analyzing and understanding digital images, and extraction of high-dimensional data from the real world in order to produce numerical or symbolic information, e.g. in the forms of decisions.[4][5][6][7] Understanding in this context means the transformation of visual images (the input of the retina) into descriptions of the world that can interface with other thought processes and elicit appropriate action. This image understanding can be seen as the disentangling of symbolic information from image data using models constructed with the aid of geometry, physics, statistics, and learning theory.[8]

The scientific discipline of computer vision is concerned with the theory behind artificial systems that extract information from images. The image data can take many forms, such as video sequences, views from multiple cameras, or multi-dimensional data from a medical scanner. The technological discipline of computer vision seeks to apply its theories and models to the construction of computer vision systems.

Sub-domains of computer vision include scene reconstruction, event detection, video tracking, object recognition, 3D pose estimation, learning, indexing, motion estimation, and image restoration.[6]

YouTube Encyclopedic

  • 1/5
    Views:
    173 365
    9 594
    69 290
    164 299
    5 043
  • ✪ Computer Vision: Crash Course Computer Science #35
  • ✪ Computer Vision vs Image Processing
  • ✪ How Computer Vision Works
  • ✪ Computer Engineering Careers and Subfields
  • ✪ Computer Vision for Faces : Best Project Award

Transcription

Hi, I’m Carrie Anne, and welcome to Crash Course Computer Science! Today, let’s start by thinking about how important vision can be. Most people rely on it to prepare food, walk around obstacles, read street signs, watch videos like this, and do hundreds of other tasks. Vision is the highest bandwidth sense, and it provides a firehose of information about the state of the world and how to act on it. For this reason, computer scientists have been trying to give computers vision for half a century, birthing the sub-field of computer vision. Its goal is to give computers the ability to extract high-level understanding from digital images and videos. As everyone with a digital camera or smartphone knows, computers are already really good at capturing photos with incredible fidelity and detail – much better than humans in fact. But as computer vision professor Fei-Fei Li recently said, “Just like to hear is the not the same as to listen. To take pictures is not the same as to see.” INTRO As a refresher, images on computers are most often stored as big grids of pixels. Each pixel is defined by a color, stored as a combination of three additive primary colors: red, green and blue. By combining different intensities of these three colors, what’s called a RGB value, we can represent any color. Perhaps the simplest computer vision algorithm – and a good place to start – is to track a colored object, like a bright pink ball. The first thing we need to do is record the ball’s color. For that, we’ll take the RGB value of the centermost pixel. With that value saved, we can give a computer program an image, and ask it to find the pixel with the closest color match. An algorithm like this might start in the upper right corner, and check each pixel, one at time, calculating the difference from our target color. Now, having looked at every pixel, the best match is very likely a pixel from our ball. We’re not limited to running this algorithm on a single photo; we can do it for every frame in a video, allowing us to track the ball over time. Of course, due to variations in lighting, shadows, and other effects, the ball on the field is almost certainly not going to be the exact same RGB value as our target color, but merely the closest match. In more extreme cases, like at a game at night, the tracking might be poor. And if one of the team's jerseys used the same color as the ball, our algorithm would get totally confused. For these reasons, color marker tracking and similar algorithms are rarely used, unless the environment can be tightly controlled. This color tracking example was able to search pixel-by-pixel, because colors are stored inside of single pixels. But this approach doesn’t work for features larger than a single pixel, like edges of objects, which are inherently made up of many pixels. To identify these types of features in images, computer vision algorithms have to consider small regions of pixels, called patches. As an example, let’s talk about an algorithm that finds vertical edges in a scene, let’s say to help a drone navigate safely through a field of obstacles. To keep things simple, we’re going to convert our image into grayscale, although most algorithms can handle color. Now let’s zoom into one of these poles to see what an edge looks like up close. We can easily see where the left edge of the pole starts, because there’s a change in color that persists across many pixels vertically. We can define this behavior more formally by creating a rule that says the likelihood of a pixel being a vertical edge is the magnitude of the difference in color between some pixels to its left and some pixels to its right. The bigger the color difference between these two sets of pixels, the more likely the pixel is on an edge. If the color difference is small, it’s probably not an edge at all. The mathematical notation for this operation looks like this – it’s called a kernel or filter. It contains the values for a pixel-wise multiplication, the sum of which is saved into the center pixel. Let’s see how this works for our example pixel. I’ve gone ahead and labeled all of the pixels with their grayscale values. Now, we take our kernel, and center it over our pixel of interest. This specifies what each pixel value underneath should be multiplied by. Then, we just add up all those numbers. In this example, that gives us 147. That becomes our new pixel value. This operation, of applying a kernel to a patch of pixels, is call a convolution. Now let’s apply our kernel to another pixel. In this case, the result is 1. Just 1. In other words, it’s a very small color difference, and not an edge. If we apply our kernel to every pixel in the photo, the result looks like this, where the highest pixel values are where there are strong vertical edges. Note that horizontal edges, like those platforms in the background, are almost invisible. If we wanted to highlight those features, we’d have to use a different kernel – one that’s sensitive to horizontal edges. Both of these edge enhancing kernels are called Prewitt Operators, named after their inventor. These are just two examples of a huge variety of kernels, able to perform many different image transformations. For example, here’s a kernel that sharpens images. And here’s a kernel that blurs them. Kernels can also be used like little image cookie cutters that match only certain shapes. So, our edge kernels looked for image patches with strong differences from right to left or up and down. But we could also make kernels that are good at finding lines, with edges on both sides. And even islands of pixels surrounded by contrasting colors. These types of kernels can begin to characterize simple shapes. For example, on faces, the bridge of the nose tends to be brighter than the sides of the nose, resulting in higher values for line-sensitive kernels. Eyes are also distinctive – a dark circle sounded by lighter pixels – a pattern other kernels are sensitive to. When a computer scans through an image, most often by sliding around a search window, it can look for combinations of features indicative of a human face. Although each kernel is a weak face detector by itself, combined, they can be quite accurate. It’s unlikely that a bunch of face-like features will cluster together if they’re not a face. This was the basis of an early and influential algorithm called Viola-Jones Face Detection. Today, the hot new algorithms on the block are Convolutional Neural Networks. We talked about neural nets last episode, if you need a primer. In short, an artificial neuron – which is the building block of a neural network – takes a series of inputs, and multiplies each by a specified weight, and then sums those values all together. This should sound vaguely familiar, because it’s a lot like a convolution. In fact, if we pass a neuron 2D pixel data, rather than a one-dimensional list of inputs, it’s exactly like a convolution. The input weights are equivalent to kernel values, but unlike a predefined kernel, neural networks can learn their own useful kernels that are able to recognize interesting features in images. Convolutional Neural Networks use banks of these neurons to process image data, each outputting a new image, essentially digested by different learned kernels. These outputs are then processed by subsequent layers of neurons, allowing for convolutions on convolutions on convolutions. The very first convolutional layer might find things like edges, as that’s what a single convolution can recognize, as we’ve already discussed. The next layer might have neurons that convolve on those edge features to recognize simple shapes, comprised of edges, like corners. A layer beyond that might convolve on those corner features, and contain neurons that can recognize simple objects, like mouths and eyebrows. And this keeps going, building up in complexity, until there’s a layer that does a convolution that puts it together: eyes, ears, mouth, nose, the whole nine yards, and says “ah ha, it’s a face!” Convolutional neural networks aren’t required to be many layers deep, but they usually are, in order to recognize complex objects and scenes. That’s why the technique is considered deep learning. Both Viola-Jones and Convolutional Neural Networks can be applied to many image recognition problems, beyond faces, like recognizing handwritten text, spotting tumors in CT scans and monitoring traffic flow on roads. But we’re going to stick with faces. Regardless of what algorithm was used, once we’ve isolated a face in a photo, we can apply more specialized computer vision algorithms to pinpoint facial landmarks, like the tip of the nose and corners of the mouth. This data can be used for determining things like if the eyes are open, which is pretty easy once you have the landmarks – it’s just the distance between points. We can also track the position of the eyebrows; their relative position to the eyes can be an indicator of surprise, or delight. Smiles are also pretty straightforward to detect based on the shape of mouth landmarks. All of this information can be interpreted by emotion recognition algorithms, giving computers the ability to infer when you’re happy, sad, frustrated, confused and so on. In turn, that could allow computers to intelligently adapt their behavior... maybe offer tips when you’re confused, and not ask to install updates when you’re frustrated. This is just one example of how vision can give computers the ability to be context sensitive, that is, aware of their surroundings. And not just the physical surroundings – like if you're at work or on a train – but also your social surroundings – like if you’re in a formal business meeting versus a friend’s birthday party. You behave differently in those surroundings, and so should computing devices, if they’re smart. Facial landmarks also capture the geometry of your face, like the distance between your eyes and the height of your forehead. This is one form of biometric data, and it allows computers with cameras to recognize you. Whether it’s your smartphone automatically unlocking itself when it sees you, or governments tracking people using CCTV cameras, the applications of face recognition seem limitless. There have also been recent breakthroughs in landmark tracking for hands and whole bodies, giving computers the ability to interpret a user’s body language, and what hand gestures they’re frantically waving at their internet connected microwave. As we’ve talked about many times in this series, abstraction is the key to building complex systems, and the same is true in computer vision. At the hardware level, you have engineers building better and better cameras, giving computers improved sight with each passing year, which I can’t say for myself. Using that camera data, you have computer vision algorithms crunching pixels to find things like faces and hands. And then, using output from those algorithms, you have even more specialized algorithms for interpreting things like user facial expression and hand gestures. On top of that, there are people building novel interactive experiences, like smart TVs and intelligent tutoring systems, that respond to hand gestures and emotion. Each of these levels are active areas of research, with breakthroughs happening every year. And that’s just the tip of the iceberg. Today, computer vision is everywhere – whether it’s barcodes being scanned at stores, self-driving cars waiting at red lights, or snapchat filters superimposing mustaches. And, the most exciting thing is that computer scientists are really just getting started, enabled by recent advances in computing, like super fast GPUs. Computers with human-like ability to see is going to totally change how we interact with them. Of course, it’d also be nice if they could hear and speak, which we’ll discuss next week. I’ll see you then.

Contents

Definition

Computer vision is an interdisciplinary field that deals with how computers can be made to gain high-level understanding from digital images or videos. From the perspective of engineering, it seeks to automate tasks that the human visual system can do.[1][2][3] "Computer vision is concerned with the automatic extraction, analysis and understanding of useful information from a single image or a sequence of images. It involves the development of a theoretical and algorithmic basis to achieve automatic visual understanding."[9] As a scientific discipline, computer vision is concerned with the theory behind artificial systems that extract information from images. The image data can take many forms, such as video sequences, views from multiple cameras, or multi-dimensional data from a medical scanner.[10] As a technological discipline, computer vision seeks to apply its theories and models for the construction of computer vision systems.

History

In the late 1960s, computer vision began at universities which were pioneering artificial intelligence. It was meant to mimic the human visual system, as a stepping stone to endowing robots with intelligent behavior.[11] In 1966, it was believed that this could be achieved through a summer project, by attaching a camera to a computer and having it "describe what it saw".[12][13]

What distinguished computer vision from the prevalent field of digital image processing at that time was a desire to extract three-dimensional structure from images with the goal of achieving full scene understanding. Studies in the 1970s formed the early foundations for many of the computer vision algorithms that exist today, including extraction of edges from images, labeling of lines, non-polyhedral and polyhedral modeling, representation of objects as interconnections of smaller structures, optical flow, and motion estimation.[11]

The next decade saw studies based on more rigorous mathematical analysis and quantitative aspects of computer vision. These include the concept of scale-space, the inference of shape from various cues such as shading, texture and focus, and contour models known as snakes. Researchers also realized that many of these mathematical concepts could be treated within the same optimization framework as regularization and Markov random fields.[14] By the 1990s, some of the previous research topics became more active than the others. Research in projective 3-D reconstructions led to better understanding of camera calibration. With the advent of optimization methods for camera calibration, it was realized that a lot of the ideas were already explored in bundle adjustment theory from the field of photogrammetry. This led to methods for sparse 3-D reconstructions of scenes from multiple images. Progress was made on the dense stereo correspondence problem and further multi-view stereo techniques. At the same time, variations of graph cut were used to solve image segmentation. This decade also marked the first time statistical learning techniques were used in practice to recognize faces in images (see Eigenface). Toward the end of the 1990s, a significant change came about with the increased interaction between the fields of computer graphics and computer vision. This included image-based rendering, image morphing, view interpolation, panoramic image stitching and early light-field rendering.[11]

Recent work has seen the resurgence of feature-based methods, used in conjunction with machine learning techniques and complex optimization frameworks.[15][16]. The advancement of Deep Learning techniques has brought further life to the field of computer vision. The accuracy of deep learning algorithms on several benchmark computer vision data sets for tasks ranging from classification, segmentation and optical flow has surpassed prior methods.[citation needed]

Related fields

Artificial intelligence

Areas of artificial intelligence deal with autonomous planning or deliberation for robotic systems to navigate through an environment. A detailed understanding of these environments is required to navigate through them. Information about the environment could be provided by a computer vision system, acting as a vision sensor and providing high-level information about the environment and the robot.

Artificial intelligence and computer vision share other topics such as pattern recognition and learning techniques. Consequently, computer vision is sometimes seen as a part of the artificial intelligence field or the computer science field in general.

Information engineering

Computer vision is often considered to be part of information engineering.[17][18]

Solid-state physics

Solid-state physics is another field that is closely related to computer vision. Most computer vision systems rely on image sensors, which detect electromagnetic radiation, which is typically in the form of either visible or infra-red light. The sensors are designed using quantum physics. The process by which light interacts with surfaces is explained using physics. Physics explains the behavior of optics which are a core part of most imaging systems. Sophisticated image sensors even require quantum mechanics to provide a complete understanding of the image formation process.[11] Also, various measurement problems in physics can be addressed using computer vision, for example motion in fluids.

Neurobiology

A third field which plays an important role is neurobiology, specifically the study of the biological vision system. Over the last century, there has been an extensive study of eyes, neurons, and the brain structures devoted to processing of visual stimuli in both humans and various animals. This has led to a coarse, yet complicated, description of how "real" vision systems operate in order to solve certain vision-related tasks. These results have led to a subfield within computer vision where artificial systems are designed to mimic the processing and behavior of biological systems, at different levels of complexity. Also, some of the learning-based methods developed within computer vision (e.g. neural net and deep learning based image and feature analysis and classification) have their background in biology.

Some strands of computer vision research are closely related to the study of biological vision – indeed, just as many strands of AI research are closely tied with research into human consciousness, and the use of stored knowledge to interpret, integrate and utilize visual information. The field of biological vision studies and models the physiological processes behind visual perception in humans and other animals. Computer vision, on the other hand, studies and describes the processes implemented in software and hardware behind artificial vision systems. Interdisciplinary exchange between biological and computer vision has proven fruitful for both fields.[19]

Signal processing

Yet another field related to computer vision is signal processing. Many methods for processing of one-variable signals, typically temporal signals, can be extended in a natural way to processing of two-variable signals or multi-variable signals in computer vision. However, because of the specific nature of images there are many methods developed within computer vision which have no counterpart in processing of one-variable signals. Together with the multi-dimensionality of the signal, this defines a subfield in signal processing as a part of computer vision.

Other fields

Beside the above-mentioned views on computer vision, many of the related research topics can also be studied from a purely mathematical point of view. For example, many methods in computer vision are based on statistics, optimization or geometry. Finally, a significant part of the field is devoted to the implementation aspect of computer vision; how existing methods can be realized in various combinations of software and hardware, or how these methods can be modified in order to gain processing speed without losing too much performance. Computer vision is also used in fashion ecommerce, inventory management, patent search, furniture, and the beauty industry.[citation needed]

Distinctions

The fields most closely related to computer vision are image processing, image analysis and machine vision. There is a significant overlap in the range of techniques and applications that these cover. This implies that the basic techniques that are used and developed in these fields are similar, something which can be interpreted as there is only one field with different names. On the other hand, it appears to be necessary for research groups, scientific journals, conferences and companies to present or market themselves as belonging specifically to one of these fields and, hence, various characterizations which distinguish each of the fields from the others have been presented.

Computer graphics produces image data from 3D models, computer vision often produces 3D models from image data.[20] There is also a trend towards a combination of the two disciplines, e.g., as explored in augmented reality.

The following characterizations appear relevant but should not be taken as universally accepted::

  • Image processing and image analysis tend to focus on 2D images, how to transform one image to another, e.g., by pixel-wise operations such as contrast enhancement, local operations such as edge extraction or noise removal, or geometrical transformations such as rotating the image. This characterization implies that image processing/analysis neither require assumptions nor produce interpretations about the image content.
  • Computer vision includes 3D analysis from 2D images. This analyzes the 3D scene projected onto one or several images, e.g., how to reconstruct structure or other information about the 3D scene from one or several images. Computer vision often relies on more or less complex assumptions about the scene depicted in an image.
  • Machine vision is the process of applying a range of technologies & methods to provide imaging-based automatic inspection, process control and robot guidance[21] in industrial applications.[19] Machine vision tends to focus on applications, mainly in manufacturing, e.g., vision-based robots and systems for vision-based inspection, measurement, or picking (such as bin picking[22]). This implies that image sensor technologies and control theory often are integrated with the processing of image data to control a robot and that real-time processing is emphasised by means of efficient implementations in hardware and software. It also implies that the external conditions such as lighting can be and are often more controlled in machine vision than they are in general computer vision, which can enable the use of different algorithms.
  • There is also a field called imaging which primarily focuses on the process of producing images, but sometimes also deals with processing and analysis of images. For example, medical imaging includes substantial work on the analysis of image data in medical applications.
  • Finally, pattern recognition is a field which uses various methods to extract information from signals in general, mainly based on statistical approaches and artificial neural networks. A significant part of this field is devoted to applying these methods to image data.

Photogrammetry also overlaps with computer vision, e.g., stereophotogrammetry vs. computer stereo vision.

Applications

Applications range from tasks such as industrial machine vision systems which, say, inspect bottles speeding by on a production line, to research into artificial intelligence and computers or robots that can comprehend the world around them. The computer vision and machine vision fields have significant overlap. Computer vision covers the core technology of automated image analysis which is used in many fields. Machine vision usually refers to a process of combining automated image analysis with other methods and technologies to provide automated inspection and robot guidance in industrial applications. In many computer-vision applications, the computers are pre-programmed to solve a particular task, but methods based on learning are now becoming increasingly common. Examples of applications of computer vision include systems for:

Learning 3D shapes has been a challenging task in computer vision. Recent advances in deep learning has enabled researchers to build models that are able to generate and reconstruct 3D shapes from single or multi-view depth maps or silhouettes seamlessly and efficiently [20]
Learning 3D shapes has been a challenging task in computer vision. Recent advances in deep learning has enabled researchers to build models that are able to generate and reconstruct 3D shapes from single or multi-view depth maps or silhouettes seamlessly and efficiently [20]
DARPA's Visual Media Reasoning concept video

One of the most prominent application fields is medical computer vision, or medical image processing, characterized by the extraction of information from image data to diagnose a patient. An example of this is detection of tumours, arteriosclerosis or other malign changes; measurements of organ dimensions, blood flow, etc. are another example. It also supports medical research by providing new information: e.g., about the structure of the brain, or about the quality of medical treatments. Applications of computer vision in the medical area also includes enhancement of images interpreted by humans—ultrasonic images or X-ray images for example—to reduce the influence of noise.

A second application area in computer vision is in industry, sometimes called machine vision, where information is extracted for the purpose of supporting a manufacturing process. One example is quality control where details or final products are being automatically inspected in order to find defects. Another example is measurement of position and orientation of details to be picked up by a robot arm. Machine vision is also heavily used in agricultural process to remove undesirable food stuff from bulk material, a process called optical sorting.[24]

Military applications are probably one of the largest areas for computer vision. The obvious examples are detection of enemy soldiers or vehicles and missile guidance. More advanced systems for missile guidance send the missile to an area rather than a specific target, and target selection is made when the missile reaches the area based on locally acquired image data. Modern military concepts, such as "battlefield awareness", imply that various sensors, including image sensors, provide a rich set of information about a combat scene which can be used to support strategic decisions. In this case, automatic processing of the data is used to reduce complexity and to fuse information from multiple sensors to increase reliability.

Artist's concept of a Mars Exploration Rover, an example of an unmanned land-based vehicle. Notice the stereo cameras mounted on top of the rover.
Artist's concept of a Mars Exploration Rover, an example of an unmanned land-based vehicle. Notice the stereo cameras mounted on top of the rover.

One of the newer application areas is autonomous vehicles, which include submersibles, land-based vehicles (small robots with wheels, cars or trucks), aerial vehicles, and unmanned aerial vehicles (UAV). The level of autonomy ranges from fully autonomous (unmanned) vehicles to vehicles where computer-vision-based systems support a driver or a pilot in various situations. Fully autonomous vehicles typically use computer vision for navigation, e.g. for knowing where it is, or for producing a map of its environment (SLAM) and for detecting obstacles. It can also be used for detecting certain task specific events, e.g., a UAV looking for forest fires. Examples of supporting systems are obstacle warning systems in cars, and systems for autonomous landing of aircraft. Several car manufacturers have demonstrated systems for autonomous driving of cars, but this technology has still not reached a level where it can be put on the market. There are ample examples of military autonomous vehicles ranging from advanced missiles to UAVs for recon missions or missile guidance. Space exploration is already being made with autonomous vehicles using computer vision, e.g., NASA's Mars Exploration Rover and ESA's ExoMars Rover.

Other application areas include:

Typical tasks

Each of the application areas described above employ a range of computer vision tasks; more or less well-defined measurement problems or processing problems, which can be solved using a variety of methods. Some examples of typical computer vision tasks are presented below.

Computer vision tasks include methods for acquiring, processing, analyzing and understanding digital images, and extraction of high-dimensional data from the real world in order to produce numerical or symbolic information, e.g., in the forms of decisions.[4][5][6][7] Understanding in this context means the transformation of visual images (the input of the retina) into descriptions of the world that can interface with other thought processes and elicit appropriate action. This image understanding can be seen as the disentangling of symbolic information from image data using models constructed with the aid of geometry, physics, statistics, and learning theory.[8]

Recognition

The classical problem in computer vision, image processing, and machine vision is that of determining whether or not the image data contains some specific object, feature, or activity. Different varieties of the recognition problem are described in the literature:[citation needed]

  • Object recognition (also called object classification) – one or several pre-specified or learned objects or object classes can be recognized, usually together with their 2D positions in the image or 3D poses in the scene. Blippar, Google Goggles and LikeThat provide stand-alone programs that illustrate this functionality.
  • Identification – an individual instance of an object is recognized. Examples include identification of a specific person's face or fingerprint, identification of handwritten digits, or identification of a specific vehicle.
  • Detection – the image data are scanned for a specific condition. Examples include detection of possible abnormal cells or tissues in medical images or detection of a vehicle in an automatic road toll system. Detection based on relatively simple and fast computations is sometimes used for finding smaller regions of interesting image data which can be further analyzed by more computationally demanding techniques to produce a correct interpretation.

Currently, the best algorithms for such tasks are based on convolutional neural networks. An illustration of their capabilities is given by the ImageNet Large Scale Visual Recognition Challenge; this is a benchmark in object classification and detection, with millions of images and hundreds of object classes. Performance of convolutional neural networks, on the ImageNet tests, is now close to that of humans.[26] The best algorithms still struggle with objects that are small or thin, such as a small ant on a stem of a flower or a person holding a quill in their hand. They also have trouble with images that have been distorted with filters (an increasingly common phenomenon with modern digital cameras). By contrast, those kinds of images rarely trouble humans. Humans, however, tend to have trouble with other issues. For example, they are not good at classifying objects into fine-grained classes, such as the particular breed of dog or species of bird, whereas convolutional neural networks handle this with ease.

Several specialized tasks based on recognition exist, such as:

  • Content-based image retrieval – finding all images in a larger set of images which have a specific content. The content can be specified in different ways, for example in terms of similarity relative a target image (give me all images similar to image X), or in terms of high-level search criteria given as text input (give me all images which contain many houses, are taken during winter, and have no cars in them).
Computer vision for people counter purposes in public places, malls, shopping centres
Computer vision for people counter purposes in public places, malls, shopping centres

Motion analysis

Several tasks relate to motion estimation where an image sequence is processed to produce an estimate of the velocity either at each points in the image or in the 3D scene, or even of the camera that produces the images . Examples of such tasks are:

  • Egomotion – determining the 3D rigid motion (rotation and translation) of the camera from an image sequence produced by the camera.
  • Tracking – following the movements of a (usually) smaller set of interest points or objects (e.g., vehicles, humans or other organisms[25]) in the image sequence.
  • Optical flow – to determine, for each point in the image, how that point is moving relative to the image plane, i.e., its apparent motion. This motion is a result both of how the corresponding 3D point is moving in the scene and how the camera is moving relative to the scene.

Scene reconstruction

Given one or (typically) more images of a scene, or a video, scene reconstruction aims at computing a 3D model of the scene. In the simplest case the model can be a set of 3D points. More sophisticated methods produce a complete 3D surface model. The advent of 3D imaging not requiring motion or scanning, and related processing algorithms is enabling rapid advances in this field. Grid-based 3D sensing can be used to acquire 3D images from multiple angles. Algorithms are now available to stitch multiple 3D images together into point clouds and 3D models.[20]

Image restoration

The aim of image restoration is the removal of noise (sensor noise, motion blur, etc.) from images. The simplest possible approach for noise removal is various types of filters such as low-pass filters or median filters. More sophisticated methods assume a model of how the local image structures look, to distinguish them from noise. By first analysing the image data in terms of the local image structures, such as lines or edges, and then controlling the filtering based on local information from the analysis step, a better level of noise removal is usually obtained compared to the simpler approaches.

An example in this field is inpainting.

System methods

The organization of a computer vision system is highly application-dependent. Some systems are stand-alone applications that solve a specific measurement or detection problem, while others constitute a sub-system of a larger design which, for example, also contains sub-systems for control of mechanical actuators, planning, information databases, man-machine interfaces, etc. The specific implementation of a computer vision system also depends on whether its functionality is pre-specified or if some part of it can be learned or modified during operation. Many functions are unique to the application. There are, however, typical functions that are found in many computer vision systems.

  • Image acquisition – A digital image is produced by one or several image sensors, which, besides various types of light-sensitive cameras, include range sensors, tomography devices, radar, ultra-sonic cameras, etc. Depending on the type of sensor, the resulting image data is an ordinary 2D image, a 3D volume, or an image sequence. The pixel values typically correspond to light intensity in one or several spectral bands (gray images or colour images), but can also be related to various physical measures, such as depth, absorption or reflectance of sonic or electromagnetic waves, or nuclear magnetic resonance.[24]
  • Pre-processing – Before a computer vision method can be applied to image data in order to extract some specific piece of information, it is usually necessary to process the data in order to assure that it satisfies certain assumptions implied by the method. Examples are:
    • Re-sampling to assure that the image coordinate system is correct.
    • Noise reduction to assure that sensor noise does not introduce false information.
    • Contrast enhancement to assure that relevant information can be detected.
    • Scale space representation to enhance image structures at locally appropriate scales.
  • Feature extraction – Image features at various levels of complexity are extracted from the image data.[24] Typical examples of such features are:
More complex features may be related to texture, shape or motion.
  • Detection/segmentation – At some point in the processing a decision is made about which image points or regions of the image are relevant for further processing.[24] Examples are:
    • Selection of a specific set of interest points.
    • Segmentation of one or multiple image regions that contain a specific object of interest.
    • Segmentation of image into nested scene architecture comprising foreground, object groups, single objects or salient object[27] parts (also referred to as spatial-taxon scene hierarchy),[28] while the visual salience is often implemented as spatial and temporal attention.
    • Segmentation or co-segmentation of one or multiple videos into a series of per-frame foreground masks, while maintaining its temporal semantic continuity.[29][30]
  • High-level processing – At this step the input is typically a small set of data, for example a set of points or an image region which is assumed to contain a specific object.[24] The remaining processing deals with, for example:
    • Verification that the data satisfy model-based and application-specific assumptions.
    • Estimation of application-specific parameters, such as object pose or object size.
    • Image recognition – classifying a detected object into different categories.
    • Image registration – comparing and combining two different views of the same object.
  • Decision making Making the final decision required for the application,[24] for example:
    • Pass/fail on automatic inspection applications.
    • Match/no-match in recognition applications.
    • Flag for further human review in medical, military, security and recognition applications.

Image-understanding systems

Image-understanding systems (IUS) include three levels of abstraction as follows: low level includes image primitives such as edges, texture elements, or regions; intermediate level includes boundaries, surfaces and volumes; and high level includes objects, scenes, or events. Many of these requirements are entirely topics for further research.

The representational requirements in the designing of IUS for these levels are: representation of prototypical concepts, concept organization, spatial knowledge, temporal knowledge, scaling, and description by comparison and differentiation.

While inference refers to the process of deriving new, not explicitly represented facts from currently known facts, control refers to the process that selects which of the many inference, search, and matching techniques should be applied at a particular stage of processing. Inference and control requirements for IUS are: search and hypothesis activation, matching and hypothesis testing, generation and use of expectations, change and focus of attention, certainty and strength of belief, inference and goal satisfaction.[31]

Hardware

There are many kinds of computer vision systems; however, all of them contain these basic elements: a power source, at least one image acquisition device (camera, ccd, etc.), a processor, and control and communication cables or some kind of wireless interconnection mechanism. In addition, a practical vision system contains software, as well as a display in order to monitor the system. Vision systems for inner spaces, as most industrial ones, contain an illumination system and may be placed in a controlled environment. Furthermore, a completed system includes many accessories such as camera supports, cables and connectors.

Most computer vision systems use visible-light cameras passively viewing a scene at frame rates of at most 60 frames per second (usually far slower).

A few computer vision systems use image-acquisition hardware with active illumination or something other than visible light or both, such as structured-light 3D scanners, thermographic cameras, hyperspectral imagers, radar imaging, lidar scanners, magnetic resonance images, side-scan sonar, synthetic aperture sonar, etc. Such hardware captures "images" that are then processed often using the same computer vision algorithms used to process visible-light images.

While traditional broadcast and consumer video systems operate at a rate of 30 frames per second, advances in digital signal processing and consumer graphics hardware has made high-speed image acquisition, processing, and display possible for real-time systems on the order of hundreds to thousands of frames per second. For applications in robotics, fast, real-time video systems are critically important and often can simplify the processing needed for certain algorithms. When combined with a high-speed projector, fast image acquisition allows 3D measurement and feature tracking to be realised.[32]

Egocentric vision systems are composed of a wearable camera that automatically take pictures from a first-person perspective.

As of 2016, vision processing units are emerging as a new class of processor, to complement CPUs and graphics processing units (GPUs) in this role.[33]

See also

Lists

References

  1. ^ a b Dana H. Ballard; Christopher M. Brown (1982). Computer Vision. Prentice Hall. ISBN 978-0-13-165316-0.
  2. ^ a b Huang, T. (1996-11-19). Vandoni, Carlo, E (ed.). Computer Vision : Evolution And Promise (PDF). 19th CERN School of Computing. Geneva: CERN. pp. 21–25. doi:10.5170/CERN-1996-008.21. ISBN 978-9290830955.
  3. ^ a b Milan Sonka; Vaclav Hlavac; Roger Boyle (2008). Image Processing, Analysis, and Machine Vision. Thomson. ISBN 978-0-495-08252-1.
  4. ^ a b Reinhard Klette (2014). Concise Computer Vision. Springer. ISBN 978-1-4471-6320-6.
  5. ^ a b Linda G. Shapiro; George C. Stockman (2001). Computer Vision. Prentice Hall. ISBN 978-0-13-030796-5.
  6. ^ a b c Tim Morris (2004). Computer Vision and Image Processing. Palgrave Macmillan. ISBN 978-0-333-99451-1.
  7. ^ a b Bernd Jähne; Horst Haußecker (2000). Computer Vision and Applications, A Guide for Students and Practitioners. Academic Press. ISBN 978-0-13-085198-7.
  8. ^ a b David A. Forsyth; Jean Ponce (2003). Computer Vision, A Modern Approach. Prentice Hall. ISBN 978-0-13-085198-7.
  9. ^ http://www.bmva.org/visionoverview The British Machine Vision Association and Society for Pattern Recognition Retrieved February 20, 2017
  10. ^ Murphy, Mike. "Star Trek's "tricorder" medical scanner just got closer to becoming a reality".
  11. ^ a b c d Richard Szeliski (30 September 2010). Computer Vision: Algorithms and Applications. Springer Science & Business Media. pp. 10–16. ISBN 978-1-84882-935-0.
  12. ^ Papert, Seymour (1966-07-01). "The Summer Vision Project". MIT AI Memos (1959 - 2004). hdl:1721.1/6125.
  13. ^ Margaret Ann Boden (2006). Mind as Machine: A History of Cognitive Science. Clarendon Press. p. 781. ISBN 978-0-19-954316-8.
  14. ^ Takeo Kanade (6 December 2012). Three-Dimensional Machine Vision. Springer Science & Business Media. ISBN 978-1-4613-1981-8.
  15. ^ Nicu Sebe; Ira Cohen; Ashutosh Garg; Thomas S. Huang (3 June 2005). Machine Learning in Computer Vision. Springer Science & Business Media. ISBN 978-1-4020-3274-5.
  16. ^ William Freeman; Pietro Perona; Bernhard Scholkopf (2008). "Guest Editorial: Machine Learning for Computer Vision". International Journal of Computer Vision. 77 (1): 1. doi:10.1007/s11263-008-0127-7. ISSN 1573-1405.
  17. ^ "Information Engineering | Department of Engineering". www.eng.cam.ac.uk. Retrieved 2018-10-03.
  18. ^ "Information Engineering Main/Home Page". www.robots.ox.ac.uk. Retrieved 2018-10-03.
  19. ^ a b Steger, Carsten; Markus Ulrich; Christian Wiedemann (2018). Machine Vision Algorithms and Applications (2nd ed.). Weinheim: Wiley-VCH. p. 1. ISBN 978-3-527-41365-2. Retrieved 2018-01-30.
  20. ^ a b c "Soltani, A. A., Huang, H., Wu, J., Kulkarni, T. D., & Tenenbaum, J. B. Synthesizing 3D Shapes via Modeling Multi-View Depth Maps and Silhouettes With Deep Generative Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 1511-1519)". 2019-01-25.
  21. ^ Turek, Fred (June 2011). "Machine Vision Fundamentals, How to Make Robots See". NASA Tech Briefs Magazine. 35 (6). pages 60–62
  22. ^ https://www.robots.com/blog/viewing/the-future-of-automated-random-bin-picking
  23. ^ Wäldchen, Jana; Mäder, Patrick (2017-01-07). "Plant Species Identification Using Computer Vision Techniques: A Systematic Literature Review". Archives of Computational Methods in Engineering. 25 (2): 507–543. doi:10.1007/s11831-016-9206-z. ISSN 1134-3060. PMC 6003396. PMID 29962832.
  24. ^ a b c d e f E. Roy Davies (2005). Machine Vision: Theory, Algorithms, Practicalities. Morgan Kaufmann. ISBN 978-0-12-206093-9.
  25. ^ a b Bruijning, Marjolein; Visser, Marco D.; Hallmann, Caspar A.; Jongejans, Eelke; Golding, Nick (2018). "trackdem: Automated particle tracking to obtain population counts and size distributions from videos in r". Methods in Ecology and Evolution. 9 (4): 965–973. doi:10.1111/2041-210X.12975. ISSN 2041-210X.
  26. ^ O. Russakovsky et al., "ImageNet Large Scale Visual Recognition Challenge", 2014.
  27. ^ A. Maity (2015). "Improvised Salient Object Detection and Manipulation". arXiv:1511.02999 [cs.CV].
  28. ^ Barghout, Lauren. "Visual Taxometric Approach to Image Segmentation Using Fuzzy-Spatial Taxon Cut Yields Contextually Relevant Regions." Information Processing and Management of Uncertainty in Knowledge-Based Systems. Springer International Publishing, 2014.
  29. ^ Liu, Ziyi; Wang, Le; Hua, Gang; Zhang, Qilin; Niu, Zhenxing; Wu, Ying; Zheng, Nanning (2018). "Joint Video Object Discovery and Segmentation by Coupled Dynamic Markov Networks" (PDF). IEEE Transactions on Image Processing. 27 (12): 5840–5853. doi:10.1109/tip.2018.2859622. ISSN 1057-7149. PMID 30059300.
  30. ^ Wang, Le; Duan, Xuhuan; Zhang, Qilin; Niu, Zhenxing; Hua, Gang; Zheng, Nanning (2018-05-22). "Segment-Tube: Spatio-Temporal Action Localization in Untrimmed Videos with Per-Frame Segmentation" (PDF). Sensors. 18 (5): 1657. doi:10.3390/s18051657. ISSN 1424-8220. PMC 5982167. PMID 29789447.
  31. ^ Shapiro, Stuart C. (1992). Encyclopedia of Artificial Intelligence, Volume 1. New York: John WIley & Sons, Inc. pp. 643–646. ISBN 978-0-471-50306-4.
  32. ^ Kagami, Shingo (2010). High-speed vision systems and projectors for real-time perception of the world. IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops. 2010. pp. 100–107. doi:10.1109/CVPRW.2010.5543776. ISBN 978-1-4244-7029-7.
  33. ^ Seth Colaner (January 3, 2016). "A Third Type Of Processor For VR/AR: Movidius' Myriad 2 VPU". www.tomshardware.com.

Further reading

External links

This page was last edited on 4 February 2020, at 15:26
Basis of this page is in Wikipedia. Text is available under the CC BY-SA 3.0 Unported License. Non-text media are available under their specified licenses. Wikipedia® is a registered trademark of the Wikimedia Foundation, Inc. WIKI 2 is an independent company and has no affiliation with Wikimedia Foundation.