Generating an Eigen Matrix Genotype From Aggregated Imagery
Generating an Eigen Matrix Genotype From Aggregated Imagery
or, the “Eigen Variant”
By Kevin M. Cowan and Steve Harris
October 10, 2017
Abstract:
A familiar theme in facial recognition technology is “Facial recognition is hard, and remains far from an exact science at this point in its evolution.” Currently, the technology used to create useful, precise biometric representations and associate these representations to their human counterparts (a.k.a. Facial Recognition), is a hit-or-miss “best guess” science. A second weakness of this approach is that it exclusively considers the world from a static, two-dimensional perspective.
We propose a shift from the static, two-dimensional perspective into a dynamic, multidimensional paradigm, by hybridizing the disciplines of forensic anthropology, 3-D modeling, object-oriented remote sensing and digital imaging; and then combining this approach with the low-power footprint, high-scale searchability of modern “big data” search engine(s?) capabilities. The resulting process will harness the methodology of forensic anthropology to reverse engineer a model of an object — not only limited to faces — but taking into account a variety of distinctive scalar vectors associated with an individual. This would include, but would not be limited to: Cranial Topography, Cephalic Index, Functional Craniology, Machine Learning and Pattern Matching. This would be accomplished utilizing both static and stochastic variant vectors aggregated into a single matrix, or what we call an ‘Eigen Variant’. What sets this model apart from contemporary biometric and/or facial recognition gathering is that while most models only deal with a single, or limited, number of images, the Eigen Variant encourages image data aggregation. The more data to aggregate, the more distinctive the variant becomes. Thus to incorporate not only traditional biometric recognition strategies, but morphological, mathematical and contextual aspects as well.
The ‘Eigen Variant’
Definition
In the simplest terms, an Eigen Variant is an object containing a multidimensional set of Eigen Matrices, which in turn contain a set of Eigenvectors, the scalar values of which may be either simple or complex. The Eigenvectors contains (n) sets of scalar values that are stored for analytics and retrieval in the form of multidimensional arrays. These multidimensional arrays constitute a genotype extrapolated from a phenotype, derived from (n) sources of digital imaging which, in essence, comprise a mathematical fingerprint.
An Eigen Variant is ultimately derived, then, from the aggregation of (n) images that are a) mined for singular objects in the form of patterns and gradient variants, mathematical distinctiveness (spatial, geometric, spectral, etc); and then b) reduced to a set of scalar arrays and converted to Eigenvectors; which are then; c) matched and merged into a Eigen Matrix contained in the Eigen Variant that constitutes a genotype of the target object. This object may be anything having distinguishing characteristics that can be reduced to its scalar designations, which is to say, any physical object. Beyond the mathematical aspect, though, the Eigen Variant is an object-oriented class hierarchy capable of applying functional Machine Learning in the context of a procedural container in real-time.
Proposed Methodology
Digital Image Acquisition and Aggregation
Composite imagery would be acquired and processed from a variety of sources. The more sources, the more distinctive the genotype; thus the more sources, the better the result. The key to this process would be associating the given image with a known Eigen Variant. Rather than focusing specifically on Eigenface biometrics, the process would isolate distinct objects (E.g. jewelry, clothing, prosthetics, etc) as well as facial, cranial, and other distinctly identifiable objects (E.g. Eigenface vectors, ear shape vectors, etc). These objects would be analyzed by a Support Vector Machine for future reference, and the distinctive objects would be packaged as a set and passed along a set of microservices as follows.
Eigen Matrix Application
Each distinct object produced from the composite analysis is re-parsed with the proprietary purpose of identifying distinctive static and stochastic (random) elements in the form of crestlines, ridges and troughs that are then reduced into an Eigen Matrix. In “real world” terms, this would be something like the shape of an ear, or a particular piece of jewelry, and so on.
Eigen Variant Instantiation
The resulting Eigen Matrix is then sent along for analysis and comparison to existing matrices stored in the data engine. Here is where modern “big data” technology holds the critical advantage. Utilizing ‘big data’ indexing and querying techniques, it is possible to compare objects to billions of other objects in a matter of milliseconds, not only in standard mathematical terms, but also in terms of applied morphology. The matrix is thus analyzed from mathematical, morphological and contextual relevance and matched to an Eigen Matrix where it is then merged into the current variant, where in each evolution, its distinctiveness is enhanced resulting in a phenologically distinct object. If no suitable match is found, a new Eigen Matrix is instantiated, and categorized by similarity.
Ingestion and Analysis
Once a suitable Eigen Matrix has been found for the incoming data, a process will consider the most accurate way of incorporating this new information into the existing variant. This will utilize object-oriented remote sensing techniques coupled with a proprietary mathematical ‘image stitching’ process that will integrate the data into the existing paradigm. In “real world” terms think of this as merging two or more pictures of an ear, taken at different angles, distances, times-of-day, etc, into a single image — after removing all the extraneous information and normalizing the distinctive objects captured therein — and then further amalgamating this information into the larger whole. Because our Eigen Variant is multidimensional, it is possible to incorporate virtually any type of information (E.g. skeletal information from X-Ray/MRI, etc; spectral information from ultraviolet/infrared information, and so on). All such data can be normalized and reduced to an Eigen Matrix, which is then ingested into the larger Eigen Variant.
Persistence and Retrieval
Once the data has been analyzed and ingested into a relevant variant, it becomes persistent and retrievable via query. Using an inverted index database like Solr, a portable, clustered neural network could be established that could query and retrieve from a cross-referenced dataset of billions of documents in a matter of milliseconds, incorporating complex/compound query elements to retrieve highly-relevant results. This approach has the advantage of requiring less computational power than massive proprietary data centers.
Proof of Concept
Solr and Machine Vision: Solr Revolution 2017
https://lucenesolrrevolution2017.sched.com/event/BAwM/solr-and-machine-vision?iframe=yes&w=100%&sidebar=yes&bg=no#
II. Case Study: Machine Learning with “Labeled Faces in the Wild (LFW)”
Summary
The advantage of the Eigen Variant is that it is founded in well-understood concepts, including modern big data engine scalability and object-oriented software utilization. The combination of these technologies creates a powerful tool that requires minimal computational power. The resultant system is powerful, compact, portable, extensible and capable of artificial intelligence in the form of real time machine learning. The paradigm differs from contemporary models in that data aggregation is requisite, rather than anathema; and acquisition from a wide variety of sources increases the distinctiveness of the genotype, rather than detracting from it. This allows for highly relevant matching from a broad swath of angles, distances, and other conditions that are not possible with the conventional approach.
References:
Apache Solr
http://lucene.apache.org/solr/
The Genotype Vs. Phenotype
Three-dimensional Eigenmatrix Example
https://en.wikipedia.org/wiki/Eigenvalues_and_eigenvectors#Three-dimensional_matrix_example
Eigenface definition
https://en.wikipedia.org/wiki/Eigenface
An Introduction to Facial Recognition
http://www.cedar.buffalo.edu/~govind/CSE717/papers/IntroductionToBiometricRecognition.pdf
Facial Recognition Patent Abstract
https://www.google.com/patents/US5787186
Illumination invariant face recognition using convolutional neural networks
http://ieeexplore.ieee.org/abstract/document/7091490/
Pushing the Frontiers of Unconstrained Face Detection and Recognition: IARPA
https://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Klare_Pushing_the_Frontiers_2015_CVPR_paper.pdf
Face Recognition on Consumer Devices: Reflections on Replay Attacks
http://ieeexplore.ieee.org/abstract/document/7029643/
Cranial Topography
http://www.mccc.edu/~kerrs/documents/Morphsinus_fall14.pdf
NIR-VIS Heterogeneous Face Recognition via Cross-Spectral Joint Dictionary Learning and Reconstruction
https://www.cv-foundation.org/openaccess/content_cvpr_workshops_2015/W05/papers/Juefei-Xu_NIR-VIS_Heterogeneous_Face_2015_CVPR_paper.pdf
Shared representation learning for heterogenous face recognition
http://ieeexplore.ieee.org/abstract/document/7163093/
Customized Orthogonal Locality Preserving Projections With Soft-Margin Maximization for Face Recognition
http://ieeexplore.ieee.org/abstract/document/7078936/
Superimposition and reconstruction in forensic facial identification: a survey
https://ac.els-cdn.com/0379073895017704/1-s2.0-0379073895017704-main.pdf?_tid=ee862b54-ae96-11e7-a1e1-00000aab0f01&acdnat=1507735072_9e1b5e4095e3a9a7ca07ff9ad55c8a48
Who is this person? A comparison study of current three-dimensional facial approximation methods
http://www.sciencedirect.com/science/article/pii/S0379073813001849
Three-dimensional Cranio-Facial Reconstruction in Forensic Identification: Latest Progress and New Tendencies in the 21st Century
https://www.astm.org/DIGITAL_LIBRARY/JOURNALS/FORENSIC/PAGES/JFS2004117.htm
Facial reconstruction: utilization of computerized tomography to measure facial tissue thickness in a mixed racial population
https://ac.els-cdn.com/0379073896020105/1-s2.0-0379073896020105-main.pdf?_tid=b4745f52-ae97-11e7-89b7-00000aacb361&acdnat=1507735405_4320d655d3608a4629d831b79eefadd7
Forensic Three-Dimensional Facial Reconstruction: Historical Review and Contemporary Developments
https://www.astm.org/DIGITAL_LIBRARY/JOURNALS/FORENSIC/PAGES/JFS14176J.htm
A Fully Three-Dimensional Method for Facial Reconstruction Based on Deformable Models
https://www.astm.org/DIGITAL_LIBRARY/JOURNALS/FORENSIC/PAGES/JFS14175J.htm
The geometrical precision of virtual bone models derived from clinical computed tomography data for forensic anthropology
https://link.springer.com/article/10.1007/s00414-017-1548-z
Cephalic index
https://en.wikipedia.org/wiki/Cephalic_index
Cranial morphology and facial type: Is it appropriate to describe the face using skull terminology?
http://www.sciencedirect.com/science/article/pii/S2212443814000034
Cranial shape and size variation in human evolution: structural and functional perspective
https://link.springer.com/article/10.1007/s00381-007-0434-2
“Functional craniology
The evolution of the human cranium is consequently considered in terms of functional and structural relationships between its components, largely influenced by the allometric variations associated with the increase in the relative cranial capacity. In the human genus, the changes in the face, base, and neurocranium are characterised by a mosaic variation, in which adaptations, secondary consequences, and stochastic factors concur to generate a set of anatomical possibilities and constraints.
”
Image Stitching
https://en.wikipedia.org/wiki/Image_stitching
Anaglyph 3D
https://en.wikipedia.org/wiki/Anaglyph_3D
Stereoscopy
https://en.wikipedia.org/wiki/Stereoscopy
Euclidean Vector
https://en.wikipedia.org/wiki/Euclidean_vector
Darboux Vector
https://en.wikipedia.org/wiki/Darboux_vector
Vector_(mathematics_and_physics)
https://en.wikipedia.org/wiki/Vector_(mathematics_and_physics)
Labled Faces in the Wild
http://vis-www.cs.umass.edu/lfw/
An object‐oriented approach for analysing and characterizing urban landscape at the parcel level
http://www.tandfonline.com/doi/abs/10.1080/01431160701469065
Machine Learning
https://en.wikipedia.org/wiki/Machine_learning
Vector Space
https://en.wikipedia.org/wiki/Vector_space
Mathematical Morphology
https://en.wikipedia.org/wiki/Mathematical_morphology
The Fibonacci Sequence
https://www.livescience.com/37470-fibonacci-sequence.html
The Mandelbrot Set
http://mathworld.wolfram.com/MandelbrotSet.html
The Golden Ratio
https://en.wikipedia.org/wiki/Golden_ratio
Mathematical Interpretation between Genotype and Phenotype Spaces and Induced Geometric Crossovers
https://arxiv.org/abs/0907.3202