## Generating an Eigen Matrix Genotype From Aggregated Imagery

Generating an Eigen Matrix Genotype From Aggregated Imagery

or, the “Eigen Variant”

By Kevin M. Cowan and Steve Harris

October 10, 2017

Abstract:

A familiar theme in facial recognition technology is “Facial recognition is hard, and remains far from an exact science at this point in its evolution.” Currently, the technology used to create useful, precise biometric representations and associate these representations to their human counterparts (a.k.a. Facial Recognition), is a hit-or-miss “best guess” science. A second weakness of this approach is that it exclusively considers the world from a static, two-dimensional perspective.

We propose a shift from the static, two-dimensional perspective into a dynamic, multidimensional paradigm, by hybridizing the disciplines of forensic anthropology, 3-D modeling, object-oriented remote sensing and digital imaging; and then combining this approach with the low-power footprint, high-scale searchability of modern “big data” search engine(s?) capabilities. The resulting process will harness the methodology of forensic anthropology to reverse engineer a model of an object — not only limited to faces — but taking into account a variety of distinctive scalar vectors associated with an individual. This would include, but would not be limited to: Cranial Topography, Cephalic Index, Functional Craniology, Machine Learning and Pattern Matching. This would be accomplished utilizing both static and stochastic variant vectors aggregated into a single matrix, or what we call an ‘Eigen Variant’. What sets this model apart from contemporary biometric and/or facial recognition gathering is that while most models only deal with a single, or limited, number of images, the Eigen Variant encourages image data aggregation. The more data to aggregate, the more distinctive the variant becomes. Thus to incorporate not only traditional biometric recognition strategies, but morphological, mathematical and contextual aspects as well.

The ‘Eigen Variant’

Definition

In the simplest terms, an Eigen Variant is an object containing a multidimensional set of Eigen Matrices, which in turn contain a set of Eigenvectors, the scalar values of which may be either simple or complex. The Eigenvectors contains (n) sets of scalar values that are stored for analytics and retrieval in the form of multidimensional arrays. These multidimensional arrays constitute a genotype extrapolated from a phenotype, derived from (n) sources of digital imaging which, in essence, comprise a mathematical fingerprint.

An Eigen Variant is ultimately derived, then, from the aggregation of (n) images that are a) mined for singular objects in the form of patterns and gradient variants, mathematical distinctiveness (spatial, geometric, spectral, etc); and then b) reduced to a set of scalar arrays and converted to Eigenvectors; which are then; c) matched and merged into a Eigen Matrix contained in the Eigen Variant that constitutes a genotype of the target object. This object may be anything having distinguishing characteristics that can be reduced to its scalar designations, which is to say, any physical object. Beyond the mathematical aspect, though, the Eigen Variant is an object-oriented class hierarchy capable of applying functional Machine Learning in the context of a procedural container in real-time.

Proposed Methodology

Digital Image Acquisition and Aggregation

Composite imagery would be acquired and processed from a variety of sources. The more sources, the more distinctive the genotype; thus the more sources, the better the result. The key to this process would be associating the given image with a known Eigen Variant. Rather than focusing specifically on Eigenface biometrics, the process would isolate distinct objects (E.g. jewelry, clothing, prosthetics, etc) as well as facial, cranial, and other distinctly identifiable objects (E.g. Eigenface vectors, ear shape vectors, etc). These objects would be analyzed by a Support Vector Machine for future reference, and the distinctive objects would be packaged as a set and passed along a set of microservices as follows.

Eigen Matrix Application

Each distinct object produced from the composite analysis is re-parsed with the proprietary purpose of identifying distinctive static and stochastic (random) elements in the form of crestlines, ridges and troughs that are then reduced into an Eigen Matrix. In “real world” terms, this would be something like the shape of an ear, or a particular piece of jewelry, and so on.

Eigen Variant Instantiation

The resulting Eigen Matrix is then sent along for analysis and comparison to existing matrices stored in the data engine. Here is where modern “big data” technology holds the critical advantage. Utilizing ‘big data’ indexing and querying techniques, it is possible to compare objects to billions of other objects in a matter of milliseconds, not only in standard mathematical terms, but also in terms of applied morphology. The matrix is thus analyzed from mathematical, morphological and contextual relevance and matched to an Eigen Matrix where it is then merged into the current variant, where in each evolution, its distinctiveness is enhanced resulting in a phenologically distinct object. If no suitable match is found, a new Eigen Matrix is instantiated, and categorized by similarity.

Ingestion and Analysis

Once a suitable Eigen Matrix has been found for the incoming data, a process will consider the most accurate way of incorporating this new information into the existing variant. This will utilize object-oriented remote sensing techniques coupled with a proprietary mathematical ‘image stitching’ process that will integrate the data into the existing paradigm. In “real world” terms think of this as merging two or more pictures of an ear, taken at different angles, distances, times-of-day, etc, into a single image — after removing all the extraneous information and normalizing the distinctive objects captured therein — and then further amalgamating this information into the larger whole. Because our Eigen Variant is multidimensional, it is possible to incorporate virtually any type of information (E.g. skeletal information from X-Ray/MRI, etc; spectral information from ultraviolet/infrared information, and so on). All such data can be normalized and reduced to an Eigen Matrix, which is then ingested into the larger Eigen Variant.

Persistence and Retrieval

Once the data has been analyzed and ingested into a relevant variant, it becomes persistent and retrievable via query. Using an inverted index database like Solr, a portable, clustered neural network could be established that could query and retrieve from a cross-referenced dataset of billions of documents in a matter of milliseconds, incorporating complex/compound query elements to retrieve highly-relevant results. This approach has the advantage of requiring less computational power than massive proprietary data centers.

Proof of Concept

Solr and Machine Vision: Solr Revolution 2017

https://lucenesolrrevolution2017.sched.com/event/BAwM/solr-and-machine-vision?iframe=yes&w=100%&sidebar=yes&bg=no#

II. Case Study: Machine Learning with “Labeled Faces in the Wild (LFW)”

Summary

The advantage of the Eigen Variant is that it is founded in well-understood concepts, including modern big data engine scalability and object-oriented software utilization. The combination of these technologies creates a powerful tool that requires minimal computational power. The resultant system is powerful, compact, portable, extensible and capable of artificial intelligence in the form of real time machine learning. The paradigm differs from contemporary models in that data aggregation is requisite, rather than anathema; and acquisition from a wide variety of sources increases the distinctiveness of the genotype, rather than detracting from it. This allows for highly relevant matching from a broad swath of angles, distances, and other conditions that are not possible with the conventional approach.

References:

Apache Solr

http://lucene.apache.org/solr/

The Genotype Vs. Phenotype

Three-dimensional Eigenmatrix Example

https://en.wikipedia.org/wiki/Eigenvalues_and_eigenvectors#Three-dimensional_matrix_example

Eigenface definition

https://en.wikipedia.org/wiki/Eigenface

An Introduction to Facial Recognition

http://www.cedar.buffalo.edu/~govind/CSE717/papers/IntroductionToBiometricRecognition.pdf

Facial Recognition Patent Abstract

https://www.google.com/patents/US5787186

Illumination invariant face recognition using convolutional neural networks

http://ieeexplore.ieee.org/abstract/document/7091490/

Pushing the Frontiers of Unconstrained Face Detection and Recognition: IARPA

https://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Klare_Pushing_the_Frontiers_2015_CVPR_paper.pdf

Face Recognition on Consumer Devices: Reflections on Replay Attacks

http://ieeexplore.ieee.org/abstract/document/7029643/

Cranial Topography

http://www.mccc.edu/~kerrs/documents/Morphsinus_fall14.pdf

NIR-VIS Heterogeneous Face Recognition via Cross-Spectral Joint Dictionary Learning and Reconstruction

https://www.cv-foundation.org/openaccess/content_cvpr_workshops_2015/W05/papers/Juefei-Xu_NIR-VIS_Heterogeneous_Face_2015_CVPR_paper.pdf

Shared representation learning for heterogenous face recognition

http://ieeexplore.ieee.org/abstract/document/7163093/

Customized Orthogonal Locality Preserving Projections With Soft-Margin Maximization for Face Recognition

http://ieeexplore.ieee.org/abstract/document/7078936/

Superimposition and reconstruction in forensic facial identification: a survey

https://ac.els-cdn.com/0379073895017704/1-s2.0-0379073895017704-main.pdf?_tid=ee862b54-ae96-11e7-a1e1-00000aab0f01&acdnat=1507735072_9e1b5e4095e3a9a7ca07ff9ad55c8a48

Who is this person? A comparison study of current three-dimensional facial approximation methods

http://www.sciencedirect.com/science/article/pii/S0379073813001849

Three-dimensional Cranio-Facial Reconstruction in Forensic Identification: Latest Progress and New Tendencies in the 21st Century

https://www.astm.org/DIGITAL_LIBRARY/JOURNALS/FORENSIC/PAGES/JFS2004117.htm

Facial reconstruction: utilization of computerized tomography to measure facial tissue thickness in a mixed racial population

https://ac.els-cdn.com/0379073896020105/1-s2.0-0379073896020105-main.pdf?_tid=b4745f52-ae97-11e7-89b7-00000aacb361&acdnat=1507735405_4320d655d3608a4629d831b79eefadd7

Forensic Three-Dimensional Facial Reconstruction: Historical Review and Contemporary Developments

https://www.astm.org/DIGITAL_LIBRARY/JOURNALS/FORENSIC/PAGES/JFS14176J.htm

A Fully Three-Dimensional Method for Facial Reconstruction Based on Deformable Models

https://www.astm.org/DIGITAL_LIBRARY/JOURNALS/FORENSIC/PAGES/JFS14175J.htm

The geometrical precision of virtual bone models derived from clinical computed tomography data for forensic anthropology

https://link.springer.com/article/10.1007/s00414-017-1548-z

Cephalic index

https://en.wikipedia.org/wiki/Cephalic_index

Cranial morphology and facial type: Is it appropriate to describe the face using skull terminology?

http://www.sciencedirect.com/science/article/pii/S2212443814000034

Cranial shape and size variation in human evolution: structural and functional perspective

https://link.springer.com/article/10.1007/s00381-007-0434-2

“Functional craniology

The evolution of the human cranium is consequently considered in terms of functional and structural relationships between its components, largely influenced by the allometric variations associated with the increase in the relative cranial capacity. In the human genus, the changes in the face, base, and neurocranium are characterised by a mosaic variation, in which adaptations, secondary consequences, and stochastic factors concur to generate a set of anatomical possibilities and constraints.

”

Image Stitching

https://en.wikipedia.org/wiki/Image_stitching

Anaglyph 3D

https://en.wikipedia.org/wiki/Anaglyph_3D

Stereoscopy

https://en.wikipedia.org/wiki/Stereoscopy

Euclidean Vector

https://en.wikipedia.org/wiki/Euclidean_vector

Darboux Vector

https://en.wikipedia.org/wiki/Darboux_vector

Vector_(mathematics_and_physics)

https://en.wikipedia.org/wiki/Vector_(mathematics_and_physics)

Labled Faces in the Wild

http://vis-www.cs.umass.edu/lfw/

An object‐oriented approach for analysing and characterizing urban landscape at the parcel level

http://www.tandfonline.com/doi/abs/10.1080/01431160701469065

Machine Learning

https://en.wikipedia.org/wiki/Machine_learning

Vector Space

https://en.wikipedia.org/wiki/Vector_space

Mathematical Morphology

https://en.wikipedia.org/wiki/Mathematical_morphology

The Fibonacci Sequence

https://www.livescience.com/37470-fibonacci-sequence.html

The Mandelbrot Set

http://mathworld.wolfram.com/MandelbrotSet.html

The Golden Ratio

https://en.wikipedia.org/wiki/Golden_ratio

Mathematical Interpretation between Genotype and Phenotype Spaces and Induced Geometric Crossovers

https://arxiv.org/abs/0907.3202