About the Lecturer

Privatdozent im Fachbereich Mathematik und Informatik, Universität Bremen

2016: Habilitation mit der Venia Legendi für die Informatik

2018-2019: Vertretungsprofessur Praktische Informatik und Lehrbeauftragter in der Fakultät Informatik, Universität Koblenz-Landau

2020-heute: Wissenschaftlicher Leiter und Projektleiter in der transregionalen DFG Forschungsgruppe 3022, Datengetriebene Schadensdiagnostik mit ML

2022-heute: Wissenschaftlicher Leiter und Projektleiter in der U Bremen Research AllianceAI Center for Health Care (Add. Fertigung und Prozessoptimierung)

Teaching and research topics

My Research Topics

Sensor Data Sciences

Material-integrated sensor systems (the machine or the material as a sensor)

Sensor networks

Image processing and sensor fusion

Mobile data collection and mobile crowdsensing (the human being as a sensor)

Process optimization

Research topics

Distributed and Applied AI

Agents and multi-agent systems (computation with agents, ABX)

Machine Learning and Data Mining

Ensemble and Distributed Machine Learning

Structural Health Monitoring

Chat Bots

Simulation

Agent-based Simulation (ABM/ABS)

Coupling of real and virtual worlds (Augmented Virtuality, ABS+ABX)

Parallel and distributed simulation methods and architectures (ABS/ABX)

Introduction

Target Audience

Materials Science and Engineering

Mechanical Engineering

Vehicle construction

Industrial engineering

International Production Engineering and Management

Student currciulum module area: Bachelor and Master Specialization

In addition to lecture students, scientific workers and thesis students are invited to participate to the winter school.

Main topics in this course

Metrics, taxonomies, methods, algorithms of ...

Sensors in materials testing technology, digital sensor data, multidimensional and time-dependent sensor data

Sensor data acquisition and processing (testing and measuring methods in materials testing technology and materials science)

Features and feature analysis in image data

Machine learning and automatic feature diagnostics (robustness, explainability, noise)

Algorithms and models (especially for image data)

Applications, demonstrations, examples, Digital laboratory exercises (integrated) with real and artificial measurement data from materials testing technology (X-ray CT, tensile tests, micrographs from the Add. Manufacturing, US time signals 2D/3D, and many more from YOU!)

Fields of Application

Destructive and Non-destructive Testing in general

Structural Health Monitoring (not addressed in this course)

Structural Testing

Material Design and Testing

Processes

Your application?

X-ray Diffraction

Activity Recognition

Contrast Enhancement

Region of Interest

Crack Propagation

Microstructure Effects

Damage Detection

Geospatial Traces

Organisation

Introduction to basic methods and algorithms with simple "sandbox" tutorials and exercises (partially home/office work)

Discussion of scientific application cases and available data

Application of methods and algorithms to different complex application cases

Discussions and presentation of results

Data

What kind of data we have to process?

Input Data D ⇒ Measurements

Meta Data M ⇒ Context

Intermediate Data R ⇒ Preprocessed

Output Data F ⇒ Features

image-placeholder
(2-1)

Input Data

Dimension

1D: Time-dependent vibration or guided ultrasonic wave signals

2D: Images (visible light, infrared, X-ray)

3D: Time-dependent 2D images (videos) or 3D spatial data (tomography, LIDAR)

4D: Time-dependent 3D spatial data (tomography videos)

Variable Class

Metric/Numerical, continuous (e.g., temperature)

Metric/Numerical, discrete (e.g., number of items)

Categorical/Symbolic (e.g., damage class)

Categorical/Interval (e.g., range intervals)

Size

One data value

1000 data values

1M data values

..

Temporal Characteristics

Periodic

Sporadic

One-shot

Your data?

X-ray transmission 2D

Electron Back Scattering 2D/3D

X-ray 2D/3D

RGB Microscopy Images

IMU Accel. Data 3D/4D

Ultrasonic Signals 1D/2D/3D

Tensile Test Data

X-ray tomography

Meta Data

Any experiment is situated in a context:

Date, Place

Set-up

Operator

Environmental conditions (temperature)

Measuring technology, device, parameters

The device under test (DUT)

Dimensions

Material

..

Data formats and structure!

Data Sources and Storage

Storage

Files and Filesystem

Linear Data Model

Tree Structure

Web Storage (HTTP)

Tree Structure

File Cloud Services (Seafile)

Tree Structure

Databases (SQL, NoSQL, ..)

Table Structure

Coded Data Structure in Files

Filesystem in Files

Format and Coding

JavaScript Object Notation (JSON)

Comma Separated Values (CSV)

Yet Another Meta Language (YAML)

Numerical Python (numpy)

Hierarchical Data Format (HDF5)

Portable Network Graphics (PNG)

JPEG, BMP, TIFF, ..

Data Examples

Figure 2-1. Anomaly detection in 3D X-ray CT image volumes (composite materials with defects)

Materials 2022, 15(13), 4645; https://doi.org/10.3390/ma15134645

Figure 2-2. Breakage prediction with tensile test data (metal probes)

SciForum ECSA 2020, https://doi.org/10.3390/ecsa-7-08279

Figure 2-3. Damage detection with Guided Ultrasonic Waves (composite material with pseudo defects)

SysInt 2022, DOI: 10.1007/978-3-031-16281-7_35

Figure 2-4. Porosity analysis with metallurgical microphotographs of sliced surfaces (additive manufacturing)

Materials 2022, 15(20), 7090; https://doi.org/10.3390/ma15207090

Figure 2-5. Feature marking of micro cracks and crack propagation of metal probes after applying stress

AG Brandt, U Siegen

Features

A feature is any kind of aggregated or condensed information from data.

Categorical features, e.g., damage classes, machine states, etc.

Metric/numerical features, e.g., length of a crack, spatial position, time, temperature

Regions of interest (labelled polygons)

Anomaly

The output of a ML model is typically a categorical or metric feature variable.

There are input and output featueres.

Input signal features

Selected features from the raw signal data that pose a strong correlation with the output target features ⇒ Transformation of signal data to Intermediate variables

Examples: Statistical features of time data series (average, deviation, skewness, ..), frequency spectrum, ROI, wavelet transformation coefficients.

Output target features

The final relevant information that is obtained from the data, i.e., the answer to a question.

Your features?

Cracks

Damages

Movements

Orientation

Material Characterstics

Methods and Algorithms

A typical "Data Mining" workflow consists of different stages:

Data pre-processing (e.g., filtering)

Signal feature selection (what are the best/strongest intermediate variables) and their computation

Selecting a predictor and analysis model (e.g., an artificial neural network or a decision tree)

Fit the model to training data

Apply the model to test data

validation and improvement.

Different data processing methods and algorithms are used in the different stages.

Statistical Methods

Statistical analysis of signal data and computation of statistical aggregate variables. Mostly applied to data vectors (data series) or matrix data (images) ⇒ Horizontal Analysis

Algebraic and geometrical Methods

Analysis of data variables with respect to correlation strength and information content, e.g., Principle Component Analysis, with optional following data transformation ⇒ Vertical Analysis

Transformation Methods

Transformation of data spaces, e.g., time-to-frequency (fourier) or time-to-time/frequency (wavelet) spaces. Image processing applies often kernel-based transformations, e.g., contrast amplification, edge detection.

Regression Methods

A numerical parametrized model (e.g., polynomial function) is fitted to labelled training data ⇒ Supervised Training

Selection Methods

A tree structure is created from labelled training data ⇒ Supervised Training

Clustering Methods

A tree or functional graph structure is created from unlabelled training data ⇒ Unsupervised Training and grouping of experiments.

In general, data processing and machine learning is a functional composition.

ML searches mapping functions F(X): X → Y.

ML is commonly an approximation of a hypothesis model H ≈ F of an unknown real (world) model M

There are infinite approximations (hypothesis models), the so called model space

A model function that maps X on Y is commonly parametrized by P (parameter space):

image-placeholder
(2-2)

The structure of the model function itself can be parametrized by S, e.g., the degree of a polynomial function:

image-placeholder
(2-3)

Toolbox

Signal filtering and transformation (FFT, DWT, Hilbert and analytical signal,..)

Image Processing (Edge detection, clustering, filtering, binarization ,..)

Statistical analysis (basics, PCA, ..)

Machine Learning, supervised, unsupervised (Decision trees, regression, art. neural networks, LSTM, Self-organizing Maps, ..)

Agent-based approaches (pattern search, cellular automata)

Your methods?

Gaussian Mixture Clustering

Click & Collect

Geometric Methods

Software

The Web Browser is the Laboratory!

WorkBook (fully self-contained HTML file) for the browser

WorkShell (node.js worker, optional)

SQLite3 Database (node.js)

WEX (local access of filesystem)

Additional plug-ins (for WorkBook and WorkShell)

Your software?

Python

Matlab

Software Architecture

Figure 2-6. Work flow architecture

Toolbox

The WorkBook and WorkShell software provide a broad range of methods and algorithms, partially using well-known software packages:

Image processing with OpenCV.js (https://docs.opencv.org/3.4/d5/d10/tutorial_js_root.html)

Image processing with image.js (https://github.com/image-js/image-js)

Artificial Neural networks with neataptic (https://wagenaartje.github.io/neataptic/)

Convolutional Neural Networks with convnet.js (https://cs.stanford.edu/people/karpathy/convnetjs/)

... andy many more ...

Data and Data Features

Metrics and taxonomy of Data

Features of Data

Analysis of Data

Data

In general, data and their values can be divided into:

Scalar values, such as temperature, age, etc.

Series of scalar values, such as time series

Vector and matrix values such as images

Composite data, i.e. data structures (records)

Temporal-spatial data, i.e. time-dependent spatial data series, D={D(p,t)={d(p)i}} with i = {1,2,3,..,t}, p=⟨x,y,..⟩

Data have dimensionality 𝕏N

The values of 𝕏 are a dimension from the discrete number set ℕ, real number set ℝ, and the time scale 𝕋 or any categorical value sets 𝕊 (or subsets thereof), e.g., 𝕏=ℝ × ℝ × ℕ.

Data Reduction

The aim of data analysis is to reduce input data in terms of size and dimensionality:

image-placeholder
(3-1)

Materials Science, metrology, and construction engineering uses:

Commonly metric input variables;

Often metric or categorical output variables (incl. Boolean variables)

1: function isRaining(temp,sunrad,moisture) =
2:   temp < 0 ? → false
3:   temp > 40 ? → false
4:   (sunrad-moisture) > 30? → false
5:   true

Example 3-1. An example from measurement technology with a data reduction function ℝ^3^ → 𝔹

Data classes

Numerical and Metric values

These are values that are countable and where you can meaningfully define relations (such as smaller or larger), i.e. for all real and integers.

Examples: temperature, length, density, pore size, elongation, force, location, time

Categorical values

These are symbolic values for which either no (meaningful) order relation exists or where at least no differences can be formed.

Examples: nationality, color names (red < yellow???), Damage type, characteristic feature (anomaly?)

Scaling of numerical values

Interval scaling

For this type of attributes, only differences (addition or subtraction) make sense. For example, the temperature measured in °C or °F is interval scaled. If it is 20 °C on one day and 10 °C on the following day, it makes sense to talk about a temperature drop of 10 °C, but it does not make sense to say that it is twice as cold as the day before (C(K)∼K, but F(K)/∼K!!).

Ratio escalates

Here you can calculate both differences and ratios between values. For example, for age, one can say that someone who is 20 years old is twice as old as someone who is 10 years old.

Order relations

Nominal

The attribute values in the domain are unordered and therefore only equality comparisons make sense. That is, we can only check whether the value of the attribute is the same for two specific instances or not. For example, gender is a nominal attribute.

Ordinal

The attribute values are ordered and thus equality comparisons (is one value equal to another?) and relational comparisons (is one value smaller or larger than another?) are allowed, although it may not be possible to quantify the difference between the values!

Data classes (longitudinal)

Sensor and measurement data variables (both categorical and metric) can be further distinguished in:

Static

The variable s is not variable in time or is to be regarded as stationary (immutable) in a significant time interval t ∈ [t₀, t₁].

Dynamic

The variable s(t) is time-dependent and forms a data series (or time vector) s(t)={s₀,s₁,..st} in the case of discrete acquisition, i.e., we are talking about longitudinal data.

A digitized sensor signal is always discrete in time, but the physical variable that the sensor measures is continuous in time (note the sampling theorem)

Data

Data sets as matrices

Data can be represented in matrix form as matrix D (analogy to table form) [1]:

The vector X is the set of all variables Xi and represent the columns of the matrix D:

image-placeholder
(3-2)

Each row xj is a record of the variable set X={Xi|i=1,d} with values x and represent an individual example, instance, experiment, entitie, object, and feature vector as a d-digit tuple, depending on the application and objective:

image-placeholder
(3-3)

// JavaScript
type row = { x1:number|string, x2:number|string, .. ,
             xd:number|string }
type table = row array;

Input and Output Variables

The variable set is composed of input and output variables: Xxy=X ∩ Y

Sensors are commonly input variables X

Statements are output variables Y, i.e. results that can be derived from the input variables (by a function F):

image-placeholder
(3-4)

with u+v=d.

Example of a data matrix

Botanical data set with geometric (numerical) properties of a plant and categorical classification:

[1]

Measurement data set

Computed Strain-stress diagram

[www.precifast.de/elastizitaetsmodul-e-modul]

Measurement data from strain test

Strain [mm] | Force [kN]
0 | 0
0.1 | 0.2
0.2 | 0.7
0.3 | 1.5
0.4 | 1.7
0.5 | 1.9
0.6 | 2.0
0.7 | 0.2
0.8 | -0.5

Attributes

The measured variables X₁ to X₄ are metric data variables, the variable X₅=y is a categorical variable!

The measured variables X₁ to X₄ (i.e. sensors) are called attributes because they are properties and descriptive variables of the target variable y.

High-dimensional Data

Images I=I(x,y[,z]) are commonly two- or three-dimensional spatial data, organised in rows and columns (and levels)

Spatiotemporal data T=T(x,y[,z],t) is commonly three- or four dimensional and organised in rows, columns (levels), and discrete time points t.

Sensors

Which sensors and measurement data do you know:

Sensor

Measurement

Physical quantities such as temperature, strain, stress, time

Merged survey variables (e.g. ensemble mean values)

When measuring with sensors, a distinction is made between:

Single or single measurements (single shot)

Repeated measurements of the same physical quantity (averaging..)

Series of measured values, especially time-resolved data series:
D = {d₁,d₂,..,dn}, where commonly Δt(di,di+1) is constant

Socio-technical systems, surveys

Survey variables (answers to questions) are sensors of individual people

Merged survey variables (e.g. ensemble mean values) are sensors of groups of people

Generally available data

Social networks and social media

Databases of authorities, etc.

Sensor model

A sensor is a transducer (indicator for a property that is not directly measurable)

A sensor therefore generally maps a physical quantity x to another quantity y:

image-placeholder
(3-5)

There is usually a calibration function K(f,x,y)

Examples are:

Pressure → Voltage, Radiation → current, etc.

Social networking → Numerical radius value, votes → Politics, i.e., Assignment of numbers to objects or events according to established rules

Sensor data

Sensors S are data sources d of physical, sociological or other natural variables x that cannot be detected directly

The data values (numeric) will be in a definable interval

Knowledge of the value interval is important for later data processing, analysis, and machine learning!

Categorical values are also defined by a set

image-placeholder
(3-6)

Measurement and sensory systems

The origin of data for analysis and machine learning!

A sensor rarely comes alone.

Measurement methods

A distinction is made between two different measurement methods:

Passive measuring method

The sensory values are the result of an intrinsic property (e.g., density) or already existing external variables (temperature). The stimulus of the measurement is the component, the person, the environment.

Active measurement methods

There is an active stimulus whose response signal is detected by the sensor. An example is the ultrasonic measurement method with guided waves. The sensor signal is always dependent on the stimulus. In sociology, for example, the stimulus is a catalog of questions in a survey, the answers are the sensor variables.

Signal Features

Statistical Features

Spatial Features (Images)

Statistical Features

Assumption: Data series

But any image can be transformed into a pixel data series, too!

Any column of a data table is a data series (but unordered!)

There is a data series d related to one variable x (from sensor s):

image-placeholder
(5-1)

Feature | Formula
Sample Size | image-placeholder
Extrema | image-placeholder
Sample Mean | image-placeholder
Standard Deviation | image-placeholder
Sample Variance | image-placeholder

... and many more

Feature | Formula
N-th moment about point a, e.g., image-placeholder | image-placeholder
Gaussian Distribution | image-placeholder
Fisher Skewness | image-placeholder

... and many more

Statistical analysis is applied to the same static variable X with unordered values from repeated measurements of X under the same conditions

Statistical measures for data series (e.g., time-dependent) of dynamic variables with values from measurements under different conditions are not valid ("non-sense"). But statistical measures can be still used as signal features posing a correlation between the input signal and the target features (e.g., damages).

An ordered data series {di} can be considered as an ordered series of different variables {Xi}!

Finally, all statistical features create a new input vector (for ML) Xf derived from the original input variables X:

image-placeholder
(5-11)

Image Features

Low Level

Histogram H(I)={h₁,..,hk}, where each histogram variable represents the number of pixels within an intensity interval [i,i+Δ] (can be split into separate RGB histograms for colour images)

Average (mean) intensity I— , noise (intensity distribution statistics)

Extrema intensities min(I), max(I)

Frequency spectrum F(I)={f₁,..,fs}, where each frequency represents a wavenumber in the wave room

Intensity gradients and profiles along lines (axis)

Addition and subtraction of images (using, e.g., base-line reference images)

High Level

Intensity gradients

Edges

Geometrical figures

Object clusters

Regions-of-interest (ROI), defined by bounding boxes or closed polygons

Labelled and classified ROIs

Feature point markings

Threshold Binarization (dimensionality reduction and feature amplification)

Transformations

Reduce Picture Dimension

A simple way to reduce the dimension of our feature vector is to decrease the size of the image with decimation (downsampling) by reducing the resolution of the image.

If the color component is not relevant, we can also convert pictures to grayscale to divide the number dimension by three.

Intensity homogenisation using transfer functions

Histogram of Oriented Gradient

The HOG feature descriptor is a popular technique used in computer vision and image processing for detecting objects in digital images.

The HOG descriptor is a type of feature descriptor that encodes the shape and appearance of an object by computing the distribution of intensity gradients in an image.

Intensity Homogenization

The intensity of an image can vary significantly across the spatial x-y plane, e.g., as a result of the measuring method and conditions.

Image processing and transformation algorithms can be sensitive to intensity inhomogeneity.

Algorithms:

Histogram Equalization (HE), Brightness Preserving Bi-Histogram Equalization (BBHE)

Geometrical Image Intensity Equalization

Model-based (physical model of illumination)

Histogram Equalization

[https://docs.opencv.org/3.4/d4/d1b/tutorial_histogram_equalization.html]

It is a method that improves the contrast in an image, in order to stretch out the intensity range.

From the image below, you can see that the pixels seem clustered around the middle of the available range of intensities.

[https://github.com/YuAo/Accelerated-CLAHE]

Histogram equalization (HE) is a method in image processing of contrast adjustment using the image's histogram.

This method usually increases the global contrast of many images, especially when the usable data of the image is represented by close contrast values.

Through this adjustment, the intensities can be better distributed on the histogram.

This allows for areas of lower local contrast to gain a higher contrast.

Histogram Equalization stretch out this range.

Equalization implies mapping one distribution (the given histogram) to another distribution (a wider and more uniform distribution of intensity values) so the intensity values are spread over the whole range.

To accomplish the equalization effect, the remapping should be the cumulative distribution function (cdf). For the histogram H(i), its cumulative distribution Hcd(i) is:

image-placeholder
(5-12)

Finally, we use a simple remapping procedure to obtain the intensity values of the equalized image:

image-placeholder
(5-13)

Spatial Image Intensity Equalization

This simple Histogram Equalization is not sensitive to spatial intensity inhomogeneities and variations! Spatial uniform intensity distributions are assumed!

Intensity variations can be a result of a statistical process or due to the measuring technology and conditions

Variation can be considered as an overlay (addition) to the "real" measuring signal s(x,y)v(x,y)+n(x,y), and noise n

Methods based on a spatial filtering of the images use the assumption that the bias field (intensity inhomogeneity) consists of a low spatial frequency intensity variation ⇒ Applying a High-pass filter in the wavenumber space!?

Low-pass filtering methods can be used to extract non-uniformity

Trivial Approach

Assumption:

There is only one axis in the image with low-frequency intensity variations due to inhomogeneous illumination

The image content is statistically averaged homogeneous (i.e., equally distributed features)

The mean image intensity Imean(p) can be computed along a line l(p) (parametric equation, orientation by visual inspection along the strongest intensity variation/gradient) by using the average intensity along the perpendicular line at each point p:

image-placeholder
(5-14)

Figure 5-1. (Left) Computing the average intensity *I*~avg~(*p*) perpendicular to a line along the intensity gradient (Right) Correct all pixels perpendicular to the correction line with a equalization factor

Contrast Limited Adaptive Histogram Equalization

[https://github.com/YuAo/Accelerated-CLAHE]

CLAHE (Contrast Limited Adaptive Histogram Equalization) is an algorithm for enhancing local contrast in images, and is frequently used in application areas like underwater photography, traffic control, astronomy, and medical imaging.

CLAHE can also be used in the tone mapping operation of displaying a HDR (High Dynamic Range) image.

Adaptive histogram equalization (AHE) differs from ordinary histogram equalization in the respect that the adaptive method computes several histograms, each corresponding to a distinct section of the image, and uses them to redistribute the lightness values of the image.

It is therefore suitable for improving the local contrast and enhancing the definitions of edges in each region of an image.

AHE has a tendency to overamplify noise in relatively homogeneous regions of an image.

A variant of adaptive histogram equalization called contrast limited adaptive histogram equalization (CLAHE) prevents this by limiting the amplification.

The basic CLAHE algorithm

Compute the neighborhood histogram for each pixel in the image.

Clip each histogram at a predefined value and redistribute the clipped histogram equally among all the histogram bins.

Compute the CDF (Cumulative Distribution Function) and transformation function for each pixel using the clipped histogram.

Apply the transformation function to each pixel to get the equalized image.

Frequency Transformation

Time-dependent signal s(t) can be transformed in the frequency space S(ω) by using a frequency transformation, e.g., Discrete Fourier Transformation (DFT):

image-placeholder
(5-15)

The DFT transforms a series of complex numbers {xn} into a sequence of complex numbers {Xk}.

The transformation is reversible (as long as complex numbers, i.e., magnitude and phase, is preserved).

Low-, High-, and Bandpassfiltering can be performed by applying a mask function to the frequency distribution {Xk} and transforming back into time-space (blending in frequency space)

[TU Graz, IVU_frequency_2017]

2D DFT

Images can be transformed into the frequency space, too, called wavenumber space

A two-dimensional (2D) DFT is used (output is a matrix, too)

image-placeholder
(5-16)

[TU Graz, IVU_frequency_2017]

The signal frequency distribution is symmetric!

[TU Graz, IVU_frequency_2017]

Wavelet decomposition

Disadvantage of Fourier transformations is the lost of the time or spatial information.

A solution can be the application of a moving window of size m ≪ n, with n as the sample size (time signal: number of time samples, image: width and height).

But: The Fourier transformation delivers m/2 frequencies

If the window size is lowered, the time or spatial resolution increases, but the frequency resolution decreases!

Wavelet decomposition is a way of breaking down a signal in both space and frequency. In the case of pictures, this means breaking down the image into its horizontal, vertical, and diagonal components.

Figure 5-2. Decomposition of an image 2-D discrete wavelet transform with filter banks (2-D DWT) [Parida et al.,2017]

Figure 5-3. Example of a DWT signal decomposition of a US time-dependent signal [Bosse et al., doi:10.3390/computers10030034]

Image Gradient

The (intensity) gradient of an image is the vector ∇I(x,y). It is characterized by a magnitude m and a direction φ in the image:

Image Laplacian

Another important image transformation is the Laplacian of an image with intensity I(x,y) that is defined by:

Invariant to image rotations.

The laplacian is often used in image enhancement to increase contour effects

Higher sensitivity to noise than the gradient.

Edge Detection

Two main strategies:

Gradient strategy: detection of the local extrema in the gradient direction.

Laplacian strategy: detection of zero-crossing.

These strategies rely on the fact that edges correspond to 0-order discontinuities of the intensity function.

The derivative computation requires a pre-filtering of the images.

For instance: linear filtering for zero mean noises (e.g. white Gaussian noise and Gaussian filter) and non-linear filtering for impulse noise (median filter).

Kernel-based Folding Algorithms

Convolution is using a kernel matrix to extract certain features from images.

A kernel is a matrix, which is shifted across the image and multiplied with the input pixels covered by the kernel matrix such that the output is transformed in a certain desirable manner. Watch this in action below.

[https://towardsdatascience.com/types-of-convolution-kernels-simplified-f040cb307c37]

Measurement error and confidence

Systematic deviation (systematic error)

Deviation is caused by the sensor, environment, and sometimes physical processes

E\.g.: incorrect calibration, constantly existing faults such as friction

Can only be eliminated by carefully examining the source of the error

Random deviation (Random or statistical error)

Deviation is caused by unavoidable, irregular disturbances

with repeated measurement, individual results differ from each other

Individual results vary by an average value

Random error scattering

Random errors affect the accuracy of a measurement (noise).

Noise affects input and target feature computation (ML output)!

If one repeats a measurement of a quantity X which is falsified by pure random errors, the frequency distribution of the measured values is S = {s₁, s₂,...,sn} by a mean value image-placeholder given by a Gaussian distribution (the number of measurements N must be large).

Figure 5-4. Frequency distribution according to Gauss of measured values centered around an average value [9]

Examples: Statistical Analysis

/webwork

Plot

Plot(Array.random(200),{width:'70%',size:20})

Analysis

Math.statistics.analysis(Array.random(200))

Summary

Data can be classified into:

Categorical variables and values

Metric variables and values

Temporal static variables

Temporal dynamic variables (time series)

All sensor variables are subject to measurement errors:

Noise

Distortion

Displacement (bias)

Problem of reproducibility and systematic errors (environment!)

A (statistical) data analysis is often the first step in the ML workflow

Data Storage and Access

Representation of Data

Storage of Data

Access of data

Data Storage

In general, data and their values can be stored in/on:

File Systems, such as FAT32/VFAT, Ext 3, ....

Databases, such as SQL, NoSQL, ...

Cloud Storage, such as Seafile

Files, such as HDF(5)

But: The question is not where to store the data, the question is how to organize and access the data!

File System

A file system is composed of:

Files ⇒ data container (linear memory model)

Directories ⇒ reference tables assigning files or other directory references to names

A file system ognaizes data by a directory (folder) tree:

A directory is a node in an ordered graph

There is a root directory

Children of a directory can be leafes (files) or deeper directories

Each file and directory is associated with a name

File system tree iterations are referenced by paths

Besides the organization structure, a file system can provide data structures and block level organization of data used for the storage on hardware devices

Figure 6-1. Simple file system layout (linear file model) [https://www3.nd.edu/~pbui/teaching/cse.30341.fa18/project06.html]

Segments

Magic: The first field is always the MAGIC\_NUMBER or 0xf0f03410. The format routine places this number into the very first bytes of the super-block as a sort of file system "signature". When the file system is mounted, the OS looks for this magic number. If it is correct, then the disk is assumed to contain a valid file system. If some other number is present, then the mount fails, perhaps because the disk is not formatted or contains some other kind of data.

Blocks: The second field is the total number of blocks, which should be the same as the number of blocks on the disk.

InodeBlocks: The third field is the number of blocks set aside for storing inodes. The format routine is responsible for choosing this value, which should always be 10% of the Blocks, rounding up.

Inodes: The fourth field is the total number of inodes in those inode blocks.

Figure 6-2. I-nodes and block references [https://www3.nd.edu/~pbui/teaching/cse.30341.fa18/project06.html]

Data Types

Raw data featuring

Dimension

Size

Aggregates

Vector and time signal data

Matrix and tensor data

Image data (RGBA)

Data record tables

Databases

SQL

SQL databases organize data in tables.

A table is organized in rows and columns

A table is part of a database

Multiple databases can co-exist and handled by one database server

Data types:

Number

Text

Binary data (blob)

Figure 6-3. Basic SQL Server architecture

Page

The page is the fundamental unit of data storage and internally, SQL server organizes and stores data in smaller units known as pages.

Figure 6-4. Data Pages storing tables - Schematic Diagram [Murugesan et al., International Journal of Applied Engineering Research, 2015]

Tables

Figure 6-5. SQL table structure with rows and columns

Operations

Create

Create a new empty table with a specific schema (type signature of the columns. A table is referenced by a (data base) unique name.

Insert

Insert rows into an existing table.

Update

Update fields or entire existing rows

Select

Select fields or entire rows based on patterns

Tables cannot be nested. The data base table space is flat! But specific tables can be used to reference other tables (like I-nodes, directories in file systems, or sections in HDF structures)

Meta data, arrays, or other auxiliary structures must be encoded and decoded by the user, e.g., by using JSON formats!

Be aware of memory data layer hierarchy affecting performance (read/write): Data and DB Cache, Main Memory, File system, Storage Device(s).

Figure 6-6. Detailed and advanced SQL server architecture [https://www.guru99.com/sql-server-architecture.html]

MySQL

One of the top open-source relational database management systems (RDBMS)

Data security layers to protect sensitive data.

Scalability for when there are large amounts of data.

Open source RDBMS with two separate licensing models.

Multi-master ACID transactions through MySQL Cluster ((Atomicity, Consistency, Isolation, and Durability).

Supports both structured data (SQL) and semi-structured data (JSON).

Apache Cassandra

Is an open-source and highly scalable NoSQL database management system

Handles massive volumes of data.

One of the most scalable databases with automatic sharding.

Offers linear horizontal scaling.

Decentralized database with multi-datacenter replication and automatic replication.

Fault tolerant by automatically replicating data to multiple nodes.

PostgreSQL

It extends the SQL language and combines it with various features to scale and safely store highly complicated data workloads.

Highly secure with a robust access-control system.

Offers ACID transactional guarantee (Atomicity, Consistency, Isolation, and Durability)

PostgreSQL extension Citus Data offers Distributed SQL features.

Advanced indexes such as Partial Index and Bloom Filters.

Supports structured data (SQL), semi-structured data (JSON, XML), key-value, and spatial data.

MonogDB

It was designed to specially handle document data

Horizontal scaling via auto-sharding (method for distributing data across multiple machines).

Built-in replication through primary-secondary nodes.

Distributed multi-document ACID transactions with snapshot isolation.

Full-text search engine and data lake built on MongoDB

MLDB

Machine Learning Database, or MLDB, is an open-source system aimed at tackling big data machine learning tasks.

It can be used for data collection and storage through the training of machine learning models, or to deploy real-time prediction endpoints.

MLDB is one of the easier datasets to use, since it provides a comprehensive implementation of the SQL SELECT statement.

It treats datasets as tables, making it easier to learn and use for data analysts already versed in an existing Relational Database Management System (RDBMS).

Supports vertical scaling with higher efficiency.

... more information ...

https://www.unite.ai/10-best-databases-for-machine-learning-ai

Data Types JavaScript

Core data types are:

number

boolean

string

function

object

That's all folks!

But data can be organized in:

Arrays (poly-sorted, i.e., not compacted with heterogeneous element types possible)

Typed Arrays (mono-sorted. i.e., compacted/packed and each element with same data type):

Int8, Uint8

Int16, Uint16

Int32, Uint32

Float32, Float64

Records

Image Data

Formats:

RGBA ⇒ Uint8 [Red,Green,Blue,Alpha] [col][row]

RGB ⇒ Uint8 [Red,Green,Blue] [col][row]

GRAY8 ⇒ Uint8 [col][row]

Data layout is relevant! Commonly, [RGB], ordered by columns, organized in rows

Figure 6-7. Structure of the RGB image as a sequence of bytes in a linear memory or file model [DOI: 10.1109/CLEI.2015.7359995]

Data Tables

Array [[]]

Generic arary (rows) of arrays (columns). Any dimensionality can be composed by nesting arrays.

TypedArray TA

Linear ordered list of values. Higher dimensions must be organized by external wrapper functions or objects knowing the value order and encoding.

Structure Arrays [{}]

Each row of the array is a structure of named fields.

Matrix Matrix

Wrapper object for nested arrays with extended set of methods and operations

MatrixTA MatrixTA

Wrapper object for typed arrays with extended set of methods and operations (two- or three-dimensional)

Volumes Vol3

Wrapper object for typed arrays with extended set of methods and operations (always three-dimensional), used by CNNs

File Data Formats

CSV

Comma Separated Value format (text)

JSON

JavaScript Object Notation (text)

XML

Extensible Meta Language (text)

YAML

Yet Another Meta Language (text)

NumPy

Numerical Python Format (binary)

JSON

Array and record structure (nested, tree)

{
  "employee": {
    "name":       "sonoo",
    "salary":      56000,
    "married":    true,
    "awards" :    [1920,1990,2000]
  }
}

CSV

Flat table (not nested) with values separated by delimiter (e.g., "," or any other delimiter like the tabulator character)

Numbers and text are valid values

Each line in the file is one row in the data table

All rows should contain same number of values

x,y,z,class
1,2,3,"A"
1,4,2,"B"
4,5,0,"A"

Data Flow Architecture

The entire data processing architecture is a graph of computational nodes, data sources, and data sinks, connected by event-based channels. Nodes are connected via input and output ports.

Data Source

A data source node provides access to data, e.g., from an SQL database or from the file system. There are read and write events. Data tokens are created on request (by a read event trigger, e.g.).

Data Sink

A data sink nodes consumes data tokens and displays them. Typical display types are textual information, tables, and plots.

Computational Functions

A computational node processes data read from input ports and writes the processed data on output ports.

Examples are:

Statistics (from data tables)

Matrix operations (in-place or by creating new Matrix data)

Image operations and transformations, filters

Time-Frequency transformations, convolution, wavelet transformations

Merging of data, splitting of data, wrapping and unwrapping of data

Data format conversion

Figure 6-8. Processing graph and data flow architecture with data source, processing, and sink nodes. Event-based data flow architecture and event chains. New data provided by a node is propagated to all child nodes. Parameter changes initiate a re-computation (or display), too.

Programming

var source1 = new Data(type,{ .. });
var procfu1 = new Transformation(type,{.. });
var procfu2 = new Filter(type,{.. });
var procfu3 = new Functional(type,{.. });
var sink1   = new Display(type,{.. });
source1.output(procfun1)
source1.output(procfun2)
procfun1.output(procfun2,0)
procfun2.output(procfun2,1)
procfun3.output(sink1)

Definition 6-1. Principle composition of data flow systems using block objects and connections

SQL-RPC API

var data1 = new Data('sql',{
  url:'edu-9.de:9999',
  database : 'imagesnet12',
  table : 'imageset1'
})
var result = await data1.start();
var image  = await data1.read('*','rowid=1'); // SQL select
var schema = await data1.info();

SQLD

sqld consists of a slim native C-code implementation of the sqlite3 server storing SQL data bases in plain binary files on the local file system.

SQL data bases can be accessed by a Remote Procedure Call JSON interface, basically mapping SQL operations on a JSON structure (both request and reply).

HTTP is used to access the JSON-RPC API

Figure 6-9. SQLD architecture and JSON-RPC interface

Summary

Data can be represented by different data types, structures, formats

Data access is provided by SQL data bases in an unified way using a remote procedure call programming interface

Data processing is performed by using an event-based data-flow architecture

Cellular Automata and Image Processing

How can we apply algorithms to image data?

Relation between Images and Cellular Automata - local versa global state processing

From local to global state using Point Clustering and Polygon Hulls

Kernel-based Transformations and Convolutional Neural Networks!

Cellular Automaton

A cellular automaton (CA) is basically an active (functional) matrix

Applications:

Analysis and exploration of complex space-time dynamics and of the behavior of dynamical systems

Image transformations (state-less single-step and state-based/iterative)

Feature Analysis (Extraction, Amplification)

Parallel computation model!

Basic assumption:

Complex phenomena are the result of the collective dynamics of a very large number of parts obeying simple rules

We want to define the simplest nontrivial model of a cellular system. We base our model on the following concepts:

Cell and cellular space

Neighborhood (local interaction)

Cell state

Transition rule

We do not model all the details and characteristics of biological multicellular organisms but we obtain simple models where many interesting global and local phenomena can be observed.

Cellular Space

The cellular space constructs the cell network featuring

Dimension

Size

Neighbourhood

[Floreano et al.]

Cellular Automaton

A cellular automaton (CA) consists of a set of state-based cells CA={c₁,c₂,..,cn}.

Each cell i has a number of cell variables {x₁,..,xk}i

The variables define the data state S: S(zi)=Si={x₁,..,xk}

The next state of the cells is calculated via an activity function Act.

The state transition of a cell usually takes place in the activity function, optionally prepared by before (pre) or after (post) functions.

Scheduling

A CA is a parallel computational architecture. All cell state transitions can be computed in parallel at the same time.

But CA processing by generic computers requires a (semi-) sequential algorithm to perform cell state transitions ⇒ CA simulation with a scheduler

In a parallel system the following formula is valid for each cell (①right hand side, ②left hand side):

image-placeholder
(7-1)

Sequential computation would modify right hand side input before used!

Therefore, the old state of cells must be saved before the new state is computed!

A scheduler processes the individual calculation functions of all cells sequentially in three phases:

before: ∀ cells do pre(S)

activity: ∀ cells do Act(S)

after: ∀ cells do post(S)

image-placeholder
(7-2)

with Σ as the state of neighbourhood cells (or global state in general):

image-placeholder
(7-3)

Architecture and basic principle

The Cellular Automaton is a simulation tool that represents an artificial two-dimensional world of entities!

For visualization, each cell gets a color from a defined color palette

Figure 7-1. Cellular automaton as a network of simple communicating computational units

Cell States

There is a

Data State

Defined by the set of all cell states (variables), visible

Control State

The cell activation (update) sequence, hidden (Scheduler)

Typically, the data state can be compsoed of:

Input space X, e.g., a mapping of image pixels on cells 1:1

Intermediate hidden states T, i.e., auxiliary and temporary variables, e.g., t=x;x=y,y=t

Output space Y, e.g., mapping of cell states on an output image (pixels)

Cell Neighbourhood

Informally, it is the set of cells that can influence directly a given cell

In homogeneous cellular models it has the same shape for all cells

Neighbourhood defined some kind of communication (data transfer, but w/o synchronisation)

Moore

Moore neighbourhood is a squared region around a center cell position with a radius R.

Neumann

Neumann neighbourhood is an approximated "circular" region around a center cell position with a radius R. R=1 neighbourhood defines only the direct neighbours North, South, West, East.

Figure 7-2. Comparison of Moore and Neumann neighbourhood in different space dimensions [Floreano et al.]

Cell State Transitions

The value of the state of each cell belong to a finite set, whose elements are commonly numbers.

The value of the state is often represented by cell colors. A color function maps cell states on colors (from a palette)

There can be a special quiescent state s0.

The transition rule is the fundamental element of the CA.

It must specify the new state corresponding to each possible configuration of states of the cells in the neighborhood.

Figure 7-3. A transition rule maps states of neighbouring cells on new cell states. Cell states are visualized by colors. [Floreano et al.]

Cell updates by transition rules modify the cell states over (discrete) time. In order to start with the updating of the cells of the CA we must specify the initial state of the cells (initial conditions or seed)

[Floreano et al.]

Image processing with CA

Figure 7-4. An input image is mapped 1:1 on a cell matrix. The CA activation (iteratively) modifies the state of the cells, finally mapped 1:1 on an output (feature) image.

Kernel-based Transformations

Convolution or folding is a linear operation applied to each CA cell → Linear and state-less CA transition rules

Common kernel-based transformations:

Denoising

Intensity gradient

Edge detection / amplification

A kernel-based transformation produces always a linear output:

image-placeholder
(7-4)

with k: kernal/folding matrix, a,b: index of center point of folding matrix k

Effect | Kernel
Average | image-placeholder
Sharpening | image-placeholder
Relief | image-placeholder

Effect | Kernel
Edge, Laplace | image-placeholder
Edge, Sobel | image-placeholder
Noise?, Bosse | image-placeholder

Most kernel-based edge and gradient computations are sensitive in a specific direction (axis).

E\.g., there are two Sobel edge filter kernel, one for the x-, and one for the y-axis sensitive, finally combined:

image-placeholder
(7-11)

Iterative Transformations

Kernel-based transformations usually terminates after one step, except if the input is the output of the previous update phase!

In iterative transformations, the input of the next update phase is the output from the last (or the initial seed image data)

Edge Detection with CA

For edge detection of intensity images, some approaches first convert the intensity images into binary ones, and then evolve two-state cellular automata using specific state transition rules to determine edge pixels, while the others directly update pixel states based on the relationship of the central pixel with its neighbourhood, mostly a 1-ring von Neumann or Moore neighbourhood.

Popovici et al. proposed an edge detection approach based on the state differences between the central pixel and the pixels in its von Neumann neighbourhood. If all the absolute state differences are less than a threshold ε, then the state of the central pixel becomes 0, otherwise it remains unaltered.

The rule can be formulated as:

image-placeholder
(7-12)

where s and s⁺ are the current and the updated states of the central cell i, Ni is the von Neumann neighbourhood of the cell c, and sj is the current state value of the cell j in Ni.

In the case of feature marking, the conditional expression || < δ can be applied to other input variables (input image), and the state s represents a marked feature count only. But in this case it is a one-step update.

Wongthanavasu and Sadananda proposed a conditional rule to update the cellular state as:

image-placeholder
(7-13)

where smax and smin are the maximum and minimum states, respectively, in the von Neumann neighbourhood Ni of the central cell i.

Agent-based Image Processing: Feature Marking

Lui proposed a similar approach of neighbourhood exploration for feature marking in mages using mobile agents.

In contrast to Ca cells, the state of an agent is not fixed to a specific position.

There are explorations agents with the following behaviour:

Replication. If the agent finds a feature at the current position, it stays (or increasing the feature count), then replicate a specific number r of child agents that migrate to a neighboruing cell (randomly chosen).

Diffusion. If the agent do not find a feature, it migrates to neighboruing cell (randomly chosen).

The number of diffusions and hops is limited.

At each new place the feature detection starts again.

Conditional expression for feature detection:

image-placeholder
(7-14)

with v as the input pixel intensity and s as the feature marking count.

CA Implementation of FM-MAS

state : { input:0, output:0, weight:1, deltaweight:0 },
activity : function (x,y) {
 // neighbors: ordering [NW,N,NW,W,E,SW,S,SE]
 var countdn = this.ask(this.neighbors, 'absdiffcount', 'input',this.delta,'<='),
     mark  = countdn>=this.eps1 && countdn <= this.eps2;
 if (mark) {
   this.output += this.weight;
   if (this.dynamic) {
     // simulate new agent replications on heighbor nodes by increasing weight
     for (var i=0;i<this.replicate*this.weight;i++) {
       var delta = [0,0];
       while (delta[0]==0 && delta[1]==0)
         delta=[Math.random.select([-1,0,1]),Math.random.select([-1,0,1])];
       this.set(delta,'deltaweight',this.get(delta,'deltaweight')+1);
     }
   }
  } else if (this.dynamic && this.weight>0) {
    this.deltaweight--;
    // simulate aagent diffusion to heighbor node by increasing weight
    var delta = [0,0];
    while (delta[0]==0 && delta[1]==0)
      delta=[Math.random.select([-1,0,1]),Math.random.select([-1,0,1])];
    this.set(delta,'deltaweight',this.get(delta,'deltaweight')+1);
  }
},
after : function (x,y) {
  this.weight += this.deltaweight;
  this.deltaweight=0;
},

Learning CA

Which transition rules leads to a specific (global) emergent behaviour, e.g., edge detection?

Besides classical folding kernels, the answer is unknown.

Solution: Learn the rules, i.e., the kernel matrix! E.g., by using evolutinary and genetic algorithms.

Density-based Pointcloud Clustering DBSCAN

The output of the CA is an image with pixels representing point-wise features

But point clouds are still high-dimensioal

Dimensionality reduction by finding groups of points close together ⇒ Clustering

Density-based spatial clustering of applications with noise (DBSCAN) is a well-known data clustering algorithm that is commonly used in data mining and machine learning.

Algorithm

Based on a set of points (let’s think in a bidimensional space as exemplified in the figure), DBSCAN groups together points that are close to each other based on a distance measurement (usually Euclidean distance) and a minimum number of points.

It also marks as outliers the points that are in low-density regions.

Input

List of all point coordinates, can be pre-clustered in independent lists if the point belong to different classes

Output

List of point index lists with respect to the original input list. Each sub-list is a cluster group.

Figure 7-5. Points: Core - This is a point that has at least m points within distance n from itself. Border - This is a point that has at least one core point at a distance n. Noise - This is a point that is neither a Core nor a Border. And it has less than m points within distance n from itself. [https://www.kdnuggets.com/2020/04/dbscan-clustering-algorithm-machine-learning.html]

The algorithm proceeds by arbitrarily picking up a point in the dataset (until all points have been visited).

If there are at least ‘minPoint’ points within a radius of ‘ε’ to the point then we consider all these points to be part of the same cluster.

The clusters are then expanded by recursively repeating the neighborhood calculation for each neighboring point

Parameters

The DBSCAN algorithm basically requires 2 parameters:

[https://towardsdatascience.com/how-dbscan-works-and-why-should-i-use-it-443b4a191c80]

eps

specifies how close points should be to each other to be considered a part of a cluster. It means that if the distance between two points is lower or equal to this value (eps), these points are considered neighbors.

minPoints

the minimum number of points to form a dense region. For example, if we set the minPoints parameter as 5, then we need at least 5 points to form a dense region.

Parameter estimation

The parameter estimation is a problem for every data mining task. To choose good parameters we need to understand how they are used and have at least a basic previous knowledge about the data set that will be used.

eps

if the eps value chosen is too small, a large part of the data will not be clustered. It will be considered outliers because don’t satisfy the number of points to create a dense region. On the other hand, if the value that was chosen is too high, clusters will merge and the majority of objects will be in the same cluster. The eps should be chosen based on the distance of the dataset (we can use a k-distance graph to find it), but in general small eps values are preferable.

minPoints

As a general rule, a minimum minPoints can be derived from a number of dimensions (D) in the data set, as minPoints ≥ D + 1. Larger values are usually better for data sets with noise and will form more significant clusters. The minimum value for the minPoints must be 3, but the larger the data set, the larger the minPoints value that should be chosen.

More information:

Ester, M., Kriegel, H. P., Sander, J., & Xu, X. (1996, August). A density-based algorithm for discovering clusters in large spatial databases with noise. In Kdd (Vol. 96, №34, pp. 226–231).

Polygon Hulls

Up to here, we have still point clouds, although clustered.

For, e.g., area or perimeter computation of a cluster, a (concave) polygon hull approximation is needed (simplifying, averaging, and denoising point cloud)

Well known algorithms and software:

Graham Scan

Polylidar

The hull may not be limited to monotonic hulls without holes and caves (mixing of convex and concave hull segments)!

A hull can be described by a list of ordered boundary points

A hull can be described by a list of pice-wise linear lines (between boundary points)

Figure 7-6. The outline of a point cloud as a concave hull [https://gis.stackexchange.com/questions/143821/how-to-find-the-concave-hull-for-a-cloud-of-points-in-3d-space]

Further readings:

Implementation of a fast and efficient concave hull algorithm, http://www.it.uu.se/edu/course/homepage/projektTDB/ht13/project10/Project-10-report.pdf

Convolutional Neural Networks

CNN are suitable for classifying different structural features in the data regardless of location and orientation

A CNN is based on matrix algebra with convolution operations

For further reading: https://towardsdatascience.com/covolutional-neural-network-cb0883dd6529?gi=521747216671; G. Paaß, Artificial Intelligence, What is behind the technology of the future?, Jumper

Well-known software framework for the browser: convnet.js https://cs.stanford.edu/people/karpathy/convnetjs

Folding Operation

Figure 7-7. First calculation step (left) and second calculation step (right) in the convolution layer for a shifted small area of the input matrix. The kernel is "pushed" successively over the entire input matrix and the result matrix is filled [20]

Convolutional Neural Networks

A CNN is composed of different layers:

Convolution layers

Merging (pooling) layers

Output by classification layers (softmax)

Figure 7-8. A convolution layer contains k kernels and k result matrices, which are each combined into tensors [20]

Figure 7-9. General construction of a CNN with alternating layers of convolutions, merging, and finally binary classification [https://towardsdatascience.com]

Figure 7-10. Depending on the number of structural features, there may be a large number of the following folding and merging layers [https://towardsdatascience.com]

CNN Point Classifier

As with the CA approach, a CNN can be used to predict feature output images ⇒ CNN Image Feature Marking

The idea: A CNN is used to predict the feature class (e.g., marking of pixel belonging to a crack shape) with a small moving segment window.

The central pixel of this segment (e.g., (10,10) in a 20 × 20 pixel segment) is the pixel for which the CNN should predict the feature classification

Figure 7-11. Iterative image pixel feature marking by a moving segment window

Training is supervised with labelled input data.

Training and prediction poses high computational complexity.

The CNN can be simple, e.g.: two convoultional layers with 8 and 16 filter, respectively, one soft-max layer. But even such a simple CNN has about 10000 parameters!

Training

We need a set of labelled images. The regions with a feature are described with closed polygon paths. For each input image of size w × h, we get a feature map image of size w × h pixels. A randomly selected and shuffled segment set is used for training (created in advance). Segmentation can be strided (stride=Δx,Δy)

Prediction

The model is applied to all pixels of an input image with optional striding. The result is a feature map images of size w/stride × h/stride pixels.

Summary

Data can be represented by different data types, structures, formats

Data access is provided by SQL data bases in an unified way using a remote procedure call programming interface

Data processing is performed by using an event-based data-flow architecture

Further Reading

Bio-Inspired Artificial Intelligence: Theories, Methods, and Technologies by Dario Floreano and Claudio Mattiussi, MIT Press