Visible to the public Biblio

Filters: Keyword is measurement uncertainty  [Clear All Filters]
2023-05-12
Zhang, Qirui, Meng, Siqi, Liu, Kun, Dai, Wei.  2022.  Design of Privacy Mechanism for Cyber Physical Systems: A Nash Q-learning Approach. 2022 China Automation Congress (CAC). :6361–6365.

This paper studies the problem of designing optimal privacy mechanism with less energy cost. The eavesdropper and the defender with limited resources should choose which channel to eavesdrop and defend, respectively. A zero-sum stochastic game framework is used to model the interaction between the two players and the game is solved through the Nash Q-learning approach. A numerical example is given to verify the proposed method.

ISSN: 2688-0938

2023-01-06
Zhang, Han, Luo, Xiaoxiao, Li, Yongfu, Sima, Wenxia, Yang, Ming.  2022.  A Digital Twin Based Fault Location Method for Transmission Lines Using the Recovery Information of Instrument Transformers. 2022 IEEE International Conference on High Voltage Engineering and Applications (ICHVE). :1—4.
The parameters of transmission line vary with environmental and operating conditions, thus the paper proposes a digital twin-based transmission line model. Based on synchrophasor measurements from phasor measurement units, the proposed model can use the maximum likelihood estimation (MLE) to reduce uncertainty between the digital twin and its physical counterpart. A case study has been conducted in the paper to present the influence of the uncertainty in the measurements on the digital twin for the transmission line and analyze the effectiveness of the MLE method. The results show that the proposed digital twin-based model is effective in reducing the influence of the uncertainty in the measurements and improving the fault location accuracy.
Shaikh, Rizwan Ahmed, Sohaib Khan, Muhammad, Rashid, Imran, Abbas, Haidar, Naeem, Farrukh, Siddiqi, Muhammad Haroon.  2022.  A Framework for Human Error, Weaknesses, Threats & Mitigation Measures in an Airgapped Network. 2022 2nd International Conference on Digital Futures and Transformative Technologies (ICoDT2). :1—8.

Many organizations process and store classified data within their computer networks. Owing to the value of data that they hold; such organizations are more vulnerable to targets from adversaries. Accordingly, the sensitive organizations resort to an ‘air-gap’ approach on their networks, to ensure better protection. However, despite the physical and logical isolation, the attackers have successfully manifested their capabilities by compromising such networks; examples of Stuxnet and Agent.btz in view. Such attacks were possible due to the successful manipulation of human beings. It has been observed that to build up such attacks, persistent reconnaissance of the employees, and their data collection often forms the first step. With the rapid integration of social media into our daily lives, the prospects for data-seekers through that platform are higher. The inherent risks and vulnerabilities of social networking sites/apps have cultivated a rich environment for foreign adversaries to cherry-pick personal information and carry out successful profiling of employees assigned with sensitive appointments. With further targeted social engineering techniques against the identified employees and their families, attackers extract more and more relevant data to make an intelligent picture. Finally, all the information is fused to design their further sophisticated attacks against the air-gapped facility for data pilferage. In this regard, the success of the adversaries in harvesting the personal information of the victims largely depends upon the common errors committed by legitimate users while on duty, in transit, and after their retreat. Such errors would keep on repeating unless these are aligned with their underlying human behaviors and weaknesses, and the requisite mitigation framework is worked out.

2022-12-09
Doebbert, Thomas Robert, Fischer, Florian, Merli, Dominik, Scholl, Gerd.  2022.  On the Security of IO-Link Wireless Communication in the Safety Domain. 2022 IEEE 27th International Conference on Emerging Technologies and Factory Automation (ETFA). :1—8.

Security is an essential requirement of Industrial Control System (ICS) environments and its underlying communication infrastructure. Especially the lowest communication level within Supervisory Control and Data Acquisition (SCADA) systems - the field level - commonly lacks security measures.Since emerging wireless technologies within field level expose the lowest communication infrastructure towards potential attackers, additional security measures above the prevalent concept of air-gapped communication must be considered.Therefore, this work analyzes security aspects for the wireless communication protocol IO-Link Wireless (IOLW), which is commonly used for sensor and actuator field level communication. A possible architecture for an IOLW safety layer has already been presented recently [1].In this paper, the overall attack surface of IOLW within its typical environment is analyzed and attack preconditions are investigated to assess the effectiveness of different security measures. Additionally, enhanced security measures are evaluated for the communication systems and the results are summarized. Also, interference of security measures and functional safety principles within the communication are investigated, which do not necessarily complement one another but may also have contradictory requirements.This work is intended to discuss and propose enhancements of the IOLW standard with additional security considerations in future implementations.

2022-10-20
Alexan, Wassim, Mamdouh, Eyad, Elkhateeb, Abdelrahman, Al-Seba'ey, Fahd, Amr, Ziad, Khalil, Hana.  2021.  Securing Sensitive Data Through Corner Filters, Chaotic Maps and LSB Embedding. 2021 3rd Novel Intelligent and Leading Emerging Sciences Conference (NILES). :359—364.
This paper proposes 2 multiple layer message security schemes. Information security is carried out through the implementation of cryptography, steganography and image processing techniques. In both schemes, the sensitive data is first encrypted by employing a chaotic function. In the first proposed scheme, LSB steganography is then applied to 2D slices of a 3D image. In the second proposed scheme, a corner detection filter is first applied to the 2D slices of a 3D image, then LSB embedding is carried out in those corner-detected pixels. The number of neighboring pixels used for corner detection is varied and its effect is noted. Performance of the proposed schemes is numerically evaluated using a number of metrics, including the mean squared error (MSE), the peak signal to noise ratio (PSNR), the structure similarity index measure (SSIM), the normalized cross-correlation (NCC), the image fidelity (IF), as well as the image difference (ID). The proposed schemes exhibit superior payload capacity and security in comparison to their counterparts from the literature.
Larsen, Raphaël M.J.I., Pahl, Marc-Oliver, Coatrieux, Gouenou.  2021.  Authenticating IDS autoencoders using multipath neural networks. 2021 5th Cyber Security in Networking Conference (CSNet). :1—9.
An Intrusion Detection System (IDS) is a core element for securing critical systems. An IDS can use signatures of known attacks, or an anomaly detection model for detecting unknown attacks. Attacking an IDS is often the entry point of an attack against a critical system. Consequently, the security of IDSs themselves is imperative. To secure model-based IDSs, we propose a method to authenticate the anomaly detection model. The anomaly detection model is an autoencoder for which we only have access to input-output pairs. Inputs consist of time windows of values from sensors and actuators of an Industrial Control System. Our method is based on a multipath Neural Network (NN) classifier, a newly proposed deep learning technique. The idea is to characterize errors of an IDS's autoencoder by using a multipath NN's confidence measure \$c\$. We use the Wilcoxon-Mann-Whitney (WMW) test to detect a change in the distribution of the summary variable \$c\$, indicating that the autoencoder is not working properly. We compare our method to two baselines. They consist in using other summary variables for the WMW test. We assess the performance of these three methods using simulated data. Among others, our analysis shows that: 1) both baselines are oblivious to some autoencoder spoofing attacks while 2) the WMW test on a multipath NN's confidence measure enables detecting eventually any autoencoder spoofing attack.
2022-09-16
Simankov, Vladimir S., Buchatskiy, Pavel Yu., Shopin, Andrey V., Teploukhov, Semen V., Buchatskaya, Victoria V..  2021.  An Approach to Identifying the Type of Uncertainty of Initial Information Based on the Theory of Fuzzy Logic. 2021 XXIV International Conference on Soft Computing and Measurements (SCM). :150—153.
The article discusses an approach to identifying the uncertainty of initial information based on the theory of fuzzy logic. A system of criteria for initial information is proposed, calculated on the basis of the input sample, and characterizing the measure of uncertainty present in the system. The basic requirements for the choice of membership functions of the fuzzy inference system are indicated and the final integrated output membership function is obtained, which describes the type of uncertainty of the initial information.
2022-09-09
Teichel, Kristof, Lehtonen, Tapio, Wallin, Anders.  2021.  Assessing Time Transfer Methods for Accuracy and Reliability : Navigating the Time Transfer Trade-off Triangle. 2021 Joint Conference of the European Frequency and Time Forum and IEEE International Frequency Control Symposium (EFTF/IFCS). :1—4.
We present a collected overview on how to assess both the accuracy and reliability levels and relate them to the required effort, for different digital methods of synchronizing clocks. The presented process is intended for end users who require time synchronization but are not certain about how to judge at least one of the aspects. It can not only be used on existing technologies but should also be transferable to many future approaches. We further relate this approach to several examples. We discuss in detail the approach of medium-range White Rabbit connections over dedicated fibers, a method that occupies an extreme corner in the evaluation, where the effort is exceedingly high, but also yields excellent accuracy and significant reliability.
2022-08-12
Ooi, Boon-Yaik, Liew, Soung-Yue, Beh, Woan-Lin, Shirmohammadi, Shervin.  2021.  Inter-Batch Gap Filling Using Compressive Sampling for Low-Cost IoT Vibration Sensors. 2021 IEEE International Instrumentation and Measurement Technology Conference (I2MTC). :1—6.
To measure machinery vibration, a sensor system consisting of a 3-axis accelerometer, ADXL345, attached to a self-contained system-on-a-chip with integrated Wi-Fi capabilities, ESP8266, is a low-cost solution. In this work, we first show that in such a system, the widely used direct-read-and-send method which samples and sends individually acquired vibration data points to the server is not effective, especially using Wi-Fi connection. We show that the micro delays in each individual data transmission will limit the sensor sampling rate and will also affect the time of the acquired data points not evenly spaced. Then, we propose that vibration should be sampled in batches before sending the acquired data out from the sensor node. The vibration for each batch should be acquired continuously without any form of interruption in between the sampling process to ensure the data points are evenly spaced. To fill the data gaps between the batches, we propose the use of compressive sampling technique. Our experimental results show that the maximum sampling rate of the direct-read-and-send method is 350Hz with a standard uncertainty of 12.4, and the method loses more information compared to our proposed solution that can measure the vibration wirelessly and continuously up to 633Hz. The gaps filled using compressive sampling can achieve an accuracy in terms of mean absolute error (MAE) of up to 0.06 with a standard uncertainty of 0.002, making the low-cost vibration sensor node a cost-effective solution.
2022-07-29
Sever, Pop Septimiu, Vlad, Bande.  2021.  LC Oscillator Design Used in Sensor Measurement Based on Embedded Technology. 2021 IEEE 27th International Symposium for Design and Technology in Electronic Packaging (SIITME). :125–128.
This paper emphasizes the implementation of a conditioning circuit specific for an inductive or capacitive sensor. There are some inductive sensors, such as the proximity sensor, for which the inductance is dependent with the distance, or capacitive sensors strongly dependent with the humidity, distance, etc. This category of sensors is suitable for AC domain excitation from the measurement procedure point of view. Taking into consideration the fabrication technology, the measured physical quantity is being encoded as frequency or amplitude. To generate a sinusoidal signal with constant frequency and amplitude, the Colpitts or Hartley oscillators can be used [1], [2]. But the novelty of this paper is a different approach which reveals a microcontroller-based technology where the LC circuit works in an oscillating regime even though there is an underdamped oscillation behavior. For the oscillations’ occurrence, there will be a periodical energy injection using a driving source. One of the main advantages of the mentioned circuit is the small component number. The central unit of the embedded system will fulfil two functions: maintains the oscillating regime and measures the amplitude or frequency of the output signal. In this way, the built embedded system will be robust and easy to use due to its software configuration capabilities. As a plus, such a system can measure additional sensors used in environment parameters’ compensating procedure.
Ruderman, Michael.  2021.  Robust output feedback control of non-collocated low-damped oscillating load. 2021 29th Mediterranean Conference on Control and Automation (MED). :639–644.
For systems with order of dynamics higher than two and oscillating loads with low damping, a non-collocation of the sensing and control can deteriorate robustness of the feedback and, in worst case, even bring it to instability. Furthermore, for a contactless sensing of the oscillating mechanical load, like in the system under investigation, the control structure is often restricted to the single proportional feedback only. This paper proposes a novel robust feedback control scheme for a low-damped fourth-order system using solely the measured load displacement. For reference tracking, the loop shaping design relies on a band reject filter, while the plant uncertainties are used as robustness measure for determining the feedback gain. Since prime uncertainties are due to the stiffness of elastic link, correspondingly connecting spring, and due to the gain of actuator transducer, the loop sensitivity function with additive plant variation is used for robustness measure. In order to deal with unknown disturbances, which are inherently exciting the load oscillations independently of the loop shaping performance, an output delay-based compensator is proposed as a second control-degree-of-freedom. That one requires an estimate of the load oscillation frequency only and does not affect the shaped open-loop behavior, correspondingly sensitivity function. An extensive numerical setup of the modeled system, a two-mass oscillator with contactless sensing of the load under gravity and low damping of the connecting spring, is used for the control evaluation and assessment of its robustness.
Luo, Weifeng, Xiao, Liang.  2021.  Reinforcement Learning Based Vulnerability Analysis of Data Injection Attack for Smart Grids. 2021 40th Chinese Control Conference (CCC). :6788—6792.
Smart grids have to protect meter measurements against false data injection attacks. By modifying the meter measurements, the attacker misleads the control decisions of the control center, which results in physical damages of power systems. In this paper, we propose a reinforcement learning based vulnerability analysis scheme for data injection attack without relying on the power system topology. This scheme enables the attacker to choose the data injection attack vector based on the meter measurements, the power system status, the previous injected errors and the number of meters to compromise. By combining deep reinforcement learning with prioritized experience replay, the proposed scheme more frequently replays the successful vulnerability detection experiences while bypassing the bad data detection, which is able to accelerate the learning speed. Simulation results based on the IEEE 14 bus system show that this scheme increases the probability of successful vulnerability detection and reduce the number of meters to compromise compared with the benchmark scheme.
2022-06-09
Cohen, Myke C., Demir, Mustafa, Chiou, Erin K., Cooke, Nancy J..  2021.  The Dynamics of Trust and Verbal Anthropomorphism in Human-Autonomy Teaming. 2021 IEEE 2nd International Conference on Human-Machine Systems (ICHMS). :1–6.
Trust in autonomous teammates has been shown to be a key factor in human-autonomy team (HAT) performance, and anthropomorphism is a closely related construct that is underexplored in HAT literature. This study investigates whether perceived anthropomorphism can be measured from team communication behaviors in a simulated remotely piloted aircraft system task environment, in which two humans in unique roles were asked to team with a synthetic (i.e., autonomous) pilot agent. We compared verbal and self-reported measures of anthropomorphism with team error handling performance and trust in the synthetic pilot. Results for this study show that trends in verbal anthropomorphism follow the same patterns expected from self-reported measures of anthropomorphism, with respect to fluctuations in trust resulting from autonomy failures.
Jie, Chen.  2021.  Information Security Risk Assessment of Industrial Control System Based on Hybrid Genetic Algorithms. 2021 13th International Conference on Measuring Technology and Mechatronics Automation (ICMTMA). :423–426.
In order to solve the problem of quantitative assessment of information security risks in industrial control systems, this paper proposes a method of information security risk assessment for industrial control systems based on modular hybrid genetic algorithm. Combining with the characteristics of industrial control systems, the use of hybrid genetic algorithm evidence theory to identify, evaluate and assess assets and threats, and ultimately come to the order of the size of the impact of security threats on the specific industrial control system information security. This method can provide basis for making decisions to reduce information security risks in the control system from qualitative and quantitative aspects.
2022-05-03
HAMRIOUI, Sofiane, BOKHARI, Samira.  2021.  A new Cybersecurity Strategy for IoE by Exploiting an Optimization Approach. 2021 12th International Conference on Information and Communication Systems (ICICS). :23—28.

Today's companies are increasingly relying on Internet of Everything (IoE) to modernize their operations. The very complexes characteristics of such system expose their applications and their exchanged data to multiples risks and security breaches that make them targets for cyber attacks. The aim of our work in this paper is to provide an cybersecurity strategy whose objective is to prevent and anticipate threats related to the IoE. An economic approach is used in order to help to take decisions according to the reduction of the risks generated by the non definition of the appropriate levels of security. The considered problem have been resolved by exploiting a combinatorial optimization approach with a practical case of knapsack. We opted for a bi-objective modeling under uncertainty with a constraint of cardinality and a given budget to be respected. To guarantee a robustness of our strategy, we have also considered the criterion of uncertainty by taking into account all the possible threats that can be generated by a cyber attacks over IoE. Our strategy have been implemented and simulated under MATLAB environement and its performance results have been compared to those obtained by NSGA-II metaheuristic. Our proposed cyber security strategy recorded a clear improvment of efficiency according to the optimization of the security level and cost parametrs.

2022-03-23
Lyu, Chen, Huang, Dongmei, Jia, Qingyao, Han, Xiao, Zhang, Xiaomei, Chi, Chi-Hung, Xu, Yang.  2021.  Predictable Model for Detecting Sybil Attacks in Mobile Social Networks. 2021 IEEE Wireless Communications and Networking Conference (WCNC). :1—6.
Mobile Social Networks have become one of the most convenient services for users to share information everywhere. This crowdsourced information is often meaningful and recommended to users, e.g., reviews on Yelp or high marks on Dianping, which poses the threat of Sybil attacks. To address the problem of Sybil attacks, previous solutions mostly use indirect/direct graph model or clickstream model to detect fake accounts. However, they are either dependent on strong connections or solely preserved by servers of social networks. In this paper, we propose a novel predictable approach by exploiting users' custom patterns to distinguish Sybil attackers from normal users for the application of recommendation in mobile social networks. First, we introduce the entropy of spatial-temporal features to profile the mobility traces of normal users, which is quite different from Sybil attackers. Second, we develop discriminative entropy-based features, i.e., users' preference features, to measure the uncertainty of users' behaviors. Third, we design a smart Sybil detection model based on a binary classification approach by combining our entropy-based features with traditional behavior-based features. Finally, we examine our model and carry out extensive experiments on a real-world dataset from Dianping. Our results have demonstrated that the model can significantly improve the detection accuracy of Sybil attacks.
2021-05-13
Li, Xu, Zhong, Jinghua, Wu, Xixin, Yu, Jianwei, Liu, Xunying, Meng, Helen.  2020.  Adversarial Attacks on GMM I-Vector Based Speaker Verification Systems. ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). :6579—6583.
This work investigates the vulnerability of Gaussian Mixture Model (GMM) i-vector based speaker verification systems to adversarial attacks, and the transferability of adversarial samples crafted from GMM i-vector based systems to x-vector based systems. In detail, we formulate the GMM i-vector system as a scoring function of enrollment and testing utterance pairs. Then we leverage the fast gradient sign method (FGSM) to optimize testing utterances for adversarial samples generation. These adversarial samples are used to attack both GMM i-vector and x-vector systems. We measure the system vulnerability by the degradation of equal error rate and false acceptance rate. Experiment results show that GMM i-vector systems are seriously vulnerable to adversarial attacks, and the crafted adversarial samples are proved to be transferable and pose threats to neural network speaker embedding based systems (e.g. x-vector systems).
2021-03-29
Kummerow, A., Monsalve, C., Rösch, D., Schäfer, K., Nicolai, S..  2020.  Cyber-physical data stream assessment incorporating Digital Twins in future power systems. 2020 International Conference on Smart Energy Systems and Technologies (SEST). :1—6.

Reliable and secure grid operations become more and more challenging in context of increasing IT/OT convergence and decreasing dynamic margins in today's power systems. To ensure the correct operation of monitoring and control functions in control centres, an intelligent assessment of the different information sources is necessary to provide a robust data source in case of critical physical events as well as cyber-attacks. Within this paper, a holistic data stream assessment methodology is proposed using an expert knowledge based cyber-physical situational awareness for different steady and transient system states. This approach goes beyond existing techniques by combining high-resolution PMU data with SCADA information as well as Digital Twin and AI based anomaly detection functionalities.

Zimmo, S., Refaey, A., Shami, A..  2020.  Trusted Boot for Embedded Systems Using Hypothesis Testing Benchmark. 2020 IEEE Canadian Conference on Electrical and Computer Engineering (CCECE). :1—2.

Security has become a crucial consideration and is one of the most important design goals for an embedded system. This paper examines the type of boot sequence, and more specifically a trusted boot which utilizes the method of chain of trust. After defining these terms, this paper will examine the limitations of the existing safe boot, and finally propose the method of trusted boot based on hypothesis testing benchmark and the cost it takes to perform this method.

2020-05-15
Sugrim, Shridatt, Venkatesan, Sridhar, Youzwak, Jason A., Chiang, Cho-Yu J., Chadha, Ritu, Albanese, Massimiliano, Cam, Hasan.  2018.  Measuring the Effectiveness of Network Deception. 2018 IEEE International Conference on Intelligence and Security Informatics (ISI). :142—147.

Cyber reconnaissance is the process of gathering information about a target network for the purpose of compromising systems within that network. Network-based deception has emerged as a promising approach to disrupt attackers' reconnaissance efforts. However, limited work has been done so far on measuring the effectiveness of network-based deception. Furthermore, given that Software-Defined Networking (SDN) facilitates cyber deception by allowing network traffic to be modified and injected on-the-fly, understanding the effectiveness of employing different cyber deception strategies is critical. In this paper, we present a model to study the reconnaissance surface of a network and model the process of gathering information by attackers as interactions with a cyber defensive system that may use deception. To capture the evolution of the attackers' knowledge during reconnaissance, we design a belief system that is updated by using a Bayesian inference method. For the proposed model, we present two metrics based on KL-divergence to quantify the effectiveness of network deception. We tested the model and the two metrics by conducting experiments with a simulated attacker in an SDN-based deception system. The results of the experiments match our expectations, providing support for the model and proposed metrics.

2018-08-23
Xu, W., Yan, Z., Tian, Y., Cui, Y., Lin, J..  2017.  Detection with compressive measurements corrupted by sparse errors. 2017 9th International Conference on Wireless Communications and Signal Processing (WCSP). :1–5.

Compressed sensing can represent the sparse signal with a small number of measurements compared to Nyquist-rate samples. Considering the high-complexity of reconstruction algorithms in CS, recently compressive detection is proposed, which performs detection directly in compressive domain without reconstruction. Different from existing work that generally considers the measurements corrupted by dense noises, this paper studies the compressive detection problem when the measurements are corrupted by both dense noises and sparse errors. The sparse errors exist in many practical systems, such as the ones affected by impulse noise or narrowband interference. We derive the theoretical performance of compressive detection when the sparse error is either deterministic or random. The theoretical results are further verified by simulations.

2018-04-04
Lin, Y., Abur, A..  2017.  Identifying security vulnerabilities of weakly detectable network parameter errors. 2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton). :295–301.
This paper is concerned about the security vulnerabilities in the implementation of the Congestion Revenue Rights (CRR) markets. Such problems may be due to the weakly detectable network model parameter errors which are commonly found in power systems. CRRs are financial tools for hedging the risk of congestion charges in power markets. The reimbursements received by CRR holders are determined by the congestion patterns and Locational Marginal Prices (LMPs) in the day-ahead markets, which heavily rely on the parameters in the network model. It is recently shown that detection of errors in certain network model parameters may be very difficult. This paper's primary goal is to illustrate the lack of market security due to such vulnerabilities, i.e. CRR market calculations can be manipulated by injecting parameter errors which are not likely to be detected. A case study using the IEEE 14-bus system will illustrate the feasibility of such undetectable manipulations. Several suggestions for preventing such cyber security issues are provided at the end of the paper.
2015-05-06
Nower, N., Yasuo Tan, Lim, A.O..  2014.  Efficient Temporal and Spatial Data Recovery Scheme for Stochastic and Incomplete Feedback Data of Cyber-physical Systems. Service Oriented System Engineering (SOSE), 2014 IEEE 8th International Symposium on. :192-197.

Feedback loss can severely degrade the overall system performance, in addition, it can affect the control and computation of the Cyber-physical Systems (CPS). CPS hold enormous potential for a wide range of emerging applications including stochastic and time-critical traffic patterns. Stochastic data has a randomness in its nature which make a great challenge to maintain the real-time control whenever the data is lost. In this paper, we propose a data recovery scheme, called the Efficient Temporal and Spatial Data Recovery (ETSDR) scheme for stochastic incomplete feedback of CPS. In this scheme, we identify the temporal model based on the traffic patterns and consider the spatial effect of the nearest neighbor. Numerical results reveal that the proposed ETSDR outperforms both the weighted prediction (WP) and the exponentially weighted moving average (EWMA) algorithm regardless of the increment percentage of missing data in terms of the root mean square error, the mean absolute error, and the integral of absolute error.
 

2015-05-01
Baraldi, A., Boschetti, L., Humber, M.L..  2014.  Probability Sampling Protocol for Thematic and Spatial Quality Assessment of Classification Maps Generated From Spaceborne/Airborne Very High Resolution Images. Geoscience and Remote Sensing, IEEE Transactions on. 52:701-760.

To deliver sample estimates provided with the necessary probability foundation to permit generalization from the sample data subset to the whole target population being sampled, probability sampling strategies are required to satisfy three necessary not sufficient conditions: 1) All inclusion probabilities be greater than zero in the target population to be sampled. If some sampling units have an inclusion probability of zero, then a map accuracy assessment does not represent the entire target region depicted in the map to be assessed. 2) The inclusion probabilities must be: a) knowable for nonsampled units and b) known for those units selected in the sample: since the inclusion probability determines the weight attached to each sampling unit in the accuracy estimation formulas, if the inclusion probabilities are unknown, so are the estimation weights. This original work presents a novel (to the best of these authors' knowledge, the first) probability sampling protocol for quality assessment and comparison of thematic maps generated from spaceborne/airborne very high resolution images, where: 1) an original Categorical Variable Pair Similarity Index (proposed in two different formulations) is estimated as a fuzzy degree of match between a reference and a test semantic vocabulary, which may not coincide, and 2) both symbolic pixel-based thematic quality indicators (TQIs) and sub-symbolic object-based spatial quality indicators (SQIs) are estimated with a degree of uncertainty in measurement in compliance with the well-known Quality Assurance Framework for Earth Observation (QA4EO) guidelines. Like a decision-tree, any protocol (guidelines for best practice) comprises a set of rules, equivalent to structural knowledge, and an order of presentation of the rule set, known as procedural knowledge. The combination of these two levels of knowledge makes an original protocol worth more than the sum of its parts. The several degrees of novelty of the proposed probability sampling protocol are highlighted in this paper, at the levels of understanding of both structural and procedural knowledge, in comparison with related multi-disciplinary works selected from the existing literature. In the experimental session, the proposed protocol is tested for accuracy validation of preliminary classification maps automatically generated by the Satellite Image Automatic Mapper (SIAM™) software product from two WorldView-2 images and one QuickBird-2 image provided by DigitalGlobe for testing purposes. In these experiments, collected TQIs and SQIs are statistically valid, statistically significant, consistent across maps, and in agreement with theoretical expectations, visual (qualitative) evidence and quantitative quality indexes of operativeness (OQIs) claimed for SIAM™ by related papers. As a subsidiary conclusion, the statistically consistent and statistically significant accuracy validation of the SIAM™ pre-classification maps proposed in this contribution, together with OQIs claimed for SIAM™ by related works, make the operational (automatic, accurate, near real-time, robust, scalable) SIAM™ software product eligible for opening up new inter-disciplinary research and market opportunities in accordance with the visionary goal of the Global Earth Observation System of Systems initiative and the QA4EO international guidelines.