Biblio

With the rapid development of information technology, video surveillance system has become a key part in the security and protection system of modern cities. Especially in prisons, surveillance cameras could be found almost everywhere. However, with the continuous expansion of the surveillance network, surveillance cameras not only bring convenience, but also produce a massive amount of monitoring data, which poses huge challenges to storage, analytics and retrieval. The smart monitoring system equipped with intelligent video analytics technology can monitor as well as pre-alarm abnormal events or behaviours, which is a hot research direction in the field of surveillance. This paper combines deep learning methods, using the state-of-the-art framework for instance segmentation, called Mask R-CNN, to train the fine-tuning network on our datasets, which can efficiently detect objects in a video image while simultaneously generating a high-quality segmentation mask for each instance. The experiment show that our network is simple to train and easy to generalize to other datasets, and the mask average precision is nearly up to 98.5% on our own datasets.

In recent years, deep convolution neural networks (DCNNs) have won many contests in machine learning, object detection, and pattern recognition. Furthermore, deep learning techniques achieved exceptional performance in image classification, reaching accuracy levels beyond human capability. Malware variants from similar categories often contain similarities due to code reuse. Converting malware samples into images can cause these patterns to manifest as image features, which can be exploited for DCNN classification. Techniques for converting malware binaries into images for visualization and classification have been reported in the literature, and while these methods do reach a high level of classification accuracy on training datasets, they tend to be vulnerable to overfitting and perform poorly on previously unseen samples. In this paper, we explore and document a variety of techniques for representing malware binaries as images with the goal of discovering a format best suited for deep learning. We implement a database for malware binaries from several families, stored in hexadecimal format. These malware samples are converted into images using various approaches and are used to train a neural network to recognize visual patterns in the input and classify malware based on the feature vectors. Each image type is assessed using a variety of learning models, such as transfer learning with existing DCNN architectures and feature extraction for support vector machine classifier training. Each technique is evaluated in terms of classification accuracy, result consistency, and time per trial. Our preliminary results indicate that improved image representation has the potential to enable more effective classification of new malware.

As drone attracts much interest, the drone industry has opened their market to ordinary people, making drones to be used in daily lives. However, as it got easier for drone to be used by more people, safety and security issues have raised as accidents are much more likely to happen: colliding into people by losing control or invading secured properties. For safety purposes, it is essential for observers and drone to be aware of an approaching drone. In this paper, we introduce a comprehensive drone detection system based on machine learning. This system is designed to be operable on drones with camera. Based on the camera images, the system deduces location on image and vendor model of drone based on machine classification. The system is actually built with OpenCV library. We collected drone imagery and information for learning process. The system's output shows about 89 percent accuracy.

Video1 surveillance has been applied to various industries and business, but the video surveillance is mainly to provide video capture and display services at current, and does not care about video content. In some applications, users want to be able to monitor the video content and implement automatic prompting and alarming functions. Therefore, this paper uses the hybrid Gaussian model (GMM) to achieve the separation of the foreground and the background of the video and the experiments validate the effectiveness of the algorithm.

In this paper, we report our work on using machine learning techniques to predict back bending activity based on field data acquired in a local nursing home. The data are recorded by a privacy-aware compliance tracking system (PACTS). The objective of PACTS is to detect back-bending activities and issue real-time alerts to the participant when she bends her back excessively, which we hope could help the participant form good habits of using proper body mechanics when performing lifting/pulling tasks. We show that our algorithms can differentiate nursing staffs baseline and high-level bending activities by using human skeleton data without any expert rules.

Here we explore the applicability of traditional sliding window based convolutional neural network (CNN) detection pipeline and region based object detection techniques such as Faster Region-based CNN (R-CNN) and Region-based Fully Convolutional Networks (R-FCN) on the problem of object detection in X-ray security imagery. Within this context, with limited dataset availability, we employ a transfer learning paradigm for network training tackling both single and multiple object detection problems over a number of R-CNN/R-FCN variants. The use of first-stage region proposal within the Faster RCNN and R-FCN provide superior results than traditional sliding window driven CNN (SWCNN) approach. With the use of Faster RCNN with VGG16, pretrained on the ImageNet dataset, we achieve 88.3 mAP for a six object class X-ray detection problem. The use of R-FCN with ResNet-101, yields 96.3 mAP for the two class firearm detection problem requiring 0.1 second computation per image. Overall we illustrate the comparative performance of these techniques as object localization strategies within cluttered X-ray security imagery.

The world is witnessing a remarkable increase in the usage of video surveillance systems. Besides fulfilling an imperative security and safety purpose, it also contributes towards operations monitoring, hazard detection and facility management in industry/smart factory settings. Most existing surveillance techniques use hand-crafted features analyzed using standard machine learning pipelines for action recognition and event detection. A key shortcoming of such techniques is the inability to learn from unlabeled video streams. The entire video stream is unlabeled when the requirement is to detect irregular, unforeseen and abnormal behaviors, anomalies. Recent developments in intelligent high-level video analysis have been successful in identifying individual elements in a video frame. However, the detection of anomalies in an entire video feed requires incremental and unsupervised machine learning. This paper presents a novel approach that incorporates high-level video analysis outcomes with incremental knowledge acquisition and self-learning for autonomous video surveillance. The proposed approach is capable of detecting changes that occur over time and separating irregularities from re-occurrences, without the prerequisite of a labeled dataset. We demonstrate the proposed approach using a benchmark video dataset and the results confirm its validity and usability for autonomous video surveillance.

With crimes on the rise all around the world, video surveillance is becoming more important day by day. Due to the lack of human resources to monitor this increasing number of cameras manually, new computer vision algorithms to perform lower and higher level tasks are being developed. We have developed a new method incorporating the most acclaimed Histograms of Oriented Gradients, the theory of Visual Saliency and the saliency prediction model Deep Multi-Level Network to detect human beings in video sequences. Furthermore, we implemented the k - Means algorithm to cluster the HOG feature vectors of the positively detected windows and determined the path followed by a person in the video. We achieved a detection precision of 83.11% and a recall of 41.27%. We obtained these results 76.866 times faster than classification on normal images.

We present an object tracking framework which fuses multiple unstable video-based methods and supports automatic tracker initialization and termination. To evaluate our system, we collected a large dataset of hand-annotated 5-minute traffic surveillance videos, which we are releasing to the community. To the best of our knowledge, this is the first publicly available dataset of such long videos, providing a diverse range of real-world object variation, scale change, interaction, different resolutions and illumination conditions. In our comprehensive evaluation using this dataset, we show that our automatic object tracking system often outperforms state-of-the-art trackers, even when these are provided with proper manual initialization. We also demonstrate tracking throughput improvements of 5× or more vs. the competition.

This work introduces concepts and algorithms along with a case study validating them, to enhance the event detection, pattern recognition and anomaly identification results in real life video surveillance. The motivation for the work underlies in the observation that human behavioral patterns in general continuously evolve and adapt with time, rather than being static. First, limitations in existing work with respect to this phenomena are identified. Accordingly, the notion and algorithms of Dynamic Clustering are introduced in order to overcome these drawbacks. Correspondingly, we propose the concept of maintaining two separate sets of data in parallel, namely the Normal Plane and the Anomaly Plane, to successfully achieve the task of learning continuously. The practicability of the proposed algorithms in a real life scenario is demonstrated through a case study. From the analysis presented in this work, it is evident that a more comprehensive analysis, closely following human perception can be accomplished by incorporating the proposed notions and algorithms in a video surveillance event.

Conventional intrusion detection systems for smart grid communications rely heavily on static based attack detection techniques. In essence, signatures created from historical data are compared to incoming network traffic to identify abnormalities. In the case of attacks where no historical data exists, static based approaches become ineffective thus relinquishing system resilience and stability. Moving target defense (MTD) has shown to be effective in discouraging attackers by introducing system entropy to increase exploit costs. Increase in exploit cost leads to a decrease in profitability for an attacker. In this paper, a Moving Target Defense Intrusion Detection System (MTDIDS) is proposed for smart grid IPv6 based advanced metering infrastructure. The advantage of MTDIDS is the ability to detect anomalies across moving targets by means of planar keys thereupon increasing detection rate. Evaluation of MTDIDS was carried out in a smart grid advanced metering infrastructure simulated in MATLAB.

Deep learning techniques have demonstrated the ability to perform a variety of object recognition tasks using visible imager data; however, deep learning has not been implemented as a means to autonomously detect and assess targets of interest in a physical security system. We demonstrate the use of transfer learning on a convolutional neural network (CNN) to significantly reduce training time while keeping detection accuracy of physical security relevant targets high. Unlike many detection algorithms employed by video analytics within physical security systems, this method does not rely on temporal data to construct a background scene; targets of interest can halt motion indefinitely and still be detected by the implemented CNN. A key advantage of using deep learning is the ability for a network to improve over time. Periodic retraining can lead to better detection and higher confidence rates. We investigate training data size versus CNN test accuracy using physical security video data. Due to the large number of visible imagers, significant volume of data collected daily, and currently deployed human in the loop ground truth data, physical security systems present a unique environment that is well suited for analysis via CNNs. This could lead to the creation of algorithmic element that reduces human burden and decreases human analyzed nuisance alarms.

Unmanned Aerial Vehicles (UAVs) are autonomous aircraft that, when equipped with wireless communication interfaces, can share data among themselves when in communication range. Compared to single UAVs, using multiple UAVs as a collaborative swarm is considerably more effective for target tracking, reconnaissance, and surveillance missions because of their capacity to tackle complex problems synergistically. Success rates in target detection and tracking depend on map coverage performance, which in turn relies on network connectivity between UAVs to propagate surveillance results to avoid revisiting already observed areas. In this paper, we consider the problem of optimizing three objectives for a swarm of UAVs: (a) target detection and tracking, (b) map coverage, and (c) network connectivity. Our approach, Dual-Pheromone Clustering Hybrid Approach (DPCHA), incorporates a multi-hop clustering and a dual-pheromone ant-colony model to optimize these three objectives. Clustering keeps stable overlay networks, while attractive and repulsive pheromones mark areas of detected targets and visited areas. Additionally, DPCHA introduces a disappearing target model for dealing with temporarily invisible targets. Extensive simulations show that DPCHA produces significant improvements in the assessment of coverage fairness, cluster stability, and connection volatility. We compared our approach with a pure dual- pheromone approach and a no-base model, which removes the base station from the model. Results show an approximately 50% improvement in map coverage compared to the pure dual-pheromone approach.

In part I of a three-part series on active surveillance using depth-sensing technology, this paper proposes an algorithm to identify outdoor intrusion activities by monitoring skeletal positions from Microsoft Kinect sensor in real-time. This algorithm implements three techniques to identify a premise intrusion. The first technique observes a boundary line along the wall (or fence) of a surveilled premise for skeletal trespassing detection. The second technique observes the duration of a skeletal object within a region of a surveilled premise for loitering detection. The third technique analyzes the differences in skeletal height to identify wall climbing. Experiment results suggest that the proposed algorithm is able to detect trespassing, loitering and wall climbing at a rate of 70%, 85% and 80% respectively.

This paper address the problem of shadow detection and removal in traffic vision analysis. Basically, the presence of the shadow in the traffic sequences is imminent, and therefore leads to errors at segmentation stage and often misclassified as an object region or as a moving object. This paper presents a shadow removal method, based on both color and texture features, aiming to contribute to retrieve efficiently the moving objects whose detection are usually under the influence of cast-shadows. Additionally, in order to get a shadow-free foreground segmentation image, a morphology reconstruction algorithm is used to recover the foreground disturbed by shadow removal. Once shadows are detected, an automatic shadow removal model is proposed based on the information retrieved from the histogram shape. Experimental results on a real traffic sequence is presented to test the proposed approach and to validate the algorithm's performance.

In order to enhance the supply chain security at airports, the German federal ministry of education and research has initiated the project ESECLOG (enhanced security in the air cargo chain) which has the goal to improve the threat detection accuracy using one-sided access methods. In this paper, we present a new X-ray backscatter technology for non-intrusive imaging of suspicious objects (mainly low-Z explosives) in luggage's and parcels with only a single-sided access. A key element in this technology is the X-ray backscatter camera embedded with a special twisted-slit collimator. The developed technology has efficiently resolved the problem related to the imaging of complex interior of the object by fixing source and object positions and changing only the scanning direction of the X-ray backscatter camera. Experiments were carried out on luggages and parcels packed with mock-up dangerous materials including liquid and solid explosive simulants. In addition, the quality of the X-ray backscatter image was enhanced by employing high-resolution digital detector arrays. Experimental results are discussed and the efficiency of the present technique to detect suspicious objects in luggages and parcels is demonstrated. At the end, important applications of the proposed backscatter imaging technology to the aviation security are presented.

The detection of obstacles is a fundamental issue in autonomous navigation, as it is the main key for collision prevention. This paper presents a method for the segmentation of general obstacles by stereo vision with no need of dense disparity maps or assumptions about the scenario. A sparse set of points is selected according to a local spatial condition and then clustered in function of its neighborhood, disparity values and a cost associated with the possibility of each point being part of an obstacle. The method was evaluated in hand-labeled images from KITTI object detection benchmark and the precision and recall metrics were calculated. The quantitative and qualitative results showed satisfactory in scenarios with different types of objects.

Performance characterization of stereo methods is mandatory to decide which algorithm is useful for which application. Prevalent benchmarks mainly use the root mean squared error (RMS) with respect to ground truth disparity maps to quantify algorithm performance. We show that the RMS is of limited expressiveness for algorithm selection and introduce the HCI Stereo Metrics. These metrics assess stereo results by harnessing three semantic cues: depth discontinuities, planar surfaces, and fine geometric structures. For each cue, we extract the relevant set of pixels from existing ground truth. We then apply our evaluation functions to quantify characteristics such as edge fattening and surface smoothness. We demonstrate that our approach supports practitioners in selecting the most suitable algorithm for their application. Using the new Middlebury dataset, we show that rankings based on our metrics reveal specific algorithm strengths and weaknesses which are not quantified by existing metrics. We finally show how stacked bar charts and radar charts visually support multidimensional performance evaluation. An interactive stereo benchmark based on the proposed metrics and visualizations is available at: http://hci.iwr.uni-heidelberg.de/stereometrics.

Quadrature compressive sampling (QuadCS) is a newly introduced sub-Nyquist sampling for acquiring inphase and quadrature components of radio-frequency signals. This paper develops a target detection scheme of pulsed-type radars in the presence of digital radio frequency memory (DRFM) repeat jammers with the radar echoes sampled by the QuadCS system. For diversifying pulses, the proposed scheme first separates the target echoes from the DRFM repeat jammers using CS recovery algorithms, and then removes the jammers to perform the target detection. Because of the separation processing, the jammer leakage through range sidelobe variation of the classical match-filter processing will not appear. Simulation results confirm our findings. The proposed scheme with the data at one eighth the Nyquist rate outperforms the classic processing with Nyquist samples in the presence of DRFM repeat jammers.

The passive radar also known as Green Radar exploits the available commercial communication signals and is useful for target tracking and detection in general. Recent communications standards frequently employ Orthogonal Frequency Division Multiplexing (OFDM) waveforms and wideband for broadcasting. This paper focuses on the recent developments of the target detection algorithms in the OFDM passive radar framework where its channel estimates have been derived using the matched filter concept using the knowledge of the transmitted signals. The MUSIC algorithm, which has been modified to solve this two dimensional delay-Doppler detection problem, is first reviewed. As the target detection problem can be represented as sparse signals, this paper employs compressive sensing to compare with the detection capability of the 2-D MUSIC algorithm. It is found that the previously proposed single time sample compressive sensing cannot significantly reduce the leakage from the direct signal component. Furthermore, this paper proposes the compressive sensing method utilizing multiple time samples, namely l1-SVD, for the detection of multiple targets. In comparison between the MUSIC and compressive sensing, the results show that l1-SVD can decrease the direct signal leakage but its prerequisite of computational resources remains a major issue. This paper also presents the detection performance of these two algorithms for closely spaced targets.

We present a new method for mitigating wall return and a new greedy algorithm for detecting stationary targets after wall clutter has been cancelled. Given limited measurements of a stepped-frequency radar signal consisting of both wall and target return, our objective is to detect and localize the potential targets. Modulated Discrete Prolate Spheroidal Sequences (DPSS's) form an efficient basis for sampled bandpass signals. We mitigate the wall clutter efficiently within the compressive measurements through the use of a bandpass modulated DPSS basis. Then, in each step of an iterative algorithm for detecting the target positions, we use a modulated DPSS basis to cancel nearly all of the target return corresponding to previously selected targets. With this basis, we improve upon the target detection sensitivity of a Fourier-based technique.

Road In this paper, we focus on both the road vehicle and pedestrians detection, namely obstacle detection. At the same time, a new obstacle detection and classification technique in dynamical background is proposed. Obstacle detection is based on inverse perspective mapping and homography. Obstacle classification is based on fuzzy neural network. The estimation of the vanishing point relies on feature extraction strategy, which segments the lane markings of the images by combining a histogram-based segmentation with temporal filtering. Then, the vanishing point of each image is stabilized by means of a temporal filtering along the estimates of previous images. The IPM image is computed based on the stabilized vanishing point. The method exploits the geometrical relations between the elements in the scene so that obstacle can be detected. The estimated homography of the road plane between successive images is used for image alignment. A new fuzzy decision fusion method with fuzzy attribution for obstacle detection and classification application is described. The fuzzy decision function modifies parameters with auto-adapted algorithm to get better classification probability. It is shown that the method can achieve better classification result.
 

The technology of vehicle video detecting and tracking has been playing an important role in the ITS (Intelligent Transportation Systems) field during recent years. The occlusion phenomenon among vehicles is one of the most difficult problems related to vehicle tracking. In order to handle occlusion, this paper proposes an effective solution that applied Markov Random Field (MRF) to the traffic images. The contour of the vehicle is firstly detected by using background subtraction, then numbers of blocks with vehicle's texture and motion information are filled inside each vehicle. We extract several kinds of information of each block to process the following tracking. As for each occlusive block two groups of clique functions in MRF model are defined, which represents spatial correlation and motion coherence respectively. By calculating each occlusive block's total energy function, we finally solve the attribution problem of occlusive blocks. The experimental results show that our method can handle occlusion problems effectively and track each vehicle continuously.
 

Detecting stationary crowd groups and analyzing their behaviors have important applications in crowd video surveillance, but have rarely been studied. The contributions of this paper are in two aspects. First, a stationary crowd detection algorithm is proposed to estimate the stationary time of foreground pixels. It employs spatial-temporal filtering and motion filtering in order to be robust to noise caused by occlusions and crowd clutters. Second, in order to characterize the emergence and dispersal processes of stationary crowds and their behaviors during the stationary periods, three attributes are proposed for quantitative analysis. These attributes are recognized with a set of proposed crowd descriptors which extract visual features from the results of stationary crowd detection. The effectiveness of the proposed algorithms is shown through experiments on a benchmark dataset.

Abnormal crowd behavior detection is an important research issue in video processing and computer vision. In this paper we introduce a novel method to detect abnormal crowd behaviors in video surveillance based on interest points. A complex network-based algorithm is used to detect interest points and extract the global texture features in scenarios. The performance of the proposed method is evaluated on publicly available datasets. We present a detailed analysis of the characteristics of the crowd behavior in different density crowd scenes. The analysis of crowd behavior features and simulation results are also demonstrated to illustrate the effectiveness of our proposed method.