Biblio

Deep learning is the segment of artificial intelligence which is involved with imitating the learning approach that human beings utilize to get some different types of knowledge. Analyzing videos, a part of deep learning is one of the most basic problems of computer vision and multi-media content analysis for at least 20 years. The job is very challenging as the video contains a lot of information with large differences and difficulties. Human supervision is still required in all surveillance systems. New advancement in computer vision which are observed as an important trend in video surveillance leads to dramatic efficiency gains. We propose a CCTV based theft detection along with tracking of thieves. We use image processing to detect theft and motion of thieves in CCTV footage, without the use of sensors. This system concentrates on object detection. The security personnel can be notified about the suspicious individual committing burglary using Real-time analysis of the movement of any human from CCTV footage and thus gives a chance to avert the same.

Deep learning has undergone tremendous advancements in computer vision studies. The training of deep learning neural networks depends on a considerable amount of ground truth datasets. However, labeling ground truth data is a labor-intensive task, particularly for large-volume video analytics applications such as video surveillance and vehicles detection for autonomous driving. This paper presents a rapid and accurate method for associative searching in big image data obtained from security monitoring systems. We developed a semi-automatic moving object annotation method for improving deep learning models. The proposed method comprises three stages, namely automatic foreground object extraction, object annotation in subsequent video frames, and dataset construction using human-in-the-loop quick selection. Furthermore, the proposed method expedites dataset collection and ground truth annotation processes. In contrast to data augmentation and data generative models, the proposed method produces a large amount of real data, which may facilitate training results and avoid adverse effects engendered by artifactual data. We applied the constructed annotation dataset to train a deep learning you-only-look-once (YOLO) model to perform vehicle detection on street intersection surveillance videos. Experimental results demonstrated that the accurate detection performance was improved from a mean average precision (mAP) of 83.99 to 88.03.

Surveillance cameras, which is a form of Cyber Physical System, are deployed extensively to provide visual surveillance monitoring of activities of interest or anomalies. However, these cameras are at risks of physical security attacks against their physical attributes or configuration like tampering of their recording coverage, camera positions or recording configurations like focus and zoom factors. Such adversarial alteration of physical configuration could also be invoked through cyber security attacks against the camera's software vulnerabilities to administratively change the camera's physical configuration settings. When such Cyber Physical attacks occur, they affect the integrity of the targeted cameras that would in turn render these cameras ineffective in fulfilling the intended security functions. There is a significant measure of research work in detection mechanisms of cyber-attacks against these Cyber Physical devices, however it is understudied area with such mechanisms against integrity attacks on physical configuration. This research proposes the use of the novel use of deep learning algorithms to detect such physical attacks originating from cyber or physical spaces. Additionally, we proposed the novel use of deep learning-based video frame interpolation for such detection that has comparatively better performance to other anomaly detectors in spatiotemporal environments.

Safety is one of basic human needs so we need a security system that able to prevent crime happens. Commonly, we use surveillance video to watch environment and human behaviour in a location. However, the surveillance video can only used to record images or videos with no additional information. Therefore we need more advanced camera to get another additional information such as human position and movement. This research were able to extract those information from surveillance video footage by using human detection and tracking algorithm. The human detection framework is based on Deep Learning Convolutional Neural Networks which is a very popular branch of artificial intelligence. For tracking algorithms, channel and spatial correlation filter is used to track detected human. This system will generate and export tracked movement on footage as an additional information. This tracked movement can be analysed furthermore for another research on surveillance video problems.

In the past few years, visual information collection and transmission is increased significantly for various applications. Smart vehicles, service robotic platforms and surveillance cameras for the smart city applications are collecting a large amount of visual data. The preservation of the privacy of people presented in this data is an important factor in storage, processing, sharing and transmission of visual data across the Internet of Robotic Things (IoRT). In this paper, a novel anonymisation method for information security and privacy preservation in visual data in sharing layer of the Web of Robotic Things (WoRT) is proposed. The proposed framework uses deep neural network based semantic segmentation to preserve the privacy in video data base of the access level of the applications and users. The data is anonymised to the applications with lower level access but the applications with higher legal access level can analyze and annotated the complete data. The experimental results show that the proposed method while giving the required access to the authorities for legal applications of smart city surveillance, is capable of preserving the privacy of the people presented in the data.

Railroad trespassing is a dangerous activity with significant security and safety risks. However, regular patrolling of potential trespassing sites is infeasible due to exceedingly high resource demands and personnel costs. This raises the need to design automated trespass detection and early warning prediction techniques leveraging state-of-the-art machine learning. To meet this need, we propose a novel framework for Automated Railroad Trespassing detection System using video surveillance data called ARTS. As the core of our solution, we adopt a CNN-based deep learning architecture capable of video processing. However, these deep learning-based methods, while effective, are known to be computationally expensive and time consuming, especially when applied to a large volume of surveillance data. Leveraging the sparsity of railroad trespassing activity, ARTS corresponds to a dual-stage deep learning architecture composed of an inexpensive pre-filtering stage for activity detection, followed by a high fidelity trespass classification stage employing deep neural network. The resulting dual-stage ARTS architecture represents a flexible solution capable of trading-off accuracy with computational time. We demonstrate the efficacy of our approach on public domain surveillance data achieving 0.87 f1 score while keeping up with the enormous video volume, achieving a practical time and accuracy trade-off.

Border areas are generally considered as places where great deal of violence, intrusion and cohesion between several parties happens. This often led to danger for the life of employees, soldiers and common man working or living in border areas. Further geographical conditions like mountains, snow, forest, deserts, harsh weather and water bodies often lead to difficult access and monitoring of border areas. Proposed system uses thermal imaging camera (FLIR) for detection of various objects and infiltrators. FLIR is assigned an IP address and connected through local network to the control center. Software code captures video and subsequently the intrusion detection. A motor controlled spotlight with infrared and laser gun is used to illuminate under various conditions at the site. System also integrates sound sensor to detect specific sounds and motion sensors to sense suspicious movements. Based on the decision, a buzzer and electric current through fence for further protection can be initiated. Sensors are be integrated through IoT for an efficient control of large border area and connectivity between sites.

Convolutional Neural Networks (CNN) have shown astonishing results in the field of computer vision. This paper proposes a foreground segmentation algorithm based on CNN to tackle the practical challenges in the video surveillance system such as illumination changes, dynamic backgrounds, camouflage, and static foreground object, etc. The network is trained using the input of image sequences with respective ground-truth. The algorithm employs a CNN called VGG-16 to extract features from the input. The extracted feature maps are upsampled using a bilinear interpolation. The upsampled feature mask is passed through a sigmoid function and threshold to get the foreground mask. Binary cross entropy is used as the error function to compare the constructed foreground mask with the ground truth. The proposed algorithm was tested on two standard datasets and showed superior performance as compared to the top-ranked foreground segmentation methods.

In the field of video surveillance, the demands of intelligent video analysis services based on Deep Neural Networks (DNNs) have grown rapidly. Although most existing studies focus on the performance of DNNs pre-deployed at remote clouds, the network delay caused by computation offloading from network cameras to remote clouds is usually long and sometimes unbearable. Edge computing can enable rich services and applications in close proximity to the network cameras. However, owing to the limited computing resources of distributed edge clouds, it is challenging to satisfy low latency and high accuracy requirements for all users, especially when the number of users surges. To address this challenge, we first formulate the intelligent video surveillance task scheduling problem that minimizes the average response time while meeting the performance requirements of tasks and prove that it is NP-hard. Second, we present an adaptive DNN model selection method to identify the most effective DNN model for each task by comparing the feature similarity between the input video segment and pre-stored training videos. Third, we propose a two-stage delay-aware graph searching approach that presents a beneficial trade-off between network delay and computing delay. Experimental results demonstrate the efficiency of our approach.

Currently, video surveillance systems play an important role in ensuring the safety of citizens, their property, etc., which greatly contributes to the reduction of crime. Due to the high intrinsic value and/or high efficiency of their use for the prevention and detection of crimes, they themselves often become the objects of illegal actions (theft, damage). The main purpose of video surveillance systems is to provide continuous visual monitoring of the situation at a particular facility or territory, as well as event registration. The breakdown of the camera is detected by the loss of signal in the control center. However, the absence of a signal for reasons other than these can also be caused by an accident on the power line, a communication channel break, software or hardware breakdown of the camera itself. In this regard, there is a problem of determining the exact cause of the lack of signal and, consequently, the need for a rapid response to it. The paper proposes an approach of video surveillance arrangement according to their main functional purpose, as well as their ability to monitor each other. Based on this approach, a mathematical model of the choice of locations and conditions of location of video surveillance equipment from a set of potentially acceptable as a problem of nonlinear Boolean programming is developed. This model maximizes the functionality of the video surveillance system, taking into account the importance of areas and objects of surveillance with restrictions on the number of video surveillance of each type, the nature of the terrain and existing buildings. An algorithm for solving this problem is proposed.

Face detection is an application widely used in video surveillance systems and it is the first step for subsequent applications such as monitoring and recognition. For facial detection, there are a series of algorithms that allow the face to be extracted in a video image, among which are the Viola & Jones waterfall method and the method by geometric models using the Hausdorff distance. In this article, both algorithms are theoretically analyzed and the best one is determined by efficiency and resource optimization. Considering the most common problems in the detection of faces in a video surveillance system, such as the conditions of brightness and the angle of rotation of the face, tests have been carried out in 13 different scenarios with the best theoretically analyzed algorithm and its combination with another algorithm The images obtained, using a digital camera in the 13 scenarios, have been analyzed using Matlab code of the Viola & Jones and Viola & Jones algorithm combined with the Kanade-Lucas-Tomasi algorithm to add the feature of completing the tracking of a single object. This paper presents the detection percentages, false positives and false negatives for each image and for each simulation code, resulting in the scenarios with the most detection problems and the most accurate algorithm in face detection.

Every public or private area today is preferred to be under surveillance to ensure high levels of security. Since the surveillance happens round the clock, data gathered as a result is huge and requires a lot of manual work to go through every second of the recorded videos. This paper presents a system which can detect anomalous behaviors and alarm the user on the type of anomalous behavior. Since there are a myriad of anomalies, the classification of anomalies had to be narrowed down. There are certain anomalies which are generally seen and have a huge impact on public safety, such as explosions, road accidents, assault, shooting, etc. To narrow down the variations, this system can detect explosion, road accidents, shooting, and fighting and even output the frame of their occurrence. The model has been trained with videos belonging to these classes. The dataset used is UCF Crime dataset. Learning patterns from videos requires the learning of both spatial and temporal features. Convolutional Neural Networks (CNN) extract spatial features and Long Short-Term Memory (LSTM) networks learn the sequences. The classification, using an CNN-LSTM model achieves an accuracy of 85%.

In this paper, we propose a video surveillance system based on blockchain system. The proposed system consists of a blockchain network with trusted internal managers. The metadata of the video is recorded on the distributed ledger of the blockchain, thereby blocking the possibility of forgery of the data. The proposed architecture encrypts and stores the video, creates a license within the blockchain, and exports the video. Since the decryption key for the video is managed by the private DB of the blockchain, it is not leaked by the internal manager unauthorizedly. In addition, the internal administrator can manage and export videos safely by exporting the license generated in the blockchain to the DRM-applied video player.

The usage of small drones/UAVs has significantly increased recently. Consequently, there is a rising potential of small drones being misused for illegal activities such as terrorism, smuggling of drugs, etc. posing high-security risks. Hence, tracking and surveillance of drones are essential to prevent security breaches. The similarity in the appearance of small drone and birds in complex background makes it challenging to detect drones in surveillance videos. This paper addresses the challenge of detecting small drones in surveillance videos using popular and advanced deep learning-based object detection methods. Different CNN-based architectures such as ResNet-101 and Inception with Faster-RCNN, as well as Single Shot Detector (SSD) model was used for experiments. Due to sparse data available for experiments, pre-trained models were used while training the CNNs using transfer learning. Best results were obtained from experiments using Faster-RCNN with the base architecture of ResNet-101. Experimental analysis on different CNN architectures is presented in the paper, along with the visual analysis of the test dataset.

Video surveillance system is an important application system on the ubiquitous SG-eIoT. A comparative analysis of the traditional video surveillance scheme and the unified video surveillance solution in the eIoT environment is made. Network load and service latency parameters under the two schemes are theoretically modeled and simulated. Combined with the simulation results, the corresponding suggestions for the access of video terminals in the ubiquitous eIoT are given.

In view of the challenges with Aids to Navigation (AtoN) managements and emergency response, the present study designs and presents an AtoN real-time video surveillance system based on the AIS collision warning. The key technologies regarding with AtoN cradle head control and testing algorithms, video image fusion, system operation and implementation are demonstrated in details. Case study is performed at Guan River (China) to verify the effectiveness of the AtoN real-time video surveillance system for maritime security supervision. The research results indicate that the intellective level of the AtoN maintenance and managements could be significantly improved. The idea of designing modules brings a good flexibility and a high portability for the present surveillance system, therefore provides a guidance for the design of similar maritime surveillance systems.

Recently, Pan/Tilt/Zoom (PTZ) camera has been widely used in video surveillance systems. However, it is difficult to automatically control PTZ cameras according to moving objects in the surveillance area. This paper proposes an automatic camera control method based on a Deep-Q Network (DQN) for improving the recognition accuracy of anomaly actions in the video surveillance system. To generate PTZ camera control values, the proposed method uses the position and size information of the object which received from the video analysis system. Through implementation results, the proposed method can automatically control the PTZ camera according to moving objects.

Today, as surveillance systems are widely used for indoor and outdoor monitoring applications, there is a growing interest in real-time generation detection and there are many different applications for real-time generation detection and analysis. Two-dimensional videos; It is used in multimedia content-based indexing, information acquisition, visual surveillance and distributed cross-camera surveillance systems, human tracking, traffic monitoring and similar applications. It is of great importance for the development of systems for national security by following a moving target within the scope of military applications. In this research, a more efficient solution is proposed in addition to the existing methods. Therefore, we present YOLO, a new approach to object detection for military applications.

Advancements in semiconductor domain gave way to realize numerous applications in Video Surveillance using Computer vision and Deep learning, Video Surveillances in Industrial automation, Security, ADAS, Live traffic analysis etc. through image understanding improves efficiency. Image understanding requires input data with high precision which is dependent on Image resolution and location of camera. The data of interest can be thermal image or live feed coming for various sensors. Composite(CVBS) is a popular video interface capable of streaming upto HD(1920x1080) quality. Unlike high speed serial interfaces like HDMI/MIPI CSI, Analog composite video interface is a single wire standard supporting longer distances. Image understanding requires edge detection and classification for further processing. Sobel filter is one the most used edge detection filter which can be embedded into live stream. This paper proposes Zynq FPGA based system design for video surveillance with Sobel edge detection, where the input Composite video decoded (Analog CVBS input to YCbCr digital output), processed in HW and streamed to HDMI display simultaneously storing in SD memory for later processing. The HW design is scalable for resolutions from VGA to Full HD for 60fps and 4K for 24fps. The system is built on Xilinx ZC702 platform and TVP5146 to showcase the functional path.

With the rapid development of information technology, video surveillance system has become a key part in the security and protection system of modern cities. Especially in prisons, surveillance cameras could be found almost everywhere. However, with the continuous expansion of the surveillance network, surveillance cameras not only bring convenience, but also produce a massive amount of monitoring data, which poses huge challenges to storage, analytics and retrieval. The smart monitoring system equipped with intelligent video analytics technology can monitor as well as pre-alarm abnormal events or behaviours, which is a hot research direction in the field of surveillance. This paper combines deep learning methods, using the state-of-the-art framework for instance segmentation, called Mask R-CNN, to train the fine-tuning network on our datasets, which can efficiently detect objects in a video image while simultaneously generating a high-quality segmentation mask for each instance. The experiment show that our network is simple to train and easy to generalize to other datasets, and the mask average precision is nearly up to 98.5% on our own datasets.

Devices sharing a network compete for bandwidth, being able to transmit only a limited amount of data. This is for example the case with a network of cameras, that should record and transmit video streams to a monitor node for video surveillance. Adaptive cameras can reduce the quality of their video, thereby increasing the frame compression, to limit network congestion. In this paper, we exploit our experience with computing capacity allocation to design and implement a network bandwidth allocation strategy based on game theory, that accommodates multiple adaptive streams with convergence guarantees. We conduct some experiments with our implementation and discuss the results, together with some conclusions and future challenges.

Person re-identification is an important task in video surveillance, focusing on finding the same person across different cameras. However, most existing methods of video-based person re-identification still have some limitations (e.g., the lack of effective deep learning framework, the robustness of the model, and the same treatment for all video frames) which make them unable to achieve better recognition performance. In this paper, we propose a novel self-paced learning algorithm for video-based person re-identification, which could gradually learn from simple to complex samples for a mature and stable model. Self-paced learning is employed to enhance video-based person re-identification based on deep neural network, so that deep neural network and self-paced learning are unified into one frame. Then, based on the trained self-paced learning, we propose to employ deep reinforcement learning to discard misleading and confounding frames and find the most representative frames from video pairs. With the advantage of deep reinforcement learning, our method can learn strategies to select the optimal frame groups. Experiments show that the proposed framework outperforms the existing methods on the iLIDS-VID, PRID-2011 and MARS datasets.

Abnormal event detection in video surveillance is a valuable but challenging problem. Most methods adopt a supervised setting that requires collecting videos with only normal events for training. However, very few attempts are made under unsupervised setting that detects abnormality without priorly knowing normal events. Existing unsupervised methods detect drastic local changes as abnormality, which overlooks the global spatio-temporal context. This paper proposes a novel unsupervised approach, which not only avoids manually specifying normality for training as supervised methods do, but also takes the whole spatio-temporal context into consideration. Our approach consists of two stages: First, normality estimation stage trains an autoencoder and estimates the normal events globally from the entire unlabeled videos by a self-adaptive reconstruction loss thresholding scheme. Second, normality modeling stage feeds the estimated normal events from the previous stage into one-class support vector machine to build a refined normality model, which can further exclude abnormal events and enhance abnormality detection performance. Experiments on various benchmark datasets reveal that our method is not only able to outperform existing unsupervised methods by a large margin (up to 14.2% AUC gain), but also favorably yields comparable or even superior performance to state-of-the-art supervised methods.

Crowd surveillance will play a fundamental role in the coming generation of video surveillance systems, in particular for improving public safety and security. However, traditional camera networks are mostly not able to closely survey the entire monitoring area due to limitations in coverage, resolution and analytics performance. A smart camera network, on the other hand, offers the ability to reconfigure the sensing infrastructure by incorporating active devices such as pan-tilt-zoom (PTZ) cameras and UAV-based cameras, which enable the adaptation of coverage and target resolution over time. This paper proposes a novel decentralized approach for dynamic network reconfiguration, where cameras locally control their PTZ parameters and position, to optimally cover the entire scene. For crowded scenes, cameras must deal with a trade-off among global coverage and target resolution to effectively perform crowd analysis. We evaluate our approach in a simulated environment surveyed with fixed, PTZ, and UAV-based cameras.

Loitering is a suspicious behavior that often leads to criminal actions, such as pickpocketing and illegal entry. Tracking methods can determine suspicious behavior based on trajectory, but require continuous appearance and are difficult to scale up to multi-camera systems. Using the duration of appearance of features works on multiple cameras, but does not consider major aspects of loitering behavior, such as repeated appearance and trajectory of candidates. We introduce an entropy model that maps the location of a person's features on a heatmap. It can be used as an abstraction of trajectory tracking across multiple surveillance cameras. We evaluate our method over several datasets and compare it to other loitering detection methods. The results show that our approach has similar results to state of the art, but can provide additional interesting candidates.