Biblio

Human behavior analysis is the process that consists of activity monitoring and behavior recognition and has become the core component of intelligent applications such as security surveillance and fall detection. Generally, the techniques involved in behavior recognition include sensor and vision-based processing. During the process, the activity information is typically required to ensure a good recognition performance. On the other hand, the privacy issue attracts much attention and requires a limited range of activity monitoring accordingly. We study behavior analysis for such privacy-oriented applications. A local object tracking (LOT) technique based on an infrared sensor array is developed in a limited monitoring range and is further realized to a practical bed-exit system in the clinical test environment. The experimental results show a correct recognition rate of 99% for 6 bedside activities. In addition, 89% of participants in a satisfaction survey agree on its effectiveness.

The present study proposes a method for detecting objects with a high recall rate for human-supported video annotation. In recent years, automatic annotation techniques such as object detection and tracking have become more powerful; however, detection and tracking of occluded objects, small objects, and blurred objects are still difficult. In order to annotate such objects, manual annotation is inevitably required. For this reason, we envision a human-supported video annotation framework in which over-detected objects (i.e., false positives) are allowed to minimize oversight (i.e., false negatives) in automatic annotation and then the over-detected objects are removed manually. This study attempts to achieve human-in-the-loop object detection with an emphasis on suppressing the oversight for the former stage of processing in the aforementioned annotation framework: bi-directional deep SORT is proposed to reliably capture missed objects and annotation-free segment identification (AFSID) is proposed to identify video frames in which manual annotation is not required. These methods are reinforced each other, yielding an increase in the detection rate while reducing the burden of human intervention. Experimental comparisons using a pedestrian video dataset demonstrated that bi-directional deep SORT with AFSID was successful in capturing object candidates with a higher recall rate over the existing deep SORT while reducing the cost of manpower compared to manual annotation at regular intervals.

Video analytics has many applications in traffic control, security monitoring, action/event analysis, etc. With the adoption of deep neural networks, the accuracy of video analytics in video streams has been greatly improved. However, deep neural networks for performing video analytics are compute-intensive. In order to reduce processing time, many systems switch to the lower frame rate or resolution. State-of-the-art switching approaches adjust configurations by profiling video clips on a large configuration space. Multiple configurations are tested periodically and the cheapest one with a desired accuracy is adopted. In this paper, we propose a method that adapts the configuration by analyzing past video analytics results instead of profiling candidate configurations. Our method adopts a lower/higher resolution or frame rate when objects move slow/fast. We train a model that automatically selects the best configuration. We evaluate our method with two real-world video analytics applications: traffic tracking and pose estimation. Compared to the periodic profiling method, our method achieves 3%-12% higher accuracy with the same resource cost and 8-17x faster with comparable accuracy.

Supply chain management systems (SCM) are the most intensive and statistical RFID application for object tracking. A lot of research has been carried out to overcome security issues in the field of online/offline object tracking as well as authentication protocols involving RFID technology. Due to advancements with the Internet of Things (IoT) and embedded systems in object tracking schemes the latest research manages to deliver information about the object’s location as well as provide particulars about the state of an object. Recent research presented a proposal for an authentication and online object tracking protocol focusing on solutions for privacy issues for device identification, end-to-end authentication, and secure online object tracking. However, recent schemes have been found to be vulnerable to traceability attacks. This paper presents an enhanced end-to-end authentication scheme where the identity of the user is kept anonymous so that its actions can not be tracked, eliminating attacks related to traceability. The security of the proposed protocol is formally analyzed using the attack model of the automated security testing tool, ProVerif. The proposed scheme outperforms competing schemes based on security.

Monitoring for security and well-being in highly populated areas is a critical issue for city administrators, policy makers and urban planners. As an essential part of many dynamic and critical data-driven tasks, situational awareness (SAW) provides decision-makers a deeper insight of the meaning of urban surveillance. Thus, surveillance measures are increasingly needed. However, traditional surveillance platforms are not scalable when more cameras are added to the network. In this work, a smart surveillance as an edge service has been proposed. To accomplish the object detection, identification, and tracking tasks at the edge-fog layers, two novel lightweight algorithms are proposed for detection and tracking respectively. A prototype has been built to validate the feasibility of the idea, and the test results are very encouraging.

The incidence of abnormal road traffic events, especially abnormal traffic congestion, is becoming more and more prominent in daily traffic management in China. It has become the main research work of urban traffic management to detect and identify traffic congestion incidents in time. Efficient and accurate detection of traffic congestion incidents can provide a good strategy for traffic management. At present, the detection and recognition of traffic congestion events mainly rely on the integration of road traffic flow data and the passing data collected by electronic police or devices of checkpoint, and then estimating and forecasting road conditions through the method of big data analysis; Such methods often have some disadvantages such as low time-effect, low precision and small prediction range. Therefore, with the help of the current large and medium cities in the public security, traffic police have built video surveillance equipment, through computer vision technology to analyze the traffic flow from video monitoring, in this paper, the motion state and the changing trend of vehicle flow are obtained by using the technology of vehicle detection from video and multi-target tracking based on deep learning, so as to realize the perception and recognition of traffic congestion. The method achieves the recognition accuracy of less than 60 seconds in real-time, more than 80% in detection rate of congestion event and more than 82.5% in accuracy of detection. At the same time, it breaks through the restriction of traditional big data prediction, such as traffic flow data, truck pass data and GPS floating car data, and enlarges the scene and scope of detection.

Video Surveillance plays a pivotal role in today's world. The technologies have been advanced too much when artificial intelligence, machine learning and deep learning pitched into the system. Using above combinations, different systems are in place which helps to differentiate various suspicious behaviors from the live tracking of footages. The most unpredictable one is human behaviour and it is very difficult to find whether it is suspicious or normal. Deep learning approach is used to detect suspicious or normal activity in an academic environment, and which sends an alert message to the corresponding authority, in case of predicting a suspicious activity. Monitoring is often performed through consecutive frames which are extracted from the video. The entire framework is divided into two parts. In the first part, the features are computed from video frames and in second part, based on the obtained features classifier predict the class as suspicious or normal.

As the assets of people are growing, security and surveillance have become a matter of great concern today. When a criminal activity takes place, the role of the witness plays a major role in nabbing the criminal. The witness usually states the gender of the criminal, the pattern of the criminal's dress, facial features of the criminal, etc. Based on the identification marks provided by the witness, the criminal is searched for in the surveillance cameras. Surveillance cameras are ubiquitous and finding criminals from a huge volume of surveillance video frames is a tedious process. In order to automate the search process, proposed a novel smart methodology using deep learning. This method takes gender, shirt pattern, and spectacle status as input to find out the object as person from the video log. The performance of this method achieves an accuracy of 87% in identifying the person in the video frame.

We formulate a tracker which performs incessant decision making in order to track objects where the objects may undergo different challenges such as partial occlusions, moving camera, cluttered background etc. In the process, the agent must make a decision on whether to keep track of the object when it is occluded or has moved out of the frame temporarily based on its prediction from the previous location or to reinitialize the tracker based on the belief that the target has been lost. Instead of the heuristic methods we depend on reward and penalty based training that helps the agent reach an optimal solution via this partially observable Markov decision making (POMDP). Furthermore, we employ deeply learned compositional model to estimate human pose in order to better handle occlusion without needing human inputs. By learning compositionality of human bodies via deep neural network the agent can make better decision on presence of human in a frame or lack thereof under occlusion. We adapt skeleton based part representation and do away with the large spatial state requirement. This especially helps in cases where orientation of the target in focus is unorthodox. Finally we demonstrate that the deep reinforcement learning based training coupled with pose estimation capabilities allows us to train and tag multiple large video datasets much quicker than previous works.

Face detection is an application widely used in video surveillance systems and it is the first step for subsequent applications such as monitoring and recognition. For facial detection, there are a series of algorithms that allow the face to be extracted in a video image, among which are the Viola & Jones waterfall method and the method by geometric models using the Hausdorff distance. In this article, both algorithms are theoretically analyzed and the best one is determined by efficiency and resource optimization. Considering the most common problems in the detection of faces in a video surveillance system, such as the conditions of brightness and the angle of rotation of the face, tests have been carried out in 13 different scenarios with the best theoretically analyzed algorithm and its combination with another algorithm The images obtained, using a digital camera in the 13 scenarios, have been analyzed using Matlab code of the Viola & Jones and Viola & Jones algorithm combined with the Kanade-Lucas-Tomasi algorithm to add the feature of completing the tracking of a single object. This paper presents the detection percentages, false positives and false negatives for each image and for each simulation code, resulting in the scenarios with the most detection problems and the most accurate algorithm in face detection.

Now-a-days, video steganography has developed for a secured communication among various users. The two important factor of steganography method are embedding potency and embedding payload. Here, a Multiple Object Tracking (MOT) algorithmic programs used to detect motion object, also shows foreground mask. Discrete wavelet Transform (DWT) and Discrete Cosine Transform (DCT) are used for message embedding and extraction stage. In existing system Least significant bit method was proposed. This technique of hiding data may lose some data after some file transformation. The suggested Multiple object tracking algorithm increases embedding and extraction speed, also protects secret message against various attackers.

Today, as surveillance systems are widely used for indoor and outdoor monitoring applications, there is a growing interest in real-time generation detection and there are many different applications for real-time generation detection and analysis. Two-dimensional videos; It is used in multimedia content-based indexing, information acquisition, visual surveillance and distributed cross-camera surveillance systems, human tracking, traffic monitoring and similar applications. It is of great importance for the development of systems for national security by following a moving target within the scope of military applications. In this research, a more efficient solution is proposed in addition to the existing methods. Therefore, we present YOLO, a new approach to object detection for military applications.

A “Completely Automated Public Turing Test to Tell Computers and Humans Apart” (CAPTCHA) widely used online services so that prevents bots from automatic getting a large of accounts. Interactive video type CAPTCHAs that attempt to detect this attack by using delay time due to communication relays have been proposed. However, these approaches remain insufficiently resistant to bots. We propose a CAPTCHA that combines resistant to automated and relay attacks. In our CAPTCHA, the users recognize a moving object (target object) from among a number of randomly appearing decoy objects and tracks the target with mouse cursor. The users pass the test when they were able to track the target for a certain time. Since the target object moves quickly, the delay makes it difficult for a remote solver to break the CAPTCHA during a relay attack. It is also difficult for a bot to track the target using image processing because it has same looks of the decoys. We evaluated our CAPTCHA's resistance to relay and automated attacks. Our results show that, if our CAPTHCA's parameters are set suitable value, a relay attack cannot be established economically and false acceptance rate with bot could be reduced to 0.01% without affecting human success rate.

We present an object tracking framework which fuses multiple unstable video-based methods and supports automatic tracker initialization and termination. To evaluate our system, we collected a large dataset of hand-annotated 5-minute traffic surveillance videos, which we are releasing to the community. To the best of our knowledge, this is the first publicly available dataset of such long videos, providing a diverse range of real-world object variation, scale change, interaction, different resolutions and illumination conditions. In our comprehensive evaluation using this dataset, we show that our automatic object tracking system often outperforms state-of-the-art trackers, even when these are provided with proper manual initialization. We also demonstrate tracking throughput improvements of 5× or more vs. the competition.

We address the problem of object tracking in an underwater acoustic sensor network in which distributed nodes measure the strength of field generated by moving objects, encode the measurements into digital data packets, and transmit the packets to a fusion center in a random access manner. We allow for imperfect communication links, where information packets may be lost due to noise and collisions. The packets that are received correctly are used to estimate the objects' trajectories by employing an extended Kalman Filter, where provisions are made to accommodate a randomly changing number of obseravtions in each iteration. An adaptive rate control scheme is additionally applied to instruct the sensor nodes on how to adjust their transmission rate so as to improve the location estimation accuracy and the energy efficiency of the system. By focusing explicitly on the objects' locations, rather than working with a pre-specified grid of potential locations, we resolve the spatial quantization issues associated with sparse identification methods. Finally, we extend the method to address the possibility of objects entering and departing the observation area, thus improving the scalability of the system and relaxing the requirement for accurate knowledge of the objects' initial locations. Performance is analyzed in terms of the mean-squared localization error and the trade-offs imposed by the limited communication bandwidth.

Visual object tracking is challenging when the object appearances occur significant changes, such as scale change, background clutter, occlusion, and so on. In this paper, we crop different sizes of multiscale templates around object and input these multiscale templates into network to pretrain the network adaptive the size change of tracking object. Different from previous the tracking method based on deep convolutional neural network (CNN), we exploit deep Residual Network (ResNet) to offline train a multiscale object appearance model on the ImageNet, and then the features from pretrained network are transferred into tracking tasks. Meanwhile, the proposed method combines the multilayer convolutional features, it is robust to disturbance, scale change, and occlusion. In addition, we fuse multiscale search strategy into three kernelized correlation filter, which strengthens the ability of adaptive scale change of object. Unlike the previous methods, we directly learn object appearance change by integrating multiscale templates into the ResNet. We compared our method with other CNN-based or correlation filter tracking methods, the experimental results show that our tracking method is superior to the existing state-of-the-art tracking method on Object Tracking Benchmark (OTB-2015) and Visual Object Tracking Benchmark (VOT-2015).

This paper presents a framework for privacy-preserving video delivery system to fulfill users' privacy demands. The proposed framework leverages the inference channels in sensitive behavior prediction and object tracking in a video surveillance system for the sequence privacy protection. For such a goal, we need to capture different pieces of evidence which are used to infer the identity. The temporal, spatial and context features are extracted from the surveillance video as the observations to perceive the privacy demands and their correlations. Taking advantage of quantifying various evidence and utility, we let users subscribe videos with a viewer-dependent pattern. We implement a prototype system for off-line and on-line requirements in two typical monitoring scenarios to construct extensive experiments. The evaluation results show that our system can efficiently satisfy users' privacy demands while saving over 25% more video information compared to traditional video privacy protection schemes.

Networked systems have adapted Radio Frequency identification technology (RFID) to automate their business process. The Networked RFID Systems (NRS) has some unique characteristics which raise new privacy and security concerns for organizations and their NRS systems. The businesses are always having new realization of business needs using NRS. One of the most recent business realization of NRS implementation on large scale distributed systems (such as Internet of Things (IoT), supply chain) is to ensure visibility and traceability of the object throughout the chain. However, this requires assurance of security and privacy to ensure lawful business operation. In this paper, we are proposing a secure tracker protocol that will ensure not only visibility and traceability of the object but also genuineness of the object and its travel path on-site. The proposed protocol is using Physically Unclonable Function (PUF), Diffie-Hellman algorithm and simple cryptographic primitives to protect privacy of the partners, injection of fake objects, non-repudiation, and unclonability. The tag only performs a simple mathematical computation (such as combination, PUF and division) that makes the proposed protocol suitable to passive tags. To verify our security claims, we performed experiment on Security Protocol Description Language (SPDL) model of the proposed protocol using automated claim verification tool Scyther. Our experiment not only verified our claims but also helped us to eliminate possible attacks identified by Scyther.

Tracking moving objects is a task of the utmost importance to the defence community. As this task requires high accuracy, rather than employing a single detector, it has become common to use multiple ones. In such cases, the tracks produced by these detectors need to be correlated (if they belong to the same sensing modality) or associated (if they were produced by different sensing modalities). In this work, we introduce Computational-Intelligence-based methods for correlating and associating various contacts and tracks pertaining to maritime vessels in an area of interest. Fuzzy k-Nearest Neighbours will be used to conduct track correlation and Fuzzy C-Means clustering will be applied for association. In that way, the uncertainty of the track correlation and association is handled through fuzzy logic. To better model the state of the moving target, the traditional Kalman Filter will be extended using an Echo State Network. Experimental results on five different types of sensing systems will be discussed to justify the choices made in the development of our approach. In particular, we will demonstrate the judiciousness of using Fuzzy k-Nearest Neighbours and Fuzzy C-Means on our tracking system and show how the extension of the traditional Kalman Filter by a recurrent neural network is superior to its extension by other methods.

The technology of vehicle video detecting and tracking has been playing an important role in the ITS (Intelligent Transportation Systems) field during recent years. The occlusion phenomenon among vehicles is one of the most difficult problems related to vehicle tracking. In order to handle occlusion, this paper proposes an effective solution that applied Markov Random Field (MRF) to the traffic images. The contour of the vehicle is firstly detected by using background subtraction, then numbers of blocks with vehicle's texture and motion information are filled inside each vehicle. We extract several kinds of information of each block to process the following tracking. As for each occlusive block two groups of clique functions in MRF model are defined, which represents spatial correlation and motion coherence respectively. By calculating each occlusive block's total energy function, we finally solve the attribution problem of occlusive blocks. The experimental results show that our method can handle occlusion problems effectively and track each vehicle continuously.
 

Multiple-object tracking is an important task in automated video surveillance. In this paper, we present a multiple-human-tracking approach that takes the single-frame human detection results as input and associates them to form trajectories while improving the original detection results by making use of reliable temporal information in a closed-loop manner. It works by first forming tracklets, from which reliable temporal information is extracted, and then refining the detection responses inside the tracklets, which also improves the accuracy of tracklets' quantities. After this, local conservative tracklet association is performed and reliable temporal information is propagated across tracklets so that more detection responses can be refined. The global tracklet association is done last to resolve association ambiguities. Experimental results show that the proposed approach improves both the association and detection results. Comparison with several state-of-the-art approaches demonstrates the effectiveness of the proposed approach.

Many surveillance cameras are using everywhere, the videos or images captured by these cameras are still dumped but they are not processed. Many methods are proposed for tracking and detecting the objects in the videos but we need the meaningful content called semantic content from these videos. Detecting Human activity recognition is quite complex. The proposed method called Semantic Content Extraction (SCE) from videos is used to identify the objects and the events present in the video. This model provides useful methodology for intruder detecting systems which provides the behavior and the activities performed by the intruder. Construction of ontology enhances the spatial and temporal relations between the objects or features extracted. Thus proposed system provides a best way for detecting the intruders, thieves and malpractices happening around us.

This paper presents a human model-based feature extraction method for a video surveillance retrieval system. The proposed method extracts, from a normalized scene, object features such as height, speed, and representative color using a simple human model based on multiple-ellipse. Experimental results show that the proposed system can effectively track moving routes of people such as a missing child, an absconder, and a suspect after events.