Biblio

Traffic in a backbone network has high forwarding rate requirements, and as the network gets larger, traffic increases and forwarding rates decrease. In a Software Defined Network (SDN), the controller can manage a global view of the network and control the forwarding of network traffic. A deterministic network has different forwarding requirements for the traffic of different priority levels. Static traffic load balancing is not flexible enough to meet the needs of users and may lead to the overloading of individual links and even network collapse. In this paper, we propose a new backbone network load balancing architecture - EDQN (Edge Deep Q-learning Network), which implements queue-based gate-shaping algorithms at the edge devices and load balancing of traffic on the backbone links. With the advantages of SDN, the link utilization of the backbone network can be improved, the delay in traffic transmission can be reduced and the throughput of traffic during transmission can be increased.

A new Domain Name System (DNS) cache poisoning attack aiming at clients has emerged recently. It induced cloud users to visit fake web sites and thus reveal information such as account passwords. However, the design of current DNS defense architecture does not formally consider the protection of clients. Although the DNS traffic encryption technology can alleviate this new attack, its deployment is as slow as the new DNS architecture. Thus we propose a lightweight adaptive intelligent defense strategy, which only needs to be deployed on the client without any configuration support of DNS. Firstly, we model the attack and defense process as a static stochastic game with incomplete information under bounded rationality conditions. Secondly, to solve the problem caused by uncertain attack strategies and large quantities of game states, we adopt a deep reinforcement learning (DRL) with guaranteed monotonic improvement. Finally, through the prototype system experiment in Alibaba Cloud, the effectiveness of our method is proved against multiple attack modes with a success rate of 97.5% approximately.

Conversational agents are the most popular AI technology in IT trends. Domain specific chatbots are now used by almost every industry in order to upgrade their customer service. The Proposed paper shows the modelling and performance of one such conversational agent created using deep learning. The proposed model utilizes NMT (Neural Machine Translation) from the TensorFlow software libraries. A BiRNN (Bidirectional Recurrent Neural Network) is used in order to process input sentences that contain large number of tokens (20-40 words). In order to understand the context of the input sentence attention model is used along with BiRNN. The conversational models usually have one drawback, that is, they sometimes provide irrelevant answer to the input. This happens quite often in conversational chatbots as the chatbot doesn't realize that it is answering without context. This drawback is solved in the proposed system using Deep Reinforcement Learning technique. Deep reinforcement Learning follows a reward system that enables the bot to differentiate between right and wrong answers. Deep Reinforcement Learning techniques allows the chatbot to understand the sentiment of the query and reply accordingly. The Deep Reinforcement Learning algorithms used in the proposed system is Q-Learning, Deep Q Neural Network (DQN) and Distributional Reinforcement Learning with Quantile Regression (QR-DQN). The performance of each algorithm is evaluated and compared in this paper in order to find the best DRL algorithm. The dataset used in the proposed system is Cornell Movie-dialogs corpus and CoQA (A Conversational Question Answering Challenge). CoQA is a large dataset that contains data collected from 8000+ conversations in the form of questions and answers. The main goal of the proposed work is to increase the relevancy of the chatbot responses and to increase the perplexity of the conversational chatbot.

Artificial intelligence technology is becoming more active in all areas of our lives day by day. This technology affects our daily life by more developing in areas such as industry 4.0, security and education. Deep reinforcement learning is one of the most developed algorithms in the field of artificial intelligence. In this study, it is aimed that three different robots in a limited area learn to move without hitting each other, fixed obstacles and the boundaries of the field. These robots have been trained using the deep reinforcement learning approach and Proximal policy optimization (PPO) policy. Instead of uses value-based methods with the discrete action space, PPO that can easily manipulate the continuous action field and successfully determine the action of the robots has been proposed. PPO policy achieves successful results in multi-agent problems, especially with the use of the Actor-Critic network. In addition, information is given about environment control and learning approaches for swarm behavior. We propose parameter sharing and behavior-based method for this study. Finally, trained model is recorded and tested in 9 different environments where the obstacles are located differently. With our method, robots can perform their tasks in closed environments in the real world without damaging anyone or anything.

Smart meters (SMs) play a pivotal rule in the smart grid by being able to report the electricity usage of consumers to the utility provider (UP) almost in real-time. However, this could leak sensitive information about the consumers to the UP or a third-party. Recent works have leveraged the availability of energy storage devices, e.g., a rechargeable battery (RB), in order to provide privacy to the consumers with minimal additional energy cost. In this paper, a privacy-cost management unit (PCMU) is proposed based on a model-free deep reinforcement learning algorithm, called deep double Q-learning (DDQL). Empirical results evaluated on actual SMs data are presented to compare DDQL with the state-of-the-art, i.e., classical Q-learning (CQL). Additionally, the performance of the method is investigated for two concrete cases where attackers aim to infer the actual demand load and the occupancy status of dwellings. Finally, an abstract information-theoretic characterization is provided.

The proliferation of artificial intelligence based systems in all walks of life raises concerns about their safety and robustness, especially for cyber-physical systems including multiple machine learning components. In this paper, we introduce robustness contracts as a framework for compositional specification and reasoning about the robustness of cyber-physical systems based on neural network (NN) components. Robustness contracts can encompass and generalize a variety of notions of robustness which were previously proposed in the literature. They can seamlessly apply to NN-based perception as well as deep reinforcement learning (RL)-enabled control applications. We present a sound and complete algorithm that can efficiently verify the satisfaction of a class of robustness contracts on NNs by leveraging notions from Lagrangian duality to identify system configurations that violate the contracts. We illustrate the effectiveness of our approach on the verification of NN-based perception systems and deep RL-based control systems.

In modern grid systems which is a typical cyber-physical System (CPS), information space and physical space are closely related. Once the communication link is interrupted, it will make a great damage to the power system. If the service path is too concentrated, the risk will be greatly increased. In order to solve this problem, this paper constructs a route planning algorithm that combines node load pressure, link load balance and service delay risk. At present, the existing intelligent algorithms are easy to fall into the local optimal value, so we chooses the deep reinforcement learning algorithm (DRL). Firstly, we build a risk assessment model. The node risk assessment index is established by using the node load pressure, and then the link risk assessment index is established by using the average service communication delay and link balance degree. The route planning problem is then solved by a route planning algorithm based on DRL. Finally, experiments are carried out in a simulation scenario of a power grid system. The results show that our method can find a lower risk path than the original Dijkstra algorithm and the Constraint-Dijkstra algorithm.

Software Defined Networking (SDN) provides opportunities for flexible and dynamic traffic engineering. However, in current SDN systems, routing strategies are based on traditional mechanisms which lack in real-time modification and less efficient resource utilization. To overcome these limitations, deep learning is used in this paper to improve the routing computation in SDN. This paper proposes Convolutional Deep Reinforcement Learning (CoDRL) model which is based on deep reinforcement learning agent for routing optimization in SDN to minimize the mean network delay and packet loss rate. The CoDRL model consists of Deep Deterministic Policy Gradients (DDPG) deep agent coupled with Convolution layer. The proposed model tends to automatically adapts the dynamic packet routing using network data obtained through the SDN controller, and provides the routing configuration that attempts to reduce network congestion and minimize the mean network delay. Hence, the proposed deep agent exhibits good convergence towards providing routing configurations that improves the network performance.

Network dynamics, such as bandwidth fluctuation and unexpected latency, hurt users' quality of experience (QoE) greatly for media services over the Internet. In this work, we propose a neural adaptive transport (NAT) framework to tackle the network dynamics for Internet-scale interactive media services. The entire NAT system has three major components: a learning based cloud overlay routing (COR) scheme for the best delivery path to bypass the network bottlenecks while offering the minimal end-to-end latency simultaneously; a residual neural network based collaborative video processing (CVP) system to trade the computational capability at client-end for QoE improvement via learned resolution scaling; and a deep reinforcement learning (DRL) based adaptive real-time streaming (ARS) strategy to select the appropriate video bitrate for maximal QoE. We have demonstrated that COR could improve the user satisfaction from 5% to 43%, CVP could reduce the bandwidth consumption more than 30% at the same quality, and DRL-based ARS can maintain the smooth streaming with \textbackslashtextless; 50% QoE improvement, respectively.

This paper investigates the use of deep reinforcement learning (DRL) in the design of a "universal" MAC protocol referred to as Deep-reinforcement Learning Multiple Access (DLMA). The design framework is partially inspired by the vision of DARPA SC2, a 3-year competition whereby competitors are to come up with a clean-slate design that "best share spectrum with any network(s), in any environment, without prior knowledge, leveraging on machine-learning technique". While the scope of DARPA SC2 is broad and involves the redesign of PHY, MAC, and Network layers, this paper's focus is narrower and only involves the MAC design. In particular, we consider the problem of sharing time slots among a multiple of time-slotted networks that adopt different MAC protocols. One of the MAC protocols is DLMA. The other two are TDMA and ALOHA. The DRL agents of DLMA do not know that the other two MAC protocols are TDMA and ALOHA. Yet, by a series of observations of the environment, its own actions, and the rewards - in accordance with the DRL algorithmic framework - a DRL agent can learn the optimal MAC strategy for harmonious co-existence with TDMA and ALOHA nodes. In particular, the use of neural networks in DRL (as opposed to traditional reinforcement learning) allows for fast convergence to optimal solutions and robustness against perturbation in hyper- parameter settings, two essential properties for practical deployment of DLMA in real wireless networks.

Person re-identification is an important task in video surveillance, focusing on finding the same person across different cameras. However, most existing methods of video-based person re-identification still have some limitations (e.g., the lack of effective deep learning framework, the robustness of the model, and the same treatment for all video frames) which make them unable to achieve better recognition performance. In this paper, we propose a novel self-paced learning algorithm for video-based person re-identification, which could gradually learn from simple to complex samples for a mature and stable model. Self-paced learning is employed to enhance video-based person re-identification based on deep neural network, so that deep neural network and self-paced learning are unified into one frame. Then, based on the trained self-paced learning, we propose to employ deep reinforcement learning to discard misleading and confounding frames and find the most representative frames from video pairs. With the advantage of deep reinforcement learning, our method can learn strategies to select the optimal frame groups. Experiments show that the proposed framework outperforms the existing methods on the iLIDS-VID, PRID-2011 and MARS datasets.

As a new research hotspot in the field of artificial intelligence, deep reinforcement learning (DRL) has achieved certain success in various fields such as robot control, computer vision, natural language processing and so on. At the same time, the possibility of its application being attacked and whether it have a strong resistance to strike has also become a hot topic in recent years. Therefore, we select the representative Deep Q Network (DQN) algorithm in deep reinforcement learning, and use the robotic automatic pathfinding application as a countermeasure application scenario for the first time, and attack DQN algorithm against the vulnerability of the adversarial samples. In this paper, we first use DQN to find the optimal path, and analyze the rules of DQN pathfinding. Then, we propose a method that can effectively find vulnerable points towards White-Box Q table variation in DQN pathfinding training. Finally, we build a simulation environment as a basic experimental platform to test our method, through multiple experiments, we can successfully find the adversarial examples and the experimental results show that the supervised method we proposed is effective.