Biblio

According to the characteristics of security threats and massive users in power mobile applications, a mobile application security situational awareness method based on big data architecture is proposed. The method uses open-source big data technology frameworks such as Kafka, Flink, Elasticsearch, etc. to complete the collection, analysis, storage and visual display of massive power mobile application data, and improve the throughput of data processing. The security situation awareness method of power mobile application takes the mobile terminal threat index as the core, divides the risk level for the mobile terminal, and predicts the terminal threat index through support vector machine regression algorithm (SVR), so as to construct the security profile of the mobile application operation terminal. Finally, through visualization services, various data such as power mobile applications and terminal assets, security operation statistics, security strategies, and alarm analysis are displayed to guide security operation and maintenance personnel to carry out power mobile application security monitoring and early warning, banning disposal and traceability analysis and other decision-making work. The experimental analysis results show that the method can meet the requirements of security situation awareness for threat assessment accuracy and response speed, and the related results have been well applied in a power company.

Online-activity-generated digital traces provide opportunities for novel services and unique insights as demonstrated in, for example, research on mining software repositories. The inability to link these traces within and among systems, such as Twitter, GitHub, or Reddit, inhibit the advances in this area. Furthermore, no single approach to integrate data from these disparate sources is likely to work. We aim to design Foreseer, an extensible framework, to design and evaluate identity matching techniques for public, large, and low-accuracy operational data. Foreseer consists of three functionally independent components designed to address the issues of discovery and preparation, storage and representation, and analysis and linking of traces from disparate online sources. The framework includes a domain specific language for manipulating traces, generating insights, and building novel services. We have applied it in a pilot study of roughly 10TB of data from Twitter, Reddit, and StackExchange including roughly 6M distinct entities and, using basic matching techniques, found roughly 83,000 matches among these sources. We plan to add additional entity extraction and identification algorithms, data from other sources, and design tools for facilitating dynamic ingestion and tagging of incoming data on a more robust infrastructure using Apache Spark or another distributed processing framework. We will then evaluate the utility and effectiveness of the framework in applications ranging from identifying malicious contributors in software repositories to the evaluation of the utility of privacy preservation schemes.

Network traffic is a rich source of information for security monitoring. However the increasing volume of data to treat raises issues, rendering holistic analysis of network traffic difficult. In this paper we propose a solution to cope with the tremendous amount of data to analyse for security monitoring perspectives. We introduce an architecture dedicated to security monitoring of local enterprise networks. The application domain of such a system is mainly network intrusion detection and prevention, but can be used as well for forensic analysis. This architecture integrates two systems, one dedicated to scalable distributed data storage and management and the other dedicated to data exploitation. DNS data, NetFlow records, HTTP traffic and honeypot data are mined and correlated in a distributed system that leverages state of the art big data solution. Data correlation schemes are proposed and their performance are evaluated against several well-known big data framework including Hadoop and Spark.