Machine Learning & Deep Learning Notes and Records

Intrusion Detection

Group Project - NotPetya analysis

Interesting questions: shut down the SMBv1 could be a good prevent method, but is there any problems when using SMBv2 and SMBv3?
Microsoft network client: Digitally sign communications (always)
Cisco Firepower Release Notes, Version 6.2.3
Cisco Firepower System Software Server Message Block File Policy Bypass Vulnerability
Prevent Petya and other Ransomware attacks by disabling SMBv1
Here’s why you need to stop using SMBv1 immediately. Even Microsoft agrees.
CVE-2018-0243 Detail

Research Paper: Utilizing J48 to conduct classification task of Network Activity Recognition

preparation

Experiment Design

Convert pcap to csv using the Wireshark
Convert csv to txt
Apply Network-intrusion-system-with-multinomial-classification for format fitting
Use wireshark to generate train and validation data and convert it / checked
Implement the existing project
If error, try to fix and explain
Record the key points when implementing
Record the outcomes when running
Evaluate the algorithm with validation dataset
machine learning regular evaluation
Analyze test dataset using tools
My algorithm
other online or existing tools

Conclusion
good points
possible error analysis

1) in the music listenning dataset, if I want to use netease music, it need to set a VPN to China to enable it, this may change the attribute features of dataset and cause final classification bias or mistakes.

where could be done better
possible ways to get better

1) use tshark or python to extract the csv file from pcap file in a more customizable way
2) config the wireshark to monitor only one application when generating test and validation data. [use wireshark to get specific application's internet package](https://blog.csdn.net/youxiansanren/article/details/48271851) / go to "Activity Monitor" of Mac to find the pid of firefox browser 441, and port is 31886 / [get the info of the application](https://blog.csdn.net/qq_24909089/article/details/90667898)
3) use the spotfiy

Convert pcap to csv

download the wireshark from official website / convert pcap to csv / convert csv to txt

python ways in CSDN

pip install scapy

confirm the format of given pcap file

###[ Ethernet ]### 
  dst       = 52:54:00:12:35:02
  src       = 08:00:27:92:ab:0d
  type      = IPv4
###[ IP ]### 
     version   = 4
     ihl       = 5
     tos       = 0x0
     len       = 40
     id        = 45488
     flags     = DF
     frag      = 0
     ttl       = 64
     proto     = tcp
     chksum    = 0x7d53
     src       = 10.0.2.15
     dst       = 34.107.221.82
     \options   \
###[ TCP ]### 
        sport     = 35546
        dport     = http
        seq       = 2415182248
        ack       = 14528222
        dataofs   = 5
        reserved  = 0
        flags     = A
        window    = 64020
        chksum    = 0xbe7
        urgptr    = 0
        options   = []

—–fail! just one row with wrong colums

tshark -r sample1.pcap -T fields -e eth.src -e eth.dst -e ip.src -e ip.dst -e ip.proto -E header=y -E separator=, -E quote=d -E occurrence=f > sample1.csv

Generate training dataset using wireshark
network trouble shotting

 1) ping www.google.com
 2) traceroute www.google.com
 3) ipconfig ifcount
 4) nslookup www.google.com
 5) whois www.google.com
 6) netstat

weka reference
use weka to predict data / complement of the previews one
detailed output file instruction
Prepare the data from raw data.
1. add “Class lable”, remember use ‘microsoft excel’ to add coloum but not the ‘mac numbers’ how to use excel to fill one coloum to one value
2. or directly use weka import csv file and convert it to arff file
  in the ‘info’ attribute, can not contain double quotation marks, which would lead to generation error —– even if choose other method, this would also consequently cause error, so this is a very important tip. replace the string in Excel with empty space-tap a space in the replace textbox of excel but not nothing

Dataset preprocessing
1. delete useless attribute ID, which does not contain any information about the intrusion detection process.

Reference

Use Lubuntu to test programs in VirtulBox

In order to use share function
(1) set up shared files
(2) when restart Lubuntu, check in terminal “df -l”, and use “sudo mount -t vboxsf [MacSharedFileName] /mnt/shareMac”
```
 sudo mount -t vboxsf Shared /mnt/shareMac
```
Install anaconda / Uninstall anaconda

APT - advanced persistent threat

Wikipedia
Refers mainly to hidden and long-term intrusion in computer, which is conducted mainly by professional attackers and aimed at specific targets.
The biggest threat is that it is long-term, which means that people can not be certain when did their computers get infected. And that makes the process of recovery even harder and a bit more unnreliable.

Security Orchestration, Automation, and Response.

official website

Famous virus

A brief history of Computer Viruses - Youtube

Stuxnet wikipedia

Reference

How to share files between host-mac and client-virtualbox-Lubuntu
How to fix ‘virtualbox failed to get display change request’ error / my solution: change the value of video memory of display settings from 16MB to 64MB would make resolution of display become much higher.