20 Oct Please read and follow the lab and answer all these questions (Highlighted in yellow). Section 1- Using Google Search Operators: Review the search results. Take a screenshot of one of th

Lab-4: Reconnaissance and Information Gathering 

A hacker uses many tools and methods to gather information about the target. There are two broad categories of information gathering methods: passive and active. These methods are detailed in the table below. In this lab, you will perform passive information gathering (gray-shaded column). In Lab 5, you will be performing active information gathering. Please review the table before starting this lab.

Information Gathering	Passive (Reconnaissance and Information Gathering) – This Week	Active (Scanning and Enumeration) – Next Week
Is the hacker contact with the target directly?	No direct contact with the target	Direct contact with the target
Are the activities logged?	No audit records on the target	Audit record might be created
What kind of tools has been used?	Web archives, Whois service, DNS servers, Search Engines	Port scanners, network scanners, vulnerability scanners (Nessus, Nmap)
What information can a hacker collect?	IP addresses, network range, telephone numbers, E-mail addresses, active machines, operating system version, network topology	Live hosts on a network, network topology, OS version, open ports on hosts, services running on hosts, running applications and their versions, patching level, vulnerabilities.

In passive information gathering, the hacker does not directly contact the target; therefore, no audit logs have been created. Both non-technical (such as employee names, birth dates, e-mail addresses) and technical information (IP addresses, domain names) can be gathered. This information can be used in many ways in the subsequent steps of the attack. For example, the phone numbers or e-mail addresses you discovered can be used in social engineering attacks. DNS records or subdomain names can be used to leverage specific attacks against hosts or URLs.

More notes on Reconnaissance and Information Gathering :

1) In this phase, an attacker may collect a lot of information without being noticed.

2) In some cases, an attacker may even discover vulnerabilities.

3) The information collected in this phase can be quite valuable when evaluated together with the information collected in the scanning and enumeration phase. For example, you might find the phone number and name of an employee in this phase, and you may find the computer IP address in the active scanning phase. You can use these two pieces of information together to leverage a social engineering attack. An attacker will increase the chance of gaining trust when s/he calls the victim's name and talk some specific about the victim's computer.

4) Companies should also perform reconnaissance and information gathering against themselves so that they can discover -before hackers- what kind of information the company and company employees disclose.

In this lab, you will practice 6 passive methods of Reconnaissance and Information Gathering. You have to use Kali VM in Sections 3, 5, and 6 of the lab. You may use Kali VM or your computer (the only thing you need will be an Internet browser) for the rest of the sections.

Section-1: Using Google Search Operators

Google search engine has many search operators that help us collect specific results about a website, eventually a company. You will use some of these operators in this lab.

1) Search the term site:franklin.edu

Notice that all results are specific to Franklin Univesity pages.

Review the search results and find three subdomains among the results. What is a subdomain: the word "library" is a subdomain of the franklin.edu domain, as https://library.franklin.edu is an active website.

2) Search the term site:amazonaws.com

This search result in millions of websites or documents that have been hosted on Amazon cloud servers.

Review the search results and find some company websites that have been hosted in the Amazon cloud. (AWS).

3) Search the term Franklin Univesity site:amazonaws.com

This search query will reveal documents and web pages having "Franklin University" keywords and hosted in Amazon AWS.

4) Search the term filetype:doc site:franklin.edu

The filetype operator produces the search results linked to the indexed files with the type indicated in the operator. Above search will reveal word files hosted in the franklin.edu domain and indexed by Google.

If the above query does not produce any results, try another query by changing the file types such as docx, ppt, pptx, pdf.

Download one file and check the metadata information. Find information such as username, author name, application version, etc. There are various methods to see the metadata information. You can right-click the file and check the details tab, as shown below. Alternatively, you can open the file with Microsoft Word and see the properties within the Word program. Use the Google search engine, if you need, to learn how to see metadata information of Microsoft office documents/PDF files.

Note: Every small piece of information is important for a meticulous pentester. A username such as john.smith can indicate that the username pattern used in the organization is name.surname. It is important to know the username patterns in the social engineering attacks. Some metadata might contain Office software and operating system version information. Version information provides information about the patch level, and a hacker/pentester can create malicious payloads specific to the versions found in metadata.

5) Search the term inurl:login site:franklin.edu

inurl operator here finds the pages that contain the "login" in the URL. By using this operator, we can discover login forms hosted by the targeted website.

Review the search results. Take a screenshot of one of the login forms.

6) Visit this page: https://www.exploit-db.com/google-hacking-database

QUESTION: Choose a query, start a Google search and analyze the search results.

Note: “The Exploit Database is a CVE compliant[footnoteRef:1] archive of public exploits and corresponding vulnerable software, developed for use by penetration testers and vulnerability researchers. Our aim is to serve the most comprehensive collection of exploits gathered through direct submissions, mailing lists, as well as other public sources, and present them in a freely-available and easy-to-navigate database. The Exploit Database is a repository for exploits and proof-of-concepts rather than advisories, making it a valuable resource for those who need actionable data right away [1: https://cve.mitre.org/data/refs/refmap/source-EXPLOIT-DB.html]

The Google Hacking Database (GHDB) is a categorized index of Internet search engine queries designed to uncover interesting and usually sensitive information made publicly available on the Internet. In most cases, this information was never meant to be made public but due to any number of factors this information was linked in a web document that was crawled by a search engine that subsequently followed that link and indexed the sensitive information." (Source of this double-quoted information: exploit-db.com about page)

Note that Google hacking (Google Dorking) is a broad topic; there are books written on this topic.

Section-2: Using archive.org

1) The Wayback Machine is an essential part of the Internet Archive project (archive.org). It is a digital archive of the World Wide Web, allows the user to go “back in time” and see what websites looked like in the past. ( https://en.wikipedia.org/wiki/Wayback_Machine)

The Wayback Machine provides useful information for the pen-testers and hackers as well.

1) Go to archive.org website

2) Type in franklin.edu to the Wayback Machine

3) See that the Wayback Machine has been archiving franklin.edu since December 23, 1996. You can check how the first webpage of Franklin was.

2) Assume that you are performing a penetration testing for Franklin University. You were checking an internal portal website. You found a link in one of the message forums.

a) This is the link you found. Click on this or type in the address:

http://www.franklin.edu/univinfo/univinfo.html

You will come up with a 404 error saying, "Sorry, this page does not exist."

As the pentester, you are curious. You wonder what information was published on this page, and that information might be useful for your pentest. As the URL contains “info”, this might be something important.

b) Type in this URL http://www.franklin.edu/univinfo/univinfo.html to the Wayback Machine and see when the webpage was archived.

As you can see, the last snapshots of this page were taken in 2002. As a pentester, you may continue your search and find some useful information, or you can discard your search because this page does not have recent snapshots.

c) Go to October 18, 2001 snapshot and see how this page looks like.

3) Now, you will perform an OSINT (Open Source Intelligence) challenge by using the Wayback Machine. Think about this case:

Paul was System Admin at x64 Corporation. He argued with his manager and left the company a few days back. Being disappointed, he started leaking sensitive data. He also deleted all the employee records.

Help our investigators to find his Phone number.

Take a screenshot of the browser window where Paul's phone is shown.

Section-3: Using gau Tool to Obtain a List of Archived URLs

A tool named gau (getallurls) fetches archived URLs from several databases, including the Wayback Machine, for any given domain. In this part, you will install this tool to your Kali VM (Virtual Machine) and use it for franklin.edu domain.

1) Open your Kali VM

2) Type in your credentials (username: kali, password: kali)

3) Open a terminal window

4) Type in the following commands:

a. sudo apt-get update (This command updates the package lists for upgrades and new packages; strongly recommended to complete the next command successfully)

b. sudo apt-get install golang (This will install golang compiler because gau was written with Go)

i. Type in kali as password when asked

ii. Press Y when asked

c. GO111MODULE=on go get -u -v github.com/lc/gau (This will download gau from github and install it)

d. cd ~/go/bin or cd /home/kali/go/bin (Go to the directory where gau has been installed)

e. ./gau franklin.edu (Run the gau against franklin.edu to find the archived webpages in franklin.edu domain)

f. It will not take much time to complete the command; however, you can press CTRL-C to stop the query if you want.

Take a screenshot of the terminal window showing the last 10 to 20 rows of the result.

Section-4: Using Shodan

Shodan is a specialized search engine that provides information about the versions of the devices connected to the Internet. A device can be anything having an IP address, including webservers, IP cameras, and even refrigerators, as long as it has been reachable by Shodan search robots.

Note: Please register Shodan before starting this lab. You will need to log in before using search filters in your searches. The first query below does not require you to login; however, you will need to register and then log in for the rest of the queries.

The Shodan website is shodan.io

1) Find all Apache web servers that the Shodan search engine has detected.

Type in apache to the search box and press enter. Review the results pages.

2) Find all Apache web servers located in China.

Type in apache country:"CN" to the search box and press enter.

Take a screenshot of the search result.

3) Find all Apache web servers located in Shanghai.

Type in apache country:"CN" city:"Shanghai" to the search box and press enter.

4) Find assets belong to an organization of your choice among the results in the Step-3.

Type in apache country:"CN" city:"Shanghai" org:"Alibaba” to the search box and press enter.

Note: Don't forget the check the website of Shodan on Black Friday. Shodan gives premium membership for just $5 instead of $49.

Section-5: Using sublist3r and amass

Sublist3r is an opensource tool to enumerate subdomains of websites using OSINT; it checks for the subdomains by using search engine data and security services like VirusTotal, ThreatCrowd.

Use your Kali VM to complete this lab. After logging into your Kali, open a Terminal window and type in the following command to install sublist3r.

sudo apt-get install sublist3r

Kali Linux may ask for a root password; if this is the case, then type in kali as the root password.

1) Find all subdomains of franklin.edu

Type in sublist3r -d franklin.edu to the terminal window.

2) Find which subdomains have port 80 is open

Type in sublist3r -d franklin.edu -p 80 to the terminal window.

Note: Because sublist3r uses OSINT to find subdomains, most subdomains may not be active at the moment. You can try searching for inactive subdomains in the WayBack Machine of archive.org. Pentesters and hackers may access useful information by using sublist3r and archive.org together.

Now you will use another tool named amass to query the franklin.edu domain. amass is an OWASP project ( https://owasp.org). It is a convenient tool for the enumeration of domain names, subdomains, associated IP addresses, and ASN numbers.

1) Find subdomains of franklin.edu along with IP addresses by using amass.

Type in amass enum -ip -d franklin.edu to the terminal window.

You can press CTRL-Z to stop the query after some results have been generated.

Take a screenshot of the terminal window (no need to capture all of the results).

Section-6: Finding DNS Records of a Domain by using nslookup

nslookup is used to query DNS servers and obtain the data stored by DNS servers such as IP addresses, hostnames, MX records, etc.

Use your Kali VM to complete this lab. Although nslookup is a versatile tool that comes with almost every operating system, the usage of parameters may change among different implementations.

Before starting below steps, change the DNS server address configured at Kali VM to Google DNS, which is 8.8.8.8. In order to do this:

1) Open a terminal window

2) Type in the following command to open the resolv.conf file. This file is used to store and configure the operating system's DNS resolver

sudo vi /etc/resolv.conf

This command will ask for the root password. Type in kali to open the resolv.conf in vi editor. The vi editor is a handy text editor that comes by default with Linux distributions.

3) In vi editor, press “i” letter to convert to the write mode.

4) Delete the IP address next to “nameserver” and type in 8.8.8.8 as the new DNS server

5) Press the "Esc" key when you finished editing.

6) Type “:wq" so that you can save your changes and quit vi.

Lab Steps:

1) Find the IP address of www.franklin.edu website

Type in nslookup franklin.edu to the terminal window.

2) Find the authoritative DNS server of the franklin.edu domain.

Type in nslookup -type=ns franklin.edu to the terminal window.

Take a screenshot of the terminal window.

3) Find the MX record of the franklin.edu domain.

Type in nslookup -type=mx franklin.edu or nslookup -query=mx franklin.edu to the terminal window.

4) Find all possible franklin.edu records stored by the DNS server.

Type in nslookup -query=any franklin.edu to the terminal window.

Final remarks

You can check the osintframework to get an idea of the scope of OSINT activities. This OSINT Framework website shows many OSINT resources, including websites and tools in a mind map.

https://osintframework.com

Weekly Learning and Reflection 

In two to three paragraphs (i.e., sentences, not bullet lists) using APA style citations if needed, summarize, and interact with the content covered in this lab. Summarize what you did as an attacker, what kind of vulnerabilities did you exploit, what might have prevented these attacks. Mention the attackers and all of the targets in your summary. You can provide topologies, sketches, graphics if you want. In particular, highlight what surprised, enlightened, or otherwise engaged you. You should think and write critically, not just about what was presented but also what you have learned through the session. You can ask questions for the things you're confused about. Questions asked here will be summarized and answered anonymously in the next class.