# Uscrapper | Powerful OSINT Webscraper For Personal Data Collection

**URL:** <https://onehack.st/t/uscrapper-powerful-osint-webscraper-for-personal-data-collection/273386>\
**Category:** Tools & Scripts\
**Tags:** programming, trading\
**Created:** [January 26, 2024, 1:21am UTC](https://onehack.st/t/uscrapper-powerful-osint-webscraper-for-personal-data-collection/273386 "2024-01-26T01:21:15Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![WhiteHat](https://onehack.st/user_avatar/onehack.st/whitehat/32/162514_2.png) [@WhiteHat](https://onehack.st/u/WhiteHat)\
**Post date:** [January 26, 2024, 1:21am UTC](https://onehack.st/t/uscrapper-powerful-osint-webscraper-for-personal-data-collection/273386/1 "2024-01-26T01:21:16Z")

</div>

# Uscrapper 2.0

[![project-image](https://camo.githubusercontent.com/05af45dbb1d4000c49cba2761bb3d93dabf658b9fba0155d8f70e8adfc1c9f8c/68747470733a2f2f736f6369616c6966792e6769742e63692f7a306d3331656e372f5573637261707065722f696d6167653f666f6e743d536f75726365253230436f646525323050726f266e616d653d31266f776e65723d31267061747465726e3d506c7573267468656d653d4461726b)](https://camo.githubusercontent.com/05af45dbb1d4000c49cba2761bb3d93dabf658b9fba0155d8f70e8adfc1c9f8c/68747470733a2f2f736f6369616c6966792e6769742e63692f7a306d3331656e372f5573637261707065722f696d6167653f666f6e743d536f75726365253230436f646525323050726f266e616d653d31266f776e65723d31267061747465726e3d506c7573267468656d653d4461726b)

Introducing Uscrapper 2.0, A powerfull OSINT webscrapper that allows users to extract various personal information from a website. It leverages web scraping techniques and regular expressions to extract email addresses, social media links, author names, geolocations, phone numbers, and usernames from both hyperlinked and non-hyperlinked sources on the webpage, supports multithreading to make this process faster, Uscrapper 2.0 is equipped with advanced Anti-webscrapping bypassing modules and supports webcrawling to scrape from various sublinks within the same domain. The tool also provides an option to generate a report containing the extracted details.

[![shields](https://camo.githubusercontent.com/7eefb2ba052806d8a9ce69863c2eeb3b03cd5935ead7bd2e9245ae2e705a1adf/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f4c696e75782d4643433632343f7374796c653d666f722d7468652d6261646765266c6f676f3d6c696e7578266c6f676f436f6c6f723d626c61636b)](https://camo.githubusercontent.com/7eefb2ba052806d8a9ce69863c2eeb3b03cd5935ead7bd2e9245ae2e705a1adf/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f4c696e75782d4643433632343f7374796c653d666f722d7468652d6261646765266c6f676f3d6c696e7578266c6f676f436f6c6f723d626c61636b)[![shields](https://camo.githubusercontent.com/80cbbdd6f85abb2ad8e2c6e16f3b9df09d50247784281901e031c3fe437ead5e/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f746d75782d3142423931463f7374796c653d666f722d7468652d6261646765266c6f676f3d746d7578266c6f676f436f6c6f723d7768697465)](https://camo.githubusercontent.com/80cbbdd6f85abb2ad8e2c6e16f3b9df09d50247784281901e031c3fe437ead5e/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f746d75782d3142423931463f7374796c653d666f722d7468652d6261646765266c6f676f3d746d7578266c6f676f436f6c6f723d7768697465)[![shields](https://camo.githubusercontent.com/8f38329b294b3995ee6d2995fad97c75a21555e43256dd66d2fa3da450706790/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f77696e646f77732532307465726d696e616c2d3444344434443f7374796c653d666f722d7468652d6261646765266c6f676f3d77696e646f77732532307465726d696e616c266c6f676f436f6c6f723d7768697465)](https://camo.githubusercontent.com/8f38329b294b3995ee6d2995fad97c75a21555e43256dd66d2fa3da450706790/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f77696e646f77732532307465726d696e616c2d3444344434443f7374796c653d666f722d7468652d6261646765266c6f676f3d77696e646f77732532307465726d696e616c266c6f676f436f6c6f723d7768697465)[![shields](https://camo.githubusercontent.com/19216a3c96c4fa2ca873d098b6587f3e285b33cce32d3776c9de2fb4ca4ad5dc/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f695465726d322d3030303030303f7374796c653d666f722d7468652d6261646765266c6f676f3d697465726d32266c6f676f436f6c6f723d7768697465)](https://camo.githubusercontent.com/19216a3c96c4fa2ca873d098b6587f3e285b33cce32d3776c9de2fb4ca4ad5dc/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f695465726d322d3030303030303f7374796c653d666f722d7468652d6261646765266c6f676f3d697465726d32266c6f676f436f6c6f723d7768697465)[![shields](https://camo.githubusercontent.com/55e4079e69ec5d8246620ecff24ed093877ab0f9011e71d8dec0a2c460c886ab/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f507974686f6e2d3337373641423f7374796c653d666f722d7468652d6261646765266c6f676f3d707974686f6e266c6f676f436f6c6f723d7768697465)](https://camo.githubusercontent.com/55e4079e69ec5d8246620ecff24ed093877ab0f9011e71d8dec0a2c460c886ab/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f507974686f6e2d3337373641423f7374796c653d666f722d7468652d6261646765266c6f676f3d707974686f6e266c6f676f436f6c6f723d7768697465)

[![project-logo](https://camo.githubusercontent.com/0f090a7623590704d431b31bc0cfc60247f49202183fcb330fbe2a83b4eff7f6/68747470733a2f2f6c68332e676f6f676c6575736572636f6e74656e742e636f6d2f64726976652d7669657765722f41495446772d794c327a594b583179455a50594c504b356272434f7a5f6a534d4c4831696c455069376a6553417630585549626b6634617264573070666c5556376c74787071707059726d644f74354e57663234506a706771786b45317a426c3d7331363030)](https://camo.githubusercontent.com/0f090a7623590704d431b31bc0cfc60247f49202183fcb330fbe2a83b4eff7f6/68747470733a2f2f6c68332e676f6f676c6575736572636f6e74656e742e636f6d2f64726976652d7669657765722f41495446772d794c327a594b583179455a50594c504b356272434f7a5f6a534d4c4831696c455069376a6553417630585549626b6634617264573070666c5556376c74787071707059726d644f74354e57663234506a706771786b45317a426c3d7331363030)

## 💡 Extracted Details:

Uscrapper extracts the following details from the provided website:

- Email Addresses: Displays email addresses found on the website.
- Social Media Links: Displays links to various social media platforms found on the website.
- Author Names: Displays the names of authors associated with the website.
- Geolocations: Displays geolocation information associated with the website.
- Non-Hyperlinked Details: Displays non-hyperlinked details found on the website including email addresses phone numbers and usernames.

## 📽 Preview:

[![project-ss](https://camo.githubusercontent.com/8461469097527cc6cd053fdc4e810052df1d5e997d030400d1aef24645a71264/68747470733a2f2f6c68332e676f6f676c6575736572636f6e74656e742e636f6d2f64726976652d7669657765722f41495446772d792d37505334386943307355324850536a6c42616e704d3452504b4a6e3347476d6d6e46596d715a3550714c794c764f34616566447a714954704f3532665077593546483879347374696b5f795956575f527a736e6c6970445578673d7332353630)](https://camo.githubusercontent.com/8461469097527cc6cd053fdc4e810052df1d5e997d030400d1aef24645a71264/68747470733a2f2f6c68332e676f6f676c6575736572636f6e74656e742e636f6d2f64726976652d7669657765722f41495446772d792d37505334386943307355324850536a6c42616e704d3452504b4a6e3347476d6d6e46596d715a3550714c794c764f34616566447a714954704f3532665077593546483879347374696b5f795956575f527a736e6c6970445578673d7332353630)

[![project-ss2](https://camo.githubusercontent.com/92aa74573a5fb3289e7f892295f31525e31f0dfb78d8322887f0fb9755252dca/68747470733a2f2f6c68332e676f6f676c6575736572636f6e74656e742e636f6d2f64726976652d7669657765722f41495446772d783656307a77336d67716e4263764b6c57524c59764e51766a7573546b2d6e764c6558437033476d454373594c656962786e5343464a7471597435304f47315956775055323254315136465852476454425265326d68346e65384b773d7331363030)](https://camo.githubusercontent.com/92aa74573a5fb3289e7f892295f31525e31f0dfb78d8322887f0fb9755252dca/68747470733a2f2f6c68332e676f6f676c6575736572636f6e74656e742e636f6d2f64726976652d7669657765722f41495446772d783656307a77336d67716e4263764b6c57524c59764e51766a7573546b2d6e764c6558437033476d454373594c656962786e5343464a7471597435304f47315956775055323254315136465852476454425265326d68346e65384b773d7331363030)

## 🤩 Whats New?:

Uscrapper 2.0:

- Introduced multiple modules to bypass anti-webscrapping techniques.
- Introducing Crawl and scrape: an advanced crawl and scrape module to scrape the websites from within.
- Implemented Multithreading to make these processes faster.

## 🛠 Installation Steps:

```auto
git clone https://github.com/z0m31en7/Uscrapper.git

```

```auto
cd Uscrapper/install/ 
python -m venv venv
source venv/bin/activate
chmod +x ./install.sh && ./install.sh #For Unix/Linux systems
cd ..

```

## 🔮 Usage:

To run Uscrapper, use the following command-line syntax:

```auto
python Uscrapper-v2.0.py [-h] [-u URL] [-c (INT)] [-t THREADS] [-O] [-ns]

```

**Arguments:**

- -h, --help: Show the help message and exit.
- -u URL, --url URL: Specify the URL of the website to extract details from.
- -c INT, --crawl INT: Specify the number of links to crawl
- -t INT, --threads INT: Specify the number of threads to use while crawling and scraping.
- -O, --generate-report: Generate a report file containing the extracted details.
- -ns, --nonstrict: Display non-strict usernames during extraction.

## 📜 Note:

- Uscrapper relies on web scraping techniques to extract information from websites. Make sure to use it responsibly and in compliance with the website’s terms of service and applicable laws.
- The accuracy and completeness of the extracted details depend on the structure and content of the website being analyzed.
- To bypass some Anti-Webscrapping methods we have used selenium which can make the overall process slower.

## 💌 Contribution:

**Want a new feature to be added?**

- Make a pull request with all the necessary details and it will be merged after a review.
- You can contribute by making the regular expressions more efficient and accurate, or by suggesting some more features that can be added.

## 🛡 License:

This project is licensed under the [MIT-LICENSE](https://github.com/z0m31en7/Uscrapper/blob/main/LICENSE)

## GitHub:

[https://github.com/z0m31en7/Uscrapper](https://github.com/z0m31en7/Uscrapper)