# DataSurgeon | Quickly Extracts IP's, Email Addresses, Hashes, Files, Credit Cards, Social Secuirty Numbers And More From Text

**URL:** <https://onehack.st/t/datasurgeon-quickly-extracts-ips-email-addresses-hashes-files-credit-cards-social-secuirty-numbers-and-more-from-text/234313>\
**Category:** Tools & Scripts\
**Tags:** tools, hacking\
**Created:** [March 12, 2023, 11:32am UTC](https://onehack.st/t/datasurgeon-quickly-extracts-ips-email-addresses-hashes-files-credit-cards-social-secuirty-numbers-and-more-from-text/234313 "2023-03-12T11:32:37Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![NoBody](https://onehack.st/user_avatar/onehack.st/nobody/32/163880_2.png) [@NoBody](https://onehack.st/u/NoBody)\
**Post date:** [March 12, 2023, 11:32am UTC](https://onehack.st/t/datasurgeon-quickly-extracts-ips-email-addresses-hashes-files-credit-cards-social-secuirty-numbers-and-more-from-text/234313/1 "2023-03-12T11:32:37Z")

</div>

# DataSurgeon

[![preview](https://github.com/Drew-Alleman/DataSurgeon/raw/main/media/main.gif)](https://github.com/Drew-Alleman/DataSurgeon/blob/main/media/main.gif) DataSurgeon (ds) is a versatile tool designed for incident response, penetration testing, and CTF challenges. It allows for the extraction of various types of sensitive information including emails, phone numbers, hashes, credit cards, URLs, IP addresses, MAC addresses, SRV DNS records and a lot more!

- Supports Windows, Linux and MacOS

# Quick Links

- [Extraction Features](https://github.com/Drew-Alleman/DataSurgeon#extraction-features)
- [Quick Install](https://github.com/Drew-Alleman/DataSurgeon#quick-install)
- [Command Line Arguments](https://github.com/Drew-Alleman/DataSurgeon#command-line-arguments)
- [Examples](https://github.com/Drew-Alleman/DataSurgeon#examples)
- [Speed Tests](https://github.com/Drew-Alleman/DataSurgeon#speed-tests)
- [Recent Updates](https://github.com/Drew-Alleman/DataSurgeon#recent-updates)
- [Project Goals](https://github.com/Drew-Alleman/DataSurgeon#project-goals)

## Extraction Features

- Emails
- Files
- Phone numbers
- Credit Cards
- Google API Private Key ID’s
- Social Security Numbers
- AWS Keys
- Bitcoin wallets
- URL’s
- IPv4 Addresses and IPv6 addresses
- MAC Addresses
- SRV DNS Records
- Extract Hashes
  - MD4 & MD5
  - SHA-1, SHA-224, SHA-256, SHA-384, SHA-512
  - SHA-3 224, SHA-3 256, SHA-3 384, SHA-3 512
  - MySQL 323, MySQL 41
  - NTLM
  - bcrypt

### Want more?

Please read the contributing guidelines [here](https://github.com/Drew-Alleman/DataSurgeon/blob/main/CONTRIBUTING.md#adding-a-new-regex--extraction-feature)

# Quick Install

Install [Rust](https://www.rust-lang.org/tools/install) and [Github](https://desktop.github.com/)

### Linux

```auto
wget -O - https://raw.githubusercontent.com/Drew-Alleman/DataSurgeon/main/install/install.sh | bash

```

### Windows

Enter the line below in an elevated powershell window.

```auto
IEX (New-Object Net.WebClient).DownloadString("https://raw.githubusercontent.com/Drew-Alleman/DataSurgeon/main/install/install.ps1")

```

Relaunch your terminal and you will be able to use `ds` from the command line.

### Mac

```auto
curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/Drew-Alleman/DataSurgeon/main/install/install.sh | sh

```

# Command Line Arguments

[![help](https://github.com/Drew-Alleman/DataSurgeon/raw/main/media/help_preview.PNG)](https://github.com/Drew-Alleman/DataSurgeon/blob/main/media/help_preview.PNG)

## Video Guide

[![Video Title](https://camo.githubusercontent.com/ee13862a193cc995424feee9de6b4d625c3a38d2060ef399adbae690ac16fcb5/68747470733a2f2f696d672e796f75747562652e636f6d2f76692f4c4162366b424f574877672f302e6a7067)](https://www.youtube.com/watch?v=LAb6kBOWHwg)

# Examples

## Extracting Files From a Remote Webiste

Here I use `wget` to make a request to stackoverflow then I forward the body text to `ds` . The `-F` option will list all files found. `--clean` is used to remove any extra text that might have been returned (such as extra html). Then the result of is sent to `uniq` which removes any non unique files found. Ig you wanted you can remove the warning message at the top ‘Reading standard input..’ by using -S.

```auto
$ wget -qO - https://www.stackoverflow.com | ds -F --clean | uniq

```

[![preview](https://github.com/Drew-Alleman/DataSurgeon/raw/main/media/wget_preview.gif)](https://github.com/Drew-Alleman/DataSurgeon/blob/main/media/wget_preview.gif)

## Extracting Mac Addresses From an Output File

Here I am pulling all mac addresses found in [autodeauth’s](https://github.com/Drew-Alleman/autodeauth) log file using the `-m` query. The `--hide` option will hide the identifer string infront of the results. In this case 'mac\_address: ’ is hidden from the output. The `-T` option is used to check the same line multiple times for matches. Normallly when a match is found the tool moves on to the next line rather then checking again.

```auto
$ ./ds -m -T --hide -f /var/log/autodeauth/log     
2023-02-26 00:28:19 - Sending 500 deauth frames to network: BC:2E:48:E5:DE:FF -- PrivateNetwork
2023-02-26 00:35:22 - Sending 500 deauth frames to network: 90:58:51:1C:C9:E1 -- TestNet

```

## Reading all files in a directory

The line below will will read all files in the current directory recursively. The `-D` option is used to display the filename (-f is required for the filename to display) and -e used to search for emails.

```auto
$ find . -type f -exec ds -f {} -CDe \;

```

[![preview](https://github.com/Drew-Alleman/DataSurgeon/raw/main/media/directory_search.gif)](https://github.com/Drew-Alleman/DataSurgeon/blob/main/media/directory_search.gif)

## CSV Output

To output your results to a CSV file, use the `-o` option followed by the name of the file you want to save your data to. The `-D` and `-X` are supported. The format is: `ds -o <FILENAME>.csv` (.csv is needed).

```auto
 $ wget -qO - https://www.stackoverflow.com | ds -o output.csv -C

```

[![preview](https://github.com/Drew-Alleman/DataSurgeon/raw/main/media/csv_output.gif)](https://github.com/Drew-Alleman/DataSurgeon/blob/main/media/csv_output.gif)

# Speed Tests

When no specific query is provided, `ds` will search through all possible types of data, which is **SIGNIFICANTLY** slower than using individual queries. The slowest query is `--files`. Its also slightly faster to use `cat` to pipe the data to `ds`.

Below is the elapsed time when processing a 5GB test file generated by [ds-test](https://github.com/Drew-Alleman/ds-test). Each test was ran 3 times and the average time was recorded.

### Computer Specs

```auto
Processor	Intel(R) Core(TM) i5-10400F CPU @ 2.90GHz, 2904 Mhz, 6 Core(s), 12 Logical Processor(s)
Ram 12.0 GB (11.9 GB usable)

```

## Searching all data types

| Command | Speed |
| --- | --- |
| `cat test.txt | ds -t` |
| `ds -t -f test.txt` | 00h:02m:05s |
| `cat test.txt | ds -t -o output.txt` |

## Using specific queries

| Command | Speed | Query Count |
| --- | --- | --- |
| `cat test.txt | ds -t -6` | 00h:00m:12s |
| `cat test.txt | ds -t -i -m` | 00h:00m:22 |
| `cat test.txt | ds -tF6c` | 00h:00m:32s |

# Recent Updates

## 3/10/2023 | 1.0.7

- Updated the AWS regex (issue [here](https://github.com/Drew-Alleman/DataSurgeon/issues/4))

## 3/9/2023 | 1.0.6

- Added support for CSV output (e.g `ds -o output.csv -f test.txt`)

## 3/7/2023 | 1.0.5

- Added -S / --suppress options to remove '[\*] Reading standard input. If you meant to analyze a file use ‘ds -f ’ (ctrl+c to exit)’ message

# Project Goals

- JSON output
- Untar/unzip and a directorty searching mode
- Base64 Detection and decoding

## GitHub:

[https://github.com/Drew-Alleman/DataSurgeon](https://github.com/Drew-Alleman/DataSurgeon)