{{tag>[hardware security crack hash password]}}
=====Amazon EC2 advanced password recovery=====
This page is for educational use only.
Used information from [[http://www.rockfishsec.com/2015/05/gpu-password-cracking-with-amazon-ec2.html|rockfishsec]] and the hashcat wiki.
====Create an instance====
* Go to Amazon's web services, EC2.
* Click Launch Instance.
* Click AWS Marketplace.
* Search for 'Linux NVIDIA', select "Amazon Linux AMI with NVIDIA GRID and TESLA GPU Driver".
* Click Continue.
* Filter by 'GPU instances'.
* Select ''g2.8xlarge''
* Click 'Review and Launch'.
* Select: Make General Purpose (SSD) the default boot volume for all instance launches from the console going forward (recommended).
* Edit your security group.
* Review the details / 16 GB storage, don't delete on termination.
* Create a new key pair.
* Launch the instance. At first i was not able to launch the instance. Opened a support case. Error: //You have requested more instances (1) than your current instance limit of 0 allows for the specified instance type.//
====Configure the software====
* Log in with your key:
chmod 400
ssh -i ec2-user@
__| __|_ )
_| ( / Amazon Linux AMI
___|\___|___|
https://aws.amazon.com/amazon-linux-ami/2016.09-release-notes/
6 package(s) needed for security, out of 11 available
Run "sudo yum update" to apply all updates.
* Update the instance and reboot if needed.
* Quick hardware check (32 CPU's and 4 GPU's):
$ lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 32
On-line CPU(s) list: 0-31
Thread(s) per core: 2
Core(s) per socket: 8
Socket(s): 2
NUMA node(s): 2
Vendor ID: GenuineIntel
CPU family: 6
Model: 45
Model name: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
Stepping: 7
CPU MHz: 2600.068
BogoMIPS: 5268.46
Hypervisor vendor: Xen
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 20480K
NUMA node0 CPU(s): 0-15
NUMA node1 CPU(s): 16-31
$ lspci
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 01)
00:02.0 VGA compatible controller: Cirrus Logic GD 5446
00:03.0 VGA compatible controller: NVIDIA Corporation GK104GL [GRID K520] (rev a1)
00:04.0 VGA compatible controller: NVIDIA Corporation GK104GL [GRID K520] (rev a1)
00:05.0 VGA compatible controller: NVIDIA Corporation GK104GL [GRID K520] (rev a1)
00:06.0 VGA compatible controller: NVIDIA Corporation GK104GL [GRID K520] (rev a1)
00:1f.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01)
===Install hashcat===
* Install packages and install from source:
sudo yum install git
git clone https://github.com/hashcat/hashcat.git
cd hashcat
mkdir -p deps/OpenCL-Headers
git clone https://github.com/KhronosGroup/OpenCL-Headers deps/OpenCL-Headers/CL
cd ../..
make
sudo make install
====Test====
* See if hashcat sees the GPU's:
./hashcat --opencl-info
hashcat (v3.10-809-g7fe575e+) starting...
OpenCL Info:
Platform ID #1
Vendor : NVIDIA Corporation
Name : NVIDIA CUDA
Version : OpenCL 1.2 CUDA 7.5.30
Device ID #1
Type : GPU
Vendor ID : 32
Vendor : NVIDIA Corporation
Name : GRID K520
Version : OpenCL 1.2 CUDA
Processor(s) : 8
Clock : 797
Memory : 1023/4095 MB allocatable
OpenCL Version : OpenCL C 1.2
Driver Version : 352.99
Device ID #2
Type : GPU
Vendor ID : 32
Vendor : NVIDIA Corporation
Name : GRID K520
Version : OpenCL 1.2 CUDA
Processor(s) : 8
Clock : 797
Memory : 1023/4095 MB allocatable
OpenCL Version : OpenCL C 1.2
Driver Version : 352.99
Device ID #3
Type : GPU
Vendor ID : 32
Vendor : NVIDIA Corporation
Name : GRID K520
Version : OpenCL 1.2 CUDA
Processor(s) : 8
Clock : 797
Memory : 1023/4095 MB allocatable
OpenCL Version : OpenCL C 1.2
Driver Version : 352.99
Device ID #4
Type : GPU
Vendor ID : 32
Vendor : NVIDIA Corporation
Name : GRID K520
Version : OpenCL 1.2 CUDA
Processor(s) : 8
Clock : 797
Memory : 1023/4095 MB allocatable
OpenCL Version : OpenCL C 1.2
Driver Version : 352.99
When starting a run, the following warning appears:
* Device #1: Outdated or broken NVIDIA driver detected!
You are STRONGLY encouraged to use the official supported NVIDIA driver
See hashcat's homepage for official supported NVIDIA drivers
Also see: https://hashcat.net/wiki/doku.php?id=frequently_asked_questions#i_may_have_the_wrong_driver_installed_what_should_i_do
You can use --force to override this but do not post error reports if you do so
====Install newer NVIDIA driver====
* Uninstall current drivers: sudo yum erase nvidia cuda -y
sudo reboot
* Download legacy driver: [[http://us.download.nvidia.com/XFree86/Linux-x86_64/367.57/NVIDIA-Linux-x86_64-367.57.run]]
* Install needed packages and the driver:
sudo yum groupinstall -y "Development tools"
sudo yum install kernel-devel-`uname -r`
sudo /bin/bash NVIDIA-Linux-x86_64-367.57.run
- Accept the license
- Yes to registering the kernel module sources with DKMS
- Yes to installing the 32-bit compatibility libraries
- OK to the disclaimer about the libvdpau and libvdpau_trace libraries
- Yes to running the nvidia-config utility
- OK to acknowledge the X config file was updated
sudo reboot
* Check correct driver version:
nvidia-smi -q | head -10
==============NVSMI LOG==============
Timestamp : Tue Nov 22 20:55:03 2016
Driver Version : 367.57
Attached GPUs : 4
GPU 0000:00:03.0
Product Name : GRID K520
Product Brand : Grid
====Benchmark====
* md5crypt:
$ sudo /usr/local/bin/hashcat -b -m 500
hashcat (v3.10-809-g7fe575e+) starting in benchmark mode...
* Device #1: Old CUDA chipset 3.0 detected, OpenCL performance is reduced.
For ideal hashcat performance on NVIDIA GPU you need Shader Model 5.0 or higher
* Device #2: Old CUDA chipset 3.0 detected, OpenCL performance is reduced.
For ideal hashcat performance on NVIDIA GPU you need Shader Model 5.0 or higher
* Device #3: Old CUDA chipset 3.0 detected, OpenCL performance is reduced.
For ideal hashcat performance on NVIDIA GPU you need Shader Model 5.0 or higher
* Device #4: Old CUDA chipset 3.0 detected, OpenCL performance is reduced.
For ideal hashcat performance on NVIDIA GPU you need Shader Model 5.0 or higher
nvmlDeviceGetFanSpeed(): Not Supported
nvmlDeviceGetFanSpeed(): Not Supported
nvmlDeviceGetFanSpeed(): Not Supported
nvmlDeviceGetFanSpeed(): Not Supported
OpenCL Platform #1: NVIDIA Corporation
======================================
* Device #1: GRID K520, 1009/4036 MB allocatable, 8MCU
* Device #2: GRID K520, 1009/4036 MB allocatable, 8MCU
* Device #3: GRID K520, 1009/4036 MB allocatable, 8MCU
* Device #4: GRID K520, 1009/4036 MB allocatable, 8MCU
Hashtype: md5crypt, MD5(Unix), FreeBSD MD5, Cisco-IOS MD5
Speed.Dev.#1.....: 927.5 kH/s (69.69ms)
Speed.Dev.#2.....: 927.3 kH/s (69.69ms)
Speed.Dev.#3.....: 927.0 kH/s (69.74ms)
Speed.Dev.#4.....: 927.0 kH/s (69.73ms)
Speed.Dev.#*.....: 3708.9 kH/s
Started: Tue Nov 22 21:17:17 2016
Stopped: Tue Nov 22 21:17:26 2016
====Intel OpenCL runtime for CPU====
* Download latest OpenCL runtime: [[https://software.intel.com/en-us/articles/opencl-drivers#latest_CPU_runtime]]:
sudo yum install redhat-lsb-core
wget http://registrationcenter-download.intel.com/akdlm/irc_nas/9019/opencl_runtime_16.1.1_x64_rh_6.4.0.25.tgz
tar zxf opencl_runtime_16.1.1_x64_rh_6.4.0.25.tgz
cd opencl_runtime_16.1.1_x64_rh_6.4.0.25
sudo ./install.sh
- skip unsupported OS
- accept the license
* Test:
hashcat -I
hashcat (v3.10-809-g7fe575e+) starting...
OpenCL Info:
Platform ID #1
Vendor : Intel(R) Corporation
Name : Intel(R) OpenCL
Version : OpenCL 1.2 LINUX
Device ID #1
Type : CPU
Vendor ID : 8
Vendor : Intel(R) Corporation
Name : Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
Version : OpenCL 1.2 (Build 25)
Processor(s) : 32
Clock : 2600
Memory : 2047/60387 MB allocatable
OpenCL Version : OpenCL C 1.2
Driver Version : 1.2.0.25
Platform ID #2
Vendor : NVIDIA Corporation
Name : NVIDIA CUDA
Version : OpenCL 1.2 CUDA 8.0.46
Device ID #2
Type : GPU
Vendor ID : 32
Vendor : NVIDIA Corporation
Name : GRID K520
Version : OpenCL 1.2 CUDA
Processor(s) : 8
Clock : 797
Memory : 1009/4036 MB allocatable
OpenCL Version : OpenCL C 1.2
Driver Version : 367.57
Device ID #3
Type : GPU
Vendor ID : 32
Vendor : NVIDIA Corporation
Name : GRID K520
Version : OpenCL 1.2 CUDA
Processor(s) : 8
Clock : 797
Memory : 1009/4036 MB allocatable
OpenCL Version : OpenCL C 1.2
Driver Version : 367.57
Device ID #4
Type : GPU
Vendor ID : 32
Vendor : NVIDIA Corporation
Name : GRID K520
Version : OpenCL 1.2 CUDA
Processor(s) : 8
Clock : 797
Memory : 1009/4036 MB allocatable
OpenCL Version : OpenCL C 1.2
Driver Version : 367.57
Device ID #5
Type : GPU
Vendor ID : 32
Vendor : NVIDIA Corporation
Name : GRID K520
Version : OpenCL 1.2 CUDA
Processor(s) : 8
Clock : 797
Memory : 1009/4036 MB allocatable
OpenCL Version : OpenCL C 1.2
Driver Version : 367.57
* Test:
Session..........: hashcat
Status...........: Running
Hash.Type........: md5crypt, MD5(Unix), FreeBSD MD5, Cisco-IOS MD5
Hash.Target......:
Time.Started.....: Tue Nov 22 21:51:59 2016 (39 secs)
Time.Estimated...: Tue Nov 22 22:27:02 2016 (34 mins, 24 secs)
Input.Mask.......:
Input.Queue......: 1/1 (100.00%)
Speed.Dev.#1.....: 126.1 kH/s (64.21ms)
Speed.Dev.#2.....: 922.8 kH/s (69.76ms)
Speed.Dev.#3.....: 923.2 kH/s (69.77ms)
Speed.Dev.#4.....: 922.8 kH/s (69.74ms)
Speed.Dev.#5.....: 922.8 kH/s (69.79ms)
Speed.Dev.#*.....: 3817.6 kH/s
Recovered........: 0/1 (0.00%) Digests, 0/1 (0.00%) Salts
Recovered/Time...: CUR:N/A,N/A,N/A AVG:0.00,0.00,0.00 (Min,Hour,Day)
Progress.........: 150454272/8031810176 (1.87%)
Rejected.........: 0/150454272 (0.00%)
Restore.Point....: 4399104/308915776 (1.42%)
Candidates.#1....:
Candidates.#2....:
Candidates.#3....:
Candidates.#4....:
Candidates.#5....:
HWMon.Dev.#1.....: N/A
HWMon.Dev.#2.....: Temp: 49c Util: 99% Core: 797Mhz Mem:2500Mhz Lanes:16
HWMon.Dev.#3.....: Temp: 46c Util: 99% Core: 797Mhz Mem:2500Mhz Lanes:16
HWMon.Dev.#4.....: Temp: 52c Util:100% Core: 797Mhz Mem:2500Mhz Lanes:16
HWMon.Dev.#5.....: Temp: 44c Util:100% Core: 797Mhz Mem:2500Mhz Lanes:16
htop:
1 [|||||||||||||||||98.0%] 9 [|||||||||||||||||98.0%] 17 [|||||||||||||||||97.4%] 25 [|||||||||||||||||98.1%]
2 [|||||||||||||||||98.7%] 10 [|||||||||||||||||98.1%] 18 [|||||||||||||||||98.0%] 26 [|||||||||||||||||97.5%]
3 [|||||||||||||||||98.0%] 11 [|||||||||||||||||98.1%] 19 [|||||||||||||||||98.0%] 27 [|||||||||||||||||98.2%]
4 [|||||||||||||||||97.4%] 12 [|||||||||||||||||97.5%] 20 [|||||||||||||||||98.7%] 28 [|||||||||||||||||97.4%]
5 [|||||||||||||||||98.0%] 13 [|||||||||||||||||98.1%] 21 [|||||||||||||||||98.0%] 29 [|||||||||||||||||98.1%]
6 [|||||||||||||||||98.0%] 14 [|||||||||||||||||98.1%] 22 [|||||||||||||||||98.0%] 30 [|||||||||||||||||98.1%]
7 [|||||||||||||||||98.0%] 15 [|||||||||||||||||98.1%] 23 [|||||||||||||||||98.0%] 31 [|||||||||||||||||98.1%]
8 [|||||||||||||||||98.0%] 16 [|||||||||||||||||97.4%] 24 [|||||||||||||||||98.0%] 32 [|||||||||||||||||97.5%]
Mem[|||| 1803/60387MB] Tasks: 34, 71 thr; 34 running
Swp[ 0/0MB] Load average: 27.78 11.48 4.39
Uptime: 00:59:20
$ uptime
21:55:29 up 1:00, 2 users, load average: 28.19, 15.47, 6.41