Ben's notes

Linux, Unix, network, radio...

User Tools

Site Tools


hashcat_amazon

Amazon EC2 advanced password recovery

This page is for educational use only.

Used information from rockfishsec and the hashcat wiki.

Create an instance

  • Go to Amazon's web services, EC2.
  • Click Launch Instance.
  • Click AWS Marketplace.
  • Search for 'Linux NVIDIA', select “Amazon Linux AMI with NVIDIA GRID and TESLA GPU Driver”.
  • Click Continue.
  • Filter by 'GPU instances'.
  • Select g2.8xlarge
  • Click 'Review and Launch'.
  • Select: Make General Purpose (SSD) the default boot volume for all instance launches from the console going forward (recommended).
  • Edit your security group.
  • Review the details / 16 GB storage, don't delete on termination.
  • Create a new key pair.
  • Launch the instance. At first i was not able to launch the instance. Opened a support case. Error: You have requested more instances (1) than your current instance limit of 0 allows for the specified instance type.

Configure the software

  • Log in with your key:
    chmod 400 <keyfile>
    ssh -i <key> ec2-user@<hostname>
    
           __|  __|_  )
           _|  (     /   Amazon Linux AMI
          ___|\___|___|
    
    https://aws.amazon.com/amazon-linux-ami/2016.09-release-notes/
    6 package(s) needed for security, out of 11 available
    Run "sudo yum update" to apply all updates.
  • Update the instance and reboot if needed.
  • Quick hardware check (32 CPU's and 4 GPU's):
    $ lscpu
    Architecture:          x86_64
    CPU op-mode(s):        32-bit, 64-bit
    Byte Order:            Little Endian
    CPU(s):                32
    On-line CPU(s) list:   0-31
    Thread(s) per core:    2
    Core(s) per socket:    8
    Socket(s):             2
    NUMA node(s):          2
    Vendor ID:             GenuineIntel
    CPU family:            6
    Model:                 45
    Model name:            Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
    Stepping:              7
    CPU MHz:               2600.068
    BogoMIPS:              5268.46
    Hypervisor vendor:     Xen
    Virtualization type:   full
    L1d cache:             32K
    L1i cache:             32K
    L2 cache:              256K
    L3 cache:              20480K
    NUMA node0 CPU(s):     0-15
    NUMA node1 CPU(s):     16-31
    
    $ lspci
    00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
    00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
    00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
    00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 01)
    00:02.0 VGA compatible controller: Cirrus Logic GD 5446
    00:03.0 VGA compatible controller: NVIDIA Corporation GK104GL [GRID K520] (rev a1)
    00:04.0 VGA compatible controller: NVIDIA Corporation GK104GL [GRID K520] (rev a1)
    00:05.0 VGA compatible controller: NVIDIA Corporation GK104GL [GRID K520] (rev a1)
    00:06.0 VGA compatible controller: NVIDIA Corporation GK104GL [GRID K520] (rev a1)
    00:1f.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01)

Install hashcat

  • Install packages and install from source:
    sudo yum install git
    
    git clone https://github.com/hashcat/hashcat.git
    cd hashcat
    mkdir -p deps/OpenCL-Headers
    git clone https://github.com/KhronosGroup/OpenCL-Headers deps/OpenCL-Headers/CL
    cd ../..
    make
    sudo make install

Test

  • See if hashcat sees the GPU's:
    ./hashcat --opencl-info
    hashcat (v3.10-809-g7fe575e+) starting...
    
    OpenCL Info:
    
    Platform ID #1
      Vendor  : NVIDIA Corporation
      Name    : NVIDIA CUDA
      Version : OpenCL 1.2 CUDA 7.5.30
    
      Device ID #1
        Type           : GPU
        Vendor ID      : 32
        Vendor         : NVIDIA Corporation
        Name           : GRID K520
        Version        : OpenCL 1.2 CUDA
        Processor(s)   : 8
        Clock          : 797
        Memory         : 1023/4095 MB allocatable
        OpenCL Version : OpenCL C 1.2
        Driver Version : 352.99
    
      Device ID #2
        Type           : GPU
        Vendor ID      : 32
        Vendor         : NVIDIA Corporation
        Name           : GRID K520
        Version        : OpenCL 1.2 CUDA
        Processor(s)   : 8
        Clock          : 797
        Memory         : 1023/4095 MB allocatable
        OpenCL Version : OpenCL C 1.2
        Driver Version : 352.99
    
      Device ID #3
        Type           : GPU
        Vendor ID      : 32
        Vendor         : NVIDIA Corporation
        Name           : GRID K520
        Version        : OpenCL 1.2 CUDA
        Processor(s)   : 8
        Clock          : 797
        Memory         : 1023/4095 MB allocatable
        OpenCL Version : OpenCL C 1.2
        Driver Version : 352.99
    
      Device ID #4
        Type           : GPU
        Vendor ID      : 32
        Vendor         : NVIDIA Corporation
        Name           : GRID K520
        Version        : OpenCL 1.2 CUDA
        Processor(s)   : 8
        Clock          : 797
        Memory         : 1023/4095 MB allocatable
        OpenCL Version : OpenCL C 1.2
        Driver Version : 352.99
        
    
    
    When starting a run, the following warning appears:
    
    * Device #1: Outdated or broken NVIDIA driver detected!
    
    
    
    You are STRONGLY encouraged to use the official supported NVIDIA driver
    
    See hashcat's homepage for official supported NVIDIA drivers
    
    Also see: https://hashcat.net/wiki/doku.php?id=frequently_asked_questions#i_may_have_the_wrong_driver_installed_what_should_i_do
    
    You can use --force to override this but do not post error reports if you do so

Install newer NVIDIA driver

  • Uninstall current drivers:
    sudo yum erase nvidia cuda -y
    sudo reboot
  • Install needed packages and the driver:
    sudo yum groupinstall -y "Development tools"
    sudo yum install kernel-devel-`uname -r`
    
    sudo /bin/bash NVIDIA-Linux-x86_64-367.57.run
    
      - Accept the license
      - Yes to registering the kernel module sources with DKMS
      - Yes to installing the 32-bit compatibility libraries
      - OK to the disclaimer about the libvdpau and libvdpau_trace libraries
      - Yes to running the nvidia-config utility
      - OK to acknowledge the X config file was updated
    
    sudo reboot
  • Check correct driver version:
    nvidia-smi -q | head -10
    
    ==============NVSMI LOG==============
    
    Timestamp                           : Tue Nov 22 20:55:03 2016
    Driver Version                      : 367.57
    
    Attached GPUs                       : 4
    GPU 0000:00:03.0
        Product Name                    : GRID K520
        Product Brand                   : Grid

Benchmark

  • md5crypt:
    $ sudo /usr/local/bin/hashcat -b -m 500
    hashcat (v3.10-809-g7fe575e+) starting in benchmark mode...
    
    * Device #1: Old CUDA chipset 3.0 detected, OpenCL performance is reduced.
                 For ideal hashcat performance on NVIDIA GPU you need Shader Model 5.0 or higher
    * Device #2: Old CUDA chipset 3.0 detected, OpenCL performance is reduced.
                 For ideal hashcat performance on NVIDIA GPU you need Shader Model 5.0 or higher
    * Device #3: Old CUDA chipset 3.0 detected, OpenCL performance is reduced.
                 For ideal hashcat performance on NVIDIA GPU you need Shader Model 5.0 or higher
    * Device #4: Old CUDA chipset 3.0 detected, OpenCL performance is reduced.
                 For ideal hashcat performance on NVIDIA GPU you need Shader Model 5.0 or higher
    nvmlDeviceGetFanSpeed(): Not Supported
    
    nvmlDeviceGetFanSpeed(): Not Supported
    
    nvmlDeviceGetFanSpeed(): Not Supported
    
    nvmlDeviceGetFanSpeed(): Not Supported
    
    OpenCL Platform #1: NVIDIA Corporation
    ======================================
    * Device #1: GRID K520, 1009/4036 MB allocatable, 8MCU
    * Device #2: GRID K520, 1009/4036 MB allocatable, 8MCU
    * Device #3: GRID K520, 1009/4036 MB allocatable, 8MCU
    * Device #4: GRID K520, 1009/4036 MB allocatable, 8MCU
    
    Hashtype: md5crypt, MD5(Unix), FreeBSD MD5, Cisco-IOS MD5
    
    Speed.Dev.#1.....:   927.5 kH/s (69.69ms)
    Speed.Dev.#2.....:   927.3 kH/s (69.69ms)
    Speed.Dev.#3.....:   927.0 kH/s (69.74ms)
    Speed.Dev.#4.....:   927.0 kH/s (69.73ms)
    Speed.Dev.#*.....:  3708.9 kH/s
    
    Started: Tue Nov 22 21:17:17 2016
    Stopped: Tue Nov 22 21:17:26 2016

Intel OpenCL runtime for CPU

  • Download latest OpenCL runtime: https://software.intel.com/en-us/articles/opencl-drivers#latest_CPU_runtime:
    sudo yum install redhat-lsb-core
    
    
    wget http://registrationcenter-download.intel.com/akdlm/irc_nas/9019/opencl_runtime_16.1.1_x64_rh_6.4.0.25.tgz
    tar zxf opencl_runtime_16.1.1_x64_rh_6.4.0.25.tgz
    cd opencl_runtime_16.1.1_x64_rh_6.4.0.25
    sudo ./install.sh
    
      - skip unsupported OS
      - accept the license
  • Test:
    hashcat -I
    hashcat (v3.10-809-g7fe575e+) starting...
    
    OpenCL Info:
    
    Platform ID #1
      Vendor  : Intel(R) Corporation
      Name    : Intel(R) OpenCL
      Version : OpenCL 1.2 LINUX
    
      Device ID #1
        Type           : CPU
        Vendor ID      : 8
        Vendor         : Intel(R) Corporation
        Name           :        Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
        Version        : OpenCL 1.2 (Build 25)
        Processor(s)   : 32
        Clock          : 2600
        Memory         : 2047/60387 MB allocatable
        OpenCL Version : OpenCL C 1.2
        Driver Version : 1.2.0.25
    
    Platform ID #2
      Vendor  : NVIDIA Corporation
      Name    : NVIDIA CUDA
      Version : OpenCL 1.2 CUDA 8.0.46
    
      Device ID #2
        Type           : GPU
        Vendor ID      : 32
        Vendor         : NVIDIA Corporation
        Name           : GRID K520
        Version        : OpenCL 1.2 CUDA
        Processor(s)   : 8
        Clock          : 797
        Memory         : 1009/4036 MB allocatable
        OpenCL Version : OpenCL C 1.2
        Driver Version : 367.57
    
      Device ID #3
        Type           : GPU
        Vendor ID      : 32
        Vendor         : NVIDIA Corporation
        Name           : GRID K520
        Version        : OpenCL 1.2 CUDA
        Processor(s)   : 8
        Clock          : 797
        Memory         : 1009/4036 MB allocatable
        OpenCL Version : OpenCL C 1.2
        Driver Version : 367.57
    
      Device ID #4
        Type           : GPU
        Vendor ID      : 32
        Vendor         : NVIDIA Corporation
        Name           : GRID K520
        Version        : OpenCL 1.2 CUDA
        Processor(s)   : 8
        Clock          : 797
        Memory         : 1009/4036 MB allocatable
        OpenCL Version : OpenCL C 1.2
        Driver Version : 367.57
    
      Device ID #5
        Type           : GPU
        Vendor ID      : 32
        Vendor         : NVIDIA Corporation
        Name           : GRID K520
        Version        : OpenCL 1.2 CUDA
        Processor(s)   : 8
        Clock          : 797
        Memory         : 1009/4036 MB allocatable
        OpenCL Version : OpenCL C 1.2
        Driver Version : 367.57
  • Test:
    Session..........: hashcat
    Status...........: Running
    Hash.Type........: md5crypt, MD5(Unix), FreeBSD MD5, Cisco-IOS MD5
    Hash.Target......: 
    Time.Started.....: Tue Nov 22 21:51:59 2016 (39 secs)
    Time.Estimated...: Tue Nov 22 22:27:02 2016 (34 mins, 24 secs)
    Input.Mask.......:  
    Input.Queue......: 1/1 (100.00%)
    Speed.Dev.#1.....:   126.1 kH/s (64.21ms)
    Speed.Dev.#2.....:   922.8 kH/s (69.76ms)
    Speed.Dev.#3.....:   923.2 kH/s (69.77ms)
    Speed.Dev.#4.....:   922.8 kH/s (69.74ms)
    Speed.Dev.#5.....:   922.8 kH/s (69.79ms)
    Speed.Dev.#*.....:  3817.6 kH/s
    Recovered........: 0/1 (0.00%) Digests, 0/1 (0.00%) Salts
    Recovered/Time...: CUR:N/A,N/A,N/A AVG:0.00,0.00,0.00 (Min,Hour,Day)
    Progress.........: 150454272/8031810176 (1.87%)
    Rejected.........: 0/150454272 (0.00%)
    Restore.Point....: 4399104/308915776 (1.42%)
    Candidates.#1....:  
    Candidates.#2....:  
    Candidates.#3....:  
    Candidates.#4....:  
    Candidates.#5....:  
    HWMon.Dev.#1.....: N/A
    HWMon.Dev.#2.....: Temp: 49c Util: 99% Core: 797Mhz Mem:2500Mhz Lanes:16
    HWMon.Dev.#3.....: Temp: 46c Util: 99% Core: 797Mhz Mem:2500Mhz Lanes:16
    HWMon.Dev.#4.....: Temp: 52c Util:100% Core: 797Mhz Mem:2500Mhz Lanes:16
    HWMon.Dev.#5.....: Temp: 44c Util:100% Core: 797Mhz Mem:2500Mhz Lanes:16
    
    
    htop:
    
      1  [|||||||||||||||||98.0%]    9  [|||||||||||||||||98.0%]     17 [|||||||||||||||||97.4%]    25 [|||||||||||||||||98.1%]
      2  [|||||||||||||||||98.7%]    10 [|||||||||||||||||98.1%]     18 [|||||||||||||||||98.0%]    26 [|||||||||||||||||97.5%]
      3  [|||||||||||||||||98.0%]    11 [|||||||||||||||||98.1%]     19 [|||||||||||||||||98.0%]    27 [|||||||||||||||||98.2%]
      4  [|||||||||||||||||97.4%]    12 [|||||||||||||||||97.5%]     20 [|||||||||||||||||98.7%]    28 [|||||||||||||||||97.4%]
      5  [|||||||||||||||||98.0%]    13 [|||||||||||||||||98.1%]     21 [|||||||||||||||||98.0%]    29 [|||||||||||||||||98.1%]
      6  [|||||||||||||||||98.0%]    14 [|||||||||||||||||98.1%]     22 [|||||||||||||||||98.0%]    30 [|||||||||||||||||98.1%]
      7  [|||||||||||||||||98.0%]    15 [|||||||||||||||||98.1%]     23 [|||||||||||||||||98.0%]    31 [|||||||||||||||||98.1%]
      8  [|||||||||||||||||98.0%]    16 [|||||||||||||||||97.4%]     24 [|||||||||||||||||98.0%]    32 [|||||||||||||||||97.5%]
      Mem[||||                                     1803/60387MB]     Tasks: 34, 71 thr; 34 running
      Swp[                                                0/0MB]     Load average: 27.78 11.48 4.39
                                                                     Uptime: 00:59:20
    
    $ uptime
     21:55:29 up  1:00,  2 users,  load average: 28.19, 15.47, 6.41
hashcat_amazon.txt · Last modified: 2021/10/09 15:14 by 127.0.0.1