---

title: Quick incident triage with Linux from the bar, airplane, or ...

author: Jeff Beley @jbeley
date: 01-APR-2019
Mutated by Mark McCurdy for North Texas Cyber

---

About me

• Nose to the monitor type of guy. Get to the point.

• Keep notes and make docs on anything of importance

• Give me anything that runs bash with GNU tools

• Twitter: @marcurdy

• Github: github.com/marcurdy

About Jeff Beley

• OSS Evangelist

• The nosiest guy on your network

• docker fanatic

• Twitter: @jbeley

Badguy[.]net call from FBI

Always the true story for all corporations

• Agent Smith calls $CLIENT stating they have a machine talking to badguy[.]net

  • No other information is typically given by the FBI

• $CLIENT finds a massive ransomware outbreak

  • Files across many hosts start self-encrypting
  • Is this a full domain compromise?
  • You don't get to execute on that many machines without credentials

• $CLIENT calls us to unravel the puzzle and save the company from ruin

What are we hunting for most often

• Tradecraft tools

• mimikatz

• PowerShell Empire

• Cobalt Strike

• Lateral movement

• psexec

• RDP sessions from the internet? From within your company?

• Packaging of data in preparation for exfiltration

• Executition of 7-zip, rar, zip compression utilities to package data

• On disk artifacts that tell the story and create a timeline

First we're going to need tools, but not lots of tools

Tools
Linux Client because "GNU"	CyLR (artifact gathering)
plaso (timeline for targeted artifact pulls)	volatility (memory analysis)
yara ( supercharged pattern matcher )	loki (tradecraft hunting)
imount ( exposing disk images)	scalpel ( file carving )

Total cost: 0$

No big box e-discovery dongles required

SSH Client

• The Genuine OpenSSH client built into Linux and MacOS
• Putty - SSH Client for Windows
• iTerm2 for MacOS (not an SSH client but an amazing terminal emulator)

Workflow Enhancers

• byobu

  • keeps session alive by use of status bar
  • status bar configurable via byobu-config
  • Can use GNU Screen bindings or tmux
  • installable via apt-get or brew
  • Japanese for folding screen

• GNU Screen

  • more ubiquious than byobu
  • no automated status bar
  • installable via apt-get or brew

Workflow Enhancers CONT'd

• A timestamped bash history file (customization of /etc/bash.bashrc)

# set the number commands for bash to remember in the command history
HISTFILESIZE=1000000
# set the maximum number of lines to store in the bash history file
HISTSIZE=1000000
# set the format of the timestamps stored in  the bash history file
HISTTIMEFORMAT='%F %T '
# force bash to write to history file  upon returning to a prompt,
# this ensures bash logs commands even when the interactive shell session terminates
PROMPT_COMMAND='history -a'

Exposing images

• imount

works with E01s, VMDKS, RAW, bitlocker, LVM

imount /path/to/VMDK

• There are other options to change the default behavior, but the defaults work

NOTE: Userspace filesystems (NTFS, EWF, etc) are slow.
Targeted collections over full disk capture enable quick triage

Let's start hunting

• badguy[.]net has historically resolved to 45.xxx.xxx.xxx so we'll start searching at Robtex

First indicators

We ask for a memory capture from an infected host known to be infected.

• Easy way
• Hard way

As an Incident Responder, we can query anything using volatility

Second Indicators

Suspect patient zero is a VM. It was suspended not stopped. Why?
Memory is stored in a single file

vol.py -f memdump.vmem --profile=Win2008R2SP1x64_23418 imagecopy -O memdump.mem

GNU strings then GNU strings again

strings -o -el memdump.mem > memdump.txt
strings -o  memdump.mem >> memdump.txt
egrep -iwF 'badguy.net|45.xxx.xxx.xxx' memdump.txt > badguy.txt
less badguy.txt

• Why do we have to run GNU strings twice?

• What are the arguments to egrep used?

• Differences between Mac OS strings and GNU strings

A brief pivot

Attack graph...We've seen some stuff

Possible Tools for visualization include

• Splunk (restricted but free license)
• Elastic
• Microsoft's PowerBI, really

Cobalt Strike OSINT Profiling with Shodan.io

• The Shodan report for 45.xxx.xxx.xxx is pretty interesting.

Cobalt Strike OSINT Profiling CONT'd

• HTTP or HTTPS open returning error code 404
  • generally running nginx but sometimes Apache httpd
• SSH on Ubuntu
• RDP open with a hostname matching a generic WIN-(randomish string)
  Possible NAT to a windows box elsewhere or a VirtualBox VM running Windows
• ngrok tunneling service
• SMB
• winrm

Protip: For a double Shodan win, their black friday sale is criminally cheap

If you have greater visibility (Passive DNS+ service)

• Default Cobalt Strike SSL Cert with SHA-1 hash 6ece5ece4192683d2d84e25b0ba7e04f9cb7eb7c
• SSL Certs with no locality information

Based on the OSINT

• Probably Cobalt Strike. Follow this lead

Back to volatility to further inspect memory

• yarascan
• cobalt strike plugin
• strings
  • Make sure your strings are very targeted, this is a very expensive process

Cobalt Strike yara (JPCERT)

rule CobaltStrike_JPCERT {
                    strings:
                       $v1 = { 73 70 72 6E 67 00}
                       $v2 = { 69 69 69 69 69 69 69 69}
                    condition:
                       $v1 and $v2
}

vol.py -f memdump.mem --profile=Win2008R2SP1x64_23418 yarascan -y /tmp/cobalt.yar

Cobalt Strike JPCERT Plugin

vol.py --plugins=/plugings/cobalt -f memdump.mem --profile=Win2008R2SP1x64_23418 cobaltstrikeconfig -p 5352

• This yara definition is included in docker pull jbeley/loki

Lateral movement

• Pull Memory
• Windows Event (evtx) logs
• Windows prefetch artifacts

Plaso for making a timeline out of most any Windows artifacts

• We can send this to plaso and walk through it chronologically
  • Event logs
  • Prefetch
  • MFT/filestat
  • IIS
  • Recycle Bin

amcache,filestat,mft,prefetch,recycle_bin_info2,winevtx,winiis

Tradecraft tools

• loki IOC scanner
  • Uses yara and logic to find attacker tools on disk

Loki Sample findings

○ Plain text, weakly encrypted (AES-32) . Looks like the local admin password (which was cracked in 0.068 secs) was  xxxxxxx  
§ FILE: /data/System Volume Information/DFSR/Private/XXXXXXXXXXXXXXXXXXXXXXXX.xml SCORE: 50 TYPE: XML SIZE: 560
FIRST_BYTES: xxxxxxxxx / <?xml version="1.0"
MD5: xxxxxxxxxxxxxxxxxxx
SHA1: xxxxxxxxxxxxxxxxxxx
SHA256: xxxxxxxxxxxxxxxxxxx   
CREATED: Tue Mar 15 14:43:10 2016  
MODIFIED: Thu Nov  8 03:03:44 2012  
ACCESSED: Wed Nov  7 20:02:55 2012
REASON_1: Yara Rule MATCH: Groups_cpassword SUBSCORE: 50
DESCRIPTION: Groups XML contains cpassword value, which is decrypted password - key is in MSDN http://goo.gl/mHrC8P  
REF: http://www.grouppolicy.biz/2013/11/why-passwords-in-group-policy-preference-are-very-bad/  
MATCHES: Str1:  cpassword="L Str2:  changeLogon= Str3:  description= Str4:  acctDisabled=

• Why is this bad?

• Ease of using gpprefdecrypt.py

time python gpprefdecrypt.py xxxxxxxx
MyVoiceIsMyPasswordVerifyMe

real    0m0.269s
user    0m0.168s
sys     0m0.040s

Checking for packaging in preparation for exfiltration

• scalpel

Using a customized scalpel.conf to define the file headers to seek

        7z      y       10000000        \x37\x7a\xbc\xaf\x27\x1c
        zip     y       10000000        PK\x03\x04      \x3c\xac
        rar     y       10000000        Rar!

• Installed by default on SIFT workstation and can be installed on OSX with brew

• We carved an 80GB image in just over 5 minutes

• Why are carved zip files more likely to be complete using this process?

Cheatsheets

Descrption	Command
zgrep for zip	`unzip -p zipfile.zip
zgrep for 7z	`7za x -so
Poor man's parallel	`
Find files	find /mounted/directory -iname "*PF" -ls
syncing files	rsync --inplace --partial --progress --stats -avz [email protected]:/jails/CASENAME/upload/ /LOCAL/
Listing 7z	7za l FILENAME.7z
Listing zip	unzip -v FILENAME.7z
Listing rar	unrar l FILENAME.7z
Dumping hex	`xxd FILENAME

Links

imount GitHub

scalpel.conf

byobu

cobalt strike plugin

passwords in GPO files

gpprefdecrypt.py

Questions