Original subtitles

commandline fun in this module we'll

take an introductory look at a few

popular linux command line programs

please refer to the kali linux training

site for a refresher or more in-depth

discussion

the Bosch environment Bosch is a shell

that allows us to run complex commands

and perform different tasks from a

terminal window it incorporates useful

features from both the corn shell and

sea shell

environment variables when opening a

terminal window a new Bosch process

which has its own environment variables

is initialized these variables are a

form of global storage for various

settings inherited by any applications

that are run during that terminal

session one of the most commonly

referenced environment variables is path

which is a colon separated list of

directory paths that Bosch will search

through whenever a command is run

without a full path we can view the

contents of a given environment variable

with the echo command followed by the

dollar sign character and an environment

variable name for example let's examine

the contents of the path environment

variable some other useful environment

variables include user PWD and home

which hold the values of the current

terminal users user name present working

directory and home directory

respectively we can define an

environment variable with the export

command for example if we are scanning a

target and don't want to type in the

system's IP address repeatedly we can

quickly assign it an environment

variable and use that instead

the export command makes the variable

accessible to any subprocesses we might

spawn from our current bash instance

if we set an environment variable

without export it will only be available

in the current shell let's launch

another instance of bash and try to

display the variable again

since we defined our variable without

the export command we can't access it in

the new bash instance

after exiting back to our original bash

session we have access to the variable

again now let's see what happens when we

use export to define a variable

nothing has changed so far but let's see

what happens when we launch a new Bosch

instance

this time we can access the variable

from the new instance

there are many other environment

variables defined by default in kali

linux we can view these by running env

at the command line take some time to

review an experiment with predefined

environment variables

tab-completion the bash shell

autocomplete function allows us to

complete filenames and directory paths

with the tab key this is such a

time-saver that makes other shells which

lack this feature feel clumsy and slow

let's take a look at how this works from

the callee user home directory

we'll start by typing the following

command when we hit the tab key the

first time after D bash suggests that

there are three directories starting

with that letter then presents our

partially completed command for us since

we decide to specify desktop will type e

followed by the tab key again at this

point the bash shell magically Auto

completes the rest of the word desktop

as this is the only choice that starts

with de refer to your lab guide for more

information

bash history tricks when engaged in a

penetration test it's important to keep

a record of commands that have been

entered into the shell

fortunately bash maintains a record of

commands that have been entered which

can be displayed with the history

command

rather than retyping along command from

our history we can make use of the

history expansion facility for example

each entry in our history is preceded by

a line number to rerun a command we

simply type an exclamation mark followed

by the line number in this case we'll re

execute the command clear which is

identified by line number 32 another

helpful history shortcut is the double

exclamation mark which repeats the last

command that was executed during our

terminal session

by default the command history is saved

to the bash history file in the users

home directory

to environment variables control history

size hist size and hist file size

his size controls the number of commands

stored in memory for the current session

and his file size configures how many

commands are kept in the history file

these variables can be edited according

to our needs and saved to the bash

configuration file dot bash RC that we

will explore later one of the simplest

ways to explore the bash history is

right from the command line prompt we

can browse through the history with some

useful keyboard shortcuts with the two

most common being the up arrow and down

arrow the up arrow Scrolls backwards in

history and the down arrow Scrolls

forward last but not least holding down

ctrl and pressing R will invoke the

reverse I search facility if we type a

letter for example C we'll get a match

for the most recent command in our

history that contains the letter C we

can keep trying to narrow down our match

and when we find the desired command

we'll press ENTER to execute it bash has

a number of history features that can

save a great deal of time and repetitive

typing

typing and redirection every program run

from the command line has three data

streams connected to it that serve as

communication channels with the external

environment these streams are defined as

standard input which is data fed into

the program standard output which is

output from the program and defaults to

the terminal and standard error which

contains error messages and are also

displayed by default in the terminal

typing using the pipe operator and

redirection using the left and right

angle bracket operators also known as

Chevron's or greater than and less than

signs connect these streams between

programs and files to accommodate a near

infinite number of possible use cases

redirecting to a new file

in the previous command examples the

output was printed to the screen

this is convenient most of the time but

we can use the right angle bracket

operator to save the output to a file to

keep it for future reference or

manipulation if we redirect the output

to a non-existent file the file will be

created automatically

however if we save the output to a file

that already exists that files content

will be replaced

be very careful with three direction

there is no undo function

redirecting to an existing file to

append additional data to an existing

file as opposed to overwriting the file

use the double right angle bracket

operator

once again be sure to exercise caution

with three directions so you don't

overwrite anything important

redirecting from a file as you may have

guessed we can use the left angle

bracket operator to send data the other

way in this example will redirect the WC

commands standard input with data

originating from the file we generated

in the previous section let's try this

with W cm which counts characters in the

file

note that this effectively connected the

contents of our file to the standard

input of the WC - M command

redirecting standard error according to

the POSIX specification the file

descriptors for standard input standard

output and standard error are defined as

zero one and two respectively these

numbers are important as they can be

used to manipulate the corresponding

data streams from the command line while

executing or joining different commands

together to get a better grasp of how

the file descriptor numbers work

consider this example which redirects

the standard error

note that error dot txt only contains

the error message generated on standard

error we did this by prepending the

stream number to the right angle bracket

operator where two corresponds to

standard error

typing continuing with the example using

the WC command let's have a look at how

to redirect the output from one command

into the input of another

we'll use the pipe character to redirect

the output of the cat command to the

input of the WC command

this concept may seem trivial but piping

together different commands is a

powerful way to manipulate all sorts of

data

text searching and manipulation in this

section we'll gain efficiency with file

and text handling by introducing a few

commands grep said cut and awk

advanced usage of some of these tools

requires a good understanding of how

regular expressions work

grep in a nutshell grep searches text

files for the occurrence of a given

regular expression and outputs any line

containing a match to standard output

which is usually the terminal screen

consider the following example

we listed all the files in the user bin

directory with LS and pipe the output

into the grep command which searches for

any line containing the string zip some

of the most commonly used switches

include - R for recursive searching and

- I to ignore the case of the text refer

to the grep man page for more

information about various options

understanding the grep tool and when to

use it can prove incredibly useful

said said is a powerful stream editor

it's also very complex so we'll only

briefly scratch the surface here at a

high level said performs text editing on

a stream of text either a set of

specific files or standard output let's

look at an example

we created a stream of text using the

echo command and then piped it to said

in order to replace the word hard with

harder note that by default the output

of the entire command has been

automatically redirected to standard

output or the terminal screen

cut the cut command is simple but often

comes in quite handy cut is used to

extract a section of text from a line

and output it to standard output some of

the most commonly used switches include

- F for the field number we are cutting

and - d to define the field delimiter

here we echoed a line of text and piped

it to the cut command to extract the

second field using a comma as a field

delimiter the same command can be used

to cut lines found in text files this

time a list of users is extracted from

Etsy password by using a colon as a

delimiter and retrieving the first field

Ark Ark is a programming language

designed for text processing and is

typically used as a data extraction and

reporting tool it is also extremely

powerful and can be quite complex so

again we will only scratch the surface

here a commonly used switch with awk is

- capital F which is the field separator

and the print command which outputs the

result text here we echoed a line and

piped it to awk

to extract the 1st and 3rd fields using

a double colon as a field separator the

most prominent difference between the

cut and awk examples we used is that cut

can only accept a single character as a

field delimiter while awk is much more

flexible

as a general rule of thumb use a single

op command instead of multiple cut

operations

practical example let's take a look at a

practical example that ties together

many of the commands we've explored so

far we've been given an apache HTTP

server log that contains evidence of an

attack our task is to use bash commands

to inspect the file and discover various

pieces of information such as who the

attackers were and what exactly happened

on the server

first we'll use the head and WC commands

to take a quick peek at the log file in

order to understand its structure

the head command displays the first 10

lines in a file and the WC command along

with the - L option displays a total

number of lines in the file notice that

the log file is text-based and contains

different space delimited fields like IP

address timestamp HTTP request etc this

is a perfectly grep friendly file and

will work well for all the tools we've

covered so far

we'll begin by searching through the

HTTP requests made to the server for all

the IP addresses recorded in this log

file to do this we'll cap the log file

and pipe the output into the cut and

sort commands this might give us a clue

about the number of potential attackers

were dealing with here we see that less

than 10 I P addresses were recorded in

the log file although this still doesn't

tell us anything about the attackers

next we'll use unique and sort to show

unique lines further refine our output

and sort the data by the number of times

each IP address access the server the -

C option of unique will prefix the

output line with the number of

occurrences a few IP addresses stand out

but we'll focus on the address that has

the highest access frequency first

to filter out the suspicious IP address

and display and count the resources that

were being requested by that IP we can

use the following sequence

from this output it seems that the IP

address was accessing the admin

directory exclusively let's inspect this

further

apparently the suspicious IP has been

involved in an HTTP brute-force attempt

against the web server we can verify

this with greps - V flag to reverse the

search and only show lines that do not

contain the word admin as we can see our

log files contain no such entry

furthermore after about a thousand

attempts it seems like the brute-force

attempt succeeded as indicated by the

HTTP 200 message this is a good example

that ties together many of the basic

commands we've explored

editing files from the command-line next

let's take a look at file editing in a

command shell environment this is an

extremely important linux skill

especially if you happen to get access

to a unix-like OS during a penetration

test although there are text editors

like G edit and leafpad that might be

more visually appealing due to their

graphic user interface we will focus on

text-based terminal editors which

emphasize both speed and versatility

everyone seems to have a preference when

it comes to text editors but we will

cover basic usage for the two most

common options Nano and VI

nano nano is one of the simplest to use

text editors to open a file and begin

editing simply run nano passing a file

name as an optional argument once the

file is opened we can immediately start

making any required changes to the file

just as we would in a graphical editor

the Nano command menu is located at the

bottom of the screen some of the most

used commands to memorize include ctrl o

to write changes to the file ctrl K to

cut the current line and control you to

uncut a line and paste it at the cursor

location ctrl W is used to search within

the file and finally ctrl X is used to

exit

for additional information regarding

nano refer to its online documentation

VI VI is an extremely powerful text

editor capable of blazing speed

especially when it comes to automating

repetitive tasks however it has a

relatively steep learning curve and is

nowhere near as simple to use as nano

due to its complexity we will only cover

the very basics here as with nano to

edit a file simply pass its name as an

argument to VI

once the file is opened enable insert

text mode to begin typing to do this

press the i key and start typing away to

disable insert text mode and go back to

command mode press the Escape key while

in command mode use DD to delete the

current line press YY to copy the

current line and press P to paste the

clipboard contents to delete the current

character under the cursor press the X

key to write the current file to disk

and remain in V I type : w

type : Q followed by an exclamation mark

to quit without writing the file to the

disk to quickly save the file and quit

VI enter : WQ exclamation point

because VI seemed so awkward to use many

users avoid it however from a

penetration testers point of view VI can

save a great deal of time in the hands

of an experienced user and is installed

on every POSIX compliant system

comparing files file comparison may seem

irrelevant but system administrators

network engineers penetration testers IT

support technicians and many other

technically oriented professionals rely

on this skill fairly often in this

section we'll take a look at a couple of

tools that can help streamline the often

tedious but rewarding process of file

comparison

calm the calm command compares to text

files displaying the lines that are

unique to each one as well as the lines

they have in common

in the first example comm display the

unique lines in scan - a text the unique

lines in scan - B dot text and the Lions

found in both files respectively

comm outputs three space offset columns

the first contains lines that are unique

to the first file or argument the second

contains lines that are unique to the

second file or argument and the third

column contains lines that are shared by

both files

the - n switch where n is either 1 2 or

3 can be used to suppress one or more

columns depending on the need let's take

a look at an example

this time come displayed only the lines

that were found in both files since we

suppress columns 1 & 2

diff the diff command is used to detect

differences between files similar to the

comm command however diff is much more

complex and supports many output formats

two of the most popular formats include

the context format and the unified

format display results in context format

use the - see option this output uses

the - indicator to show that the line

appears in the first file but not in the

second conversely the + indicator shows

that the line appears in the second file

but not in the first now let's see what

the unified format returns

the most notable difference between

these formats is that the unified format

does not show lines that match between

files making the results shorter the

indicators however have identical

meaning in both formats

vim diff vim diff opens vim an extended

version of VI with multiple files one in

each window the differences between the

files are highlighted which makes it

easier to visually inspect them there

are a few shortcuts that might be useful

for example pressing ctrl W followed by

an arrow key allows us to switch between

windows pressing the right bracket and

the letter C will jump to the next

change in the diff and the left bracket

followed by the letter C will jump to

the previous change the do key

combination will get a change from the

other window and put it in the current

one

combination we'll put the change from

the current window into the other one to

quit vim diff use the same shortcuts as

VI

managing processes the Linux kernel

manages multitasking through the use of

processes the kernel maintains

information about each process to help

keep things organized and each process

is assigned a number called a process ID

the Linux shell also introduces the

concept of jobs to ease our workflow

during a terminal session for example

this is a pipeline of two processes

which the shell considers a single job

job control refers to the ability to

selectively suspend the execution of

jobs and resume their execution at a

later time

we can achieve this with the help of

specific commands which will soon

explore

background is the previous jobs in this

module have been run in the foreground

which means the terminal is occupied and

no other commands can be executed until

the current one finishes since most of

our examples have been short and sweet

this hasn't caused a problem we will

however be running longer and more

complex commands in later modules which

we can send to the background in order

to regain control of the terminal and

execute additional commands the quickest

way to background a process is to append

an ampersand to the end of the command

to send it to the background immediately

after it starts let's try a brief

example we'll send 400 ICMP echo

requests to the local interface with the

ping command and write the results to a

file the execution automatically runs in

the background

leaving the shell free for additional

operations but what would have happened

if we had forgotten to append the

ampersand at the end of the command this

command will run in the foreground and

we are forced to either cancel the

command with control C or wait until the

command finishes to regain control of

the terminal the other option is to

suspend the job using control Z after

it's already started once a job has been

suspended we can resume it in the

background using the BG command the job

is now running in the background and we

can continue using the terminal as we

wish

while doing this keep in mind that some

processes are time sensitive and may

give incorrect results if left suspended

for too long for instance in the ping

example the echo reply may come back but

if the process is suspended when the

packet comes in the process may miss it

leading to incorrect output always

consider the context of what the

commands you are running are doing when

engaging in job control

control jobs and FG to quickly check on

the status of our ICMP echo requests we

need to make use of two additional

commands jobs and FG let's begin by

starting and suspending to long-running

commands

you

the built-in jobs utility lists the jobs

that are running in the current terminal

session running the F G command followed

by a % and job number returns a job to

the foreground will use control-c to

terminate this ping process and run jobs

again

with only one job running we can run FG

without any additional arguments to

return it to the foreground

specific jobs can also be referenced by

their process ID number or command name

process control PS and kill one of the

most useful commands for monitoring

processes on almost any unix-like

operating system is PS which is short

for process status

unlike the jobs command PS lists

processes system-wide not only for the

current terminal session PS is

considered a standard on unix-like os's

and its name is so well recognized that

even on windows powershell PS is a

predefined command alias for the get

process command --let which essentially

serves the same purpose

as a penetration tester one of the first

things to check after obtaining remote

access to a system is the software

currently running on the compromised

machine this could provide clues to help

elevate our privileges or collect

additional information in order to

acquire further access into the network

as an example let's start the leafpad

text editor

now let's try to find the leafpad

process ID from the command line using

the PS command

we'll run PS with the - EF options where

e selects all processes and F displays a

full format listing finding our leafpad

application that massive listing is

definitely not easy but since we know

the application name were looking for we

can replace the - e switch with -

capital C and specify the command name

which is leafpad in our case the process

search has returned a single result

which contains leaf pads process ID take

some time to explore the PS manual as it

really is the Swiss Army knife of

process management let's say we now want

to stop the leaf pad process without

interacting with the GUI the kill

command can help us here as its purpose

is to send a specific signal to a

process the kill command requires the

PID of the process we want to send the

signal to since we've already gathered

leaf pads process ID we can proceed

because the default signal for kill is

sig term or request termination our

application has been terminated we can

verify this with PS after killing leaf

pad

file and command monitoring it's

extremely valuable to know how to

monitor files and commands in real time

during the course of a penetration test

two commands that help with this task

include tail and watch

tail

most common use of tail is to monitor

log file entries as they are being

written

for example we may want to monitor the

Apache logs to see if a web server is

being contacted by a given client we are

attempting to attack with a client-side

exploit the dash F option is very useful

it continuously updates the output as

the target file grows since our local

Apache web server is already running

will visit the default page using

Firefox

if we go back to our terminal we should

see the requests made in the log file

another convenience which is - NX which

outputs the last X number of lines

instead of the default value of 10 tail

has a number of other useful options

outlined in its mane page

watch the watch command is used to run a

designated command at regular intervals

by default it runs every two seconds but

we can specify a different interval by

using the dash NX option to have it run

every X number of seconds

for example this command will list

logged in users output from the W

command once every 5 seconds to

terminate the watch command and return

to the interactive terminal use

control-c

downloading files next let's take a look

at some tools that can download files to

a Linux system from the command line

w get the W get command which we use

extensively downloads files using the

HTTP and FTP protocols W get can be used

along with the - capital o switch to

save the destination file with a

different name on the local machine

w get has many features and can do far

more than simply download files

refer to its documentation for more

information

Curl Curl is a tool used to transfer

data to or from a server using a host of

protocols a penetration tester can use

curl to download or upload files and

build complex requests its most basic

use is very similar to W get but with a

lowercase o to specify the output file

name

curl is incredibly versatile and you'll

find ample documentation and example

usage online

axel axel is a download accelerator that

transfers a file from an FTP or HTTP

server through multiple connections this

tool has a vast array of features but

the most common option is dash n which

is used to specify the number of

multiple connections to use we also like

using the dash a option for a more

concise progress indicator and - o to

specify a different file name for the

downloaded file

axl speeds up downloads extremely well

and is especially useful for large

downloads

customizing the Bosch environment

bash history customization earlier in

this module we discussed environment

variables and the history command we can

use a number of environment variables to

change how the history command operates

and returns data the most common include

his to control hist ignore and hist time

format the hist control variable defines

whether or not to remove duplicate

commands commands that begin with spaces

from the history or both by default both

are removed but you may find it more

useful to only omit duplicate commands

the histogram is particularly useful for

filtering out basic commands that are

run frequently such as LS exit history

BG etc let's run a few commands to see

if history ignores our specified

commands

you

as expected the ls and history commands

do not appear in our command history

lastly hist time format controls date

and/or time stamps in the output of the

history command

in this example we used percent capital

F or year-month-day ISO 8601 format and

percent capital T 24-hour time

other formats can be found in the strf

time man page

alias an alias is a string we can define

that replaces a command name aliases are

useful for replacing commonly used

commands and switches with a shorter

command or alias that we define in other

words an alias is a command that we

define ourselves built from other

commands for example we typically tend

to use LS - la to display results in a

long list including hidden files let's

take a look at how we can use an alias

to replace this command by defining our

own command LS a we can quickly execute

LS - la without having to type any

arguments at all we can see the list of

defined aliases by running alias without

arguments

persistent bash customization the

behavior of interactive shells in bash

is determined by the system-wide bash RC

file located in slash Etsy slash bash

bash RC the system-wide bash settings

can be overridden by editing the dot

bash RC file located in any users home

directory in the previous section we

explored the alias command which sets an

alias for the current terminal session

we can also insert this command into the

dot bash RC file in a user's home

directory to setup persistent alias the

dot bash RC script is executed anytime

that user logs in since this file is a

shell script we can insert any command

that could be executed from the command

prompt let's examine a few lines of our

default bash RC file

you might recognize the hist size and

hist file size environment variables

and the alias command that displays

colored output you will also find some

other aliases in this file there

commented out

wrapping up in this module we took an

introductory look at a few popular linux

command line programs remember to refer

to the kali linux training site for a

refresher or more in-depth discussion