My Linux Shell Cook-Book

How many times we struggle ourselves to remember how to write correctly a for loop in shell ?
When was the last time we did a search for string comparison in bash and test for a file timestamp?

My favourite treatment for weak memory or "shell syntax blackouts" is a simple text file called
MyBashCooks.txt which takes proudly always the first tab in Notepad++ when I am coding scripts.

I think it is void to mention that this small file is hand on the web and travels with me when I am visiting clients, generally, it is one of the very few things I have handy and don't move a second step without it.

The recipes in my cookbook share a common secret: single command line, so you can use them on the command prompt as is or break them in several lines inside your scripts.

I think we say too much on this, please enjoy and if you find a mistake, please drop me a line:

# If then else in a single line

r=ee; if [ "$r" == "" ] ;  then  echo "is null" ; else  echo "not null"; fi

# If then else ARITHMETIC Operations

x=0;if [ $x -eq 0 ]; then echo "x=0";else echo "other"; fi

# Check if word belongs to string and if it does, run a shell command 
# Compatible with all shells

string="this is a string "; word="str";test "${string#*$word}" != "$string" && echo "$word found in $string"

# Negative of the previous

string="this is a string "; word="xxx";test "${string#*$word}" != "$string" || echo "$word not found in $string"

# Combined case of found and not found....

string="this is a string "; word="xxx";test "${string#*$word}" != "$string" && echo "Executes when found." || echo "Executes when NOT found."

# single line while loop

while true; do  echo '----'; sleep 1 ; done

while true; do top -b -n 1  |  egrep  'top|Tasks|Cpu|Mem|Swap|PID|mysql' | grep -v root   ;sleep 10; echo '----' ; done

# single line for loop

for h in {app0,app1,app2,app3,index0,index1,index2,index3};  do echo $h; sleep 1;  done

# Tests the existence  of word in a string: works in all POSIX shells (bash, dash, korn...) 

string="my name is MyBighrase";word="e i"; echo "search [$word] in [$string]";test "${string#*$word}" != "$string" && echo "$word found in $string"

# From UNIX timestamp to date:

echo 1365436826 | gawk '{print strftime("%c", $0)}'

# From Date to timestamp:

date --date='04/04/2013 20:30:01' +"%s"

# Format Date output

date +"%m-%d-%y"

# Search for a string across directories with grep

find -RH theString /this/directory

# Serach for line numbers and cut a file only from line to line:

this one searches for all line numbers with date 20130607

cat  slow.log | grep -n 130607  

we get the lines of this date exists and isolate those numbers in a new file.

cat  slow.log | sed -n '16582739,17369638p' > slow_20130607.txt

# Monitoring Linux Processes

while true ; do top -b -n 1 -p 2163 >> mysql_3000.txt ; sleep 15 ;  done &

while true ; do date >> tmpfs_3000.txt ; df -h | grep mysql ; sleep 15 ;  done >> tmpfs_3000.txt &

while true ; do date >> process_3000.txt;  echo "show full processlist;" |  mysql -uroot -pmypass | grep -v Sleep >> process_3000.txt  ; sleep 15 ; done &

Digging into MySQL InnoDB Transactions

A week ago I faced a really difficult situation: one of our MySQL innodb databases was slowing down in response times really a lot. With a bit of debugging, I figure out that there were some X-locks (eXclusive locks) that did not allowed some of the later transactions to finish. Here is the procedure I followed:

1. Use MySQL processlist to find out slow operations. The first one show us the slow queries on a specific database:

select * from information_schema.processlist 
              where db='alfresco' and command='Query' order by 'time' desc; 

Since I understood that the specific slow queries are utilizing two tables I also used:

select * from information_schema.processlist 
      where db='alfresco' and info like '%alf_%_assoc%' 

2. Now since we have one idea about heavy MySQL threads, lets take a look to the transactions:
First of all see all transactions that have placed an exclusive lock on one of the tables:
Second find the lock wait transactions

select * from information_schema.INNODB_TRX 
         where trx_id in ( 
             select Lock_trx_id from information_schema.INNODB_LOCKS where lock_mode='X'  )

select * from information_schema.INNODB_TRX where trx_id in (
         select distinct blocking_trx_id 
            from information_schema.INNODB_LOCK_WAITS ) 

3. Finally find the threads along with conflicting transactions with the following query:

SELECT r.trx_id waiting_trx_id,  
       r.trx_mysql_thread_id waiting_thread,
       r.trx_query waiting_query,
       b.trx_id blocking_trx_id, 
       b.trx_mysql_thread_id blocking_thread,
       b.trx_query blocking_query
   FROM       information_schema.innodb_lock_waits w
   INNER JOIN information_schema.innodb_trx b  ON  
    b.trx_id = w.blocking_trx_id
  INNER JOIN information_schema.innodb_trx r  ON  
    r.trx_id = w.requesting_trx_id;

Beware: The thread ID (waiting and blocking) is the ID in the information_schema.processlist

In this case, clearly the transaction with id 194D110F is the blocker!
To inspect the transaction use this query:

select * from information_schema.INNODB_TRX where trx_id='194D110F'

From what we see, there is a transaction running since 19:10:57 with id 194D110F. The query or operation carried out of this transaction is null because it was deleted from the transaction cache since it is very old. To retrieve the information about the SQL statement carried from this transaction we have to use the MySQL binlog.

The transaction 194D110F was observed during a  load test performed on our system. The lock was not released for 2 h.
The locking thread in MySQL was seen as follows from the slow.log:

---TRANSACTION 194D110F, ACTIVE 7431 sec
74 lock struct(s), heap size 14776, 101 row lock(s), undo log entries 64
MySQL thread id 7526, OS thread handle 0x7fd28ec38700, query id 10005877 application-host X.X.44.28 alfresco
Trx read view will not see trx with id >= 194D1110, sees < 194C03CD