Showing posts with label linux. Show all posts
Showing posts with label linux. Show all posts

Thursday, 9 July 2020

Linux Authentication and Access - A different approach

A small briefing that covers the basic points of creating a new user in a Linux host. We start with a simple username/password old and insecure authentication method and then we add RSA certificate to make the user authentication modern and more robust. In the second section, we copy the private key to our windows machine and we configure windows PuTTY so that we connect to the Linux server using RSA certificates.
In the second part of the article, we make a very short introduction to Chef Technology and how this can be used in order to automate user provisioning on a host. This example does exactly the same user provisioning operations shown in the first part but this time with Chef.
In the last section we introduce we introduce a simple Jenkins pipeline that demonstrates the use of the sftp and ssh commands for the linux user we previously created. 
Just 10 minutes reading.

Read the full article here:

Thursday, 17 October 2019

Elementary Linux Performance Monitoring

The basic tool here is top
Monitoring a single process can be done with -p option, in the next example we measure the MySQL process:

[root@(db-master) ~]# top -p 2521
top - 15:42:54 up 40 days, 10:46,  4 users,  load average: 0.14, 0.24, 0.48
Tasks:   1 total,   0 running,   1 sleeping,   0 stopped,   0 zombie
%Cpu0  :  1.0 us,  1.0 sy,  0.0 ni, 98.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu1  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem:  32551020 total, 32285684 used,   265336 free,   149660 buffers
KiB Swap:  3129340 total,   402572 used,  2726768 free. 16662620 cached Mem

 2521 mysql     20   0 18.725g 0.014t   4548 S 6.000 46.50   2735:03 mysqld

Load Average is a linux/unix mystery: Linux load averages are "system load averages" that show the running thread (task) demand on the system as an average number of running plus waiting threads. This measures demand, which can be greater than what the system is currently processing. 
For an extended excellent article around Linux Load Average, refer to Brendan Gregg's Blog

On the other hand good old ps which is available on all UNIX flavors and LINUX distributions can also help. The following command shows the most CPU consuming processes  in ascending order along with their virtual size 

[root@(db-master) ~]# ps -e -o pid,pcpu,vsz,comm= | sort -n  --key=3
 1669  0.0 752396 isecespd
 1759  0.0 1561472 isectpd
 2521 52.4 19634584 mysqld

To get the process tree try pstree -aAl:

[root@(db-master) ~]# pstree -aAl
systemd --switched-root --system --deserialize 24
  |-VGAuthService -s
  |-agetty --noclear tty1 linux
  |-automount -p /var/run/
  |   `-5*[{automount}]
  |-cron -n
  |-dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation
  |-discagnt /etc/init.d/discagnt start
  |   `-discagnt
  |-haveged -w 1024 -v 0 -F

For systems that do not have  pstree  try ps -ejH  

To get information about threads created by processes  try  ps -eLf

To get information about disk performance try iostat:

 [root@(mmcp_prod_corp)(db-master) ~]# iostat -dcm
Linux 4.4.121-92.117-default (mo-1400a55c2)     10/17/19        _x86_64_        (8 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           7.22    0.00    0.59    1.19    0.00   91.00

Device:            tps    MB_read/s    MB_wrtn/s    MB_read    MB_wrtn
sda               1.56         0.01         0.01      44144      51244
sdb             146.49         5.48         1.79   19159479    6250758

Finally to see all open files by a process such as data/shared objects/dynamic libraries and sockets use lsof. In the following example we can see all open files of mysql process:

[root@(db-master) ~]# lsof -p 2521
mysqld  2521 mysql  cwd    DIR              254,2         4096  6815769 /monsoon/mysql/data
mysqld  2521 mysql  rtd    DIR              254,0         4096        2 /
mysqld  2521 mysql  txt    REG              254,0    250387936   794500 /usr/sbin/mysqld
mysqld  2521 mysql  mem    REG              254,0        97056  1065145 /lib64/
mysqld  2521 mysql  mem    REG              254,0        26976  1065107 /lib64/

To see the TCP listening server sockets on a linux server, we can do that with netstat -tulpn

[root@(db-master) ~]# netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0  *               LISTEN      2521/mysqld
tcp        0      0  *               LISTEN      3282/discagnt
tcp        0      0    *               LISTEN      3289/sshd
tcp        0      0  *               LISTEN      3671/master
tcp        0      0  *               LISTEN      3671/master
tcp        0      0*               LISTEN      38622/0
tcp        0      0 :::7938                 :::*                    LISTEN      3317/nsrexecd
tcp        0      0 :::5666                 :::*                    LISTEN      1/systemd
udp     4352      0    *                           1521/wickedd-dhcp4
udp        0      0*                           3343/ntpd

while for all open TCP sockets:

[root@(db-master) ~]# netstat -t
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 ESTABLISHED
tcp        0     64 mo-1400a55c2.zone1.:ssh ESTABLISHED
tcp        0      0 ESTABLISHED

Tuesday, 12 September 2017

disk-benchmark A mutlipurpose benchmark program that can simulate your application's I/O performance

disk-benchmark tool - get it here!

Sometimes we need to have a prior estimation of I/O performance of a program we plan to develop or we currently posses.
This may be triggered by a number of reasons:
  • Order specific Disk hardware in advance
  • Plan to rent cloud based volume from a cloud provider
  • Estimate the total performance of your application in order to establish operational scenarios and calculate KPIs.
  • Check the cloud providers SLA compliance.
In the past I dealt with all those challenges using standard Linux methods for benchamarking a volume like the classic one:

dd if=/dev/zero of=/root/testfile bs=1G count=1 oflag=direct

Or other similar methods or tools like iostat.

The problem with all those methods, is that you can have an idea of how your disk performs in general, but not according to a given scenario, for example:
  • 20 concurrent users each of them reads and writes of a random file of size between 20k and 1 MB with a pause of 2 seconds for 5 mins.
  • 10 concurrent users each of them reads/ writes a file of 60kb with a pause of 2 seconds after read repeatedly for 100 times. 

Unless you go to very sophisticated tools like JMeter,  you don't really have something very handy. On the other hand, sophisticated tools most of the times, have a significant learning curve but of course in most cases, you want something to use it in the next 5 mins with very simple options just like the above scenarios. To amend this situation, last year, I developed a small C program that can be used to do the job, the disk-benchmark program available on Illumine IT Consulting GitHub URL:

This is a benchmark program to test Hard Drives, SSD Drives, HBAs, RAID Adapters & Storage Controllers. This is a really simple C program that you can compile using the standard GNU/gcc compiler that comes with your Linux distribution.

How to setup the disk-benchmark in your Linux system:
Installation of the disk-benchmark is as simple as this:

# git clone
# cd disk-benchmark/src/
# gcc disk-benchmark.c -o disk-benchmark  -l pthread -lrt  -O3  -Wall
# ls -l disk-benchmark
-rwxr-xr-x 1 root root 23365 Apr 15 10:23 disk-benchmark

A simple scenario implementation using disk-benchmark

Scenario: 10 concurrent users each writing and reading a file of size ~10MB in /var.  Each user pauses for some seconds randomly picked from the interval [2,10] sec.  The command that implements the above scenario has as follows:

[root@mo-8f752419d src]# ./disk-benchmark -p /var -t 10 -a 10000000 -E 2:10

Test scenario:
test path=/var
Threads=10, sleep sec between write/read = 1, repeats per thread=5, random pick sleep sec from [2 10]
Lower file size=1024, Upper file size=10240, Absolute file size=10000000
Read/Write buffer size=8192,  Buff Siz W 0, Buf Siz R 0,
Do write only=0, Delete files=1
Print values only=0 dont print scenario info= 0, dont print clocks=0 dont print headers=0 print date=1
Work Continously=0  Work Continously Sleep Brake=5

T=7, Avg W=0.016134 Avg R=0.002160 Total W=0.080671 Total R=0.010801 Total Time=0.091473 Sleep=4.600000  Avg File Size =10000000.000000
T=2, Avg W=0.014436 Avg R=0.002411 Total W=0.072179 Total R=0.012056 Total Time=0.084234 Sleep=4.800000  Avg File Size =10000000.000000
T=4, Avg W=0.016104 Avg R=0.002189 Total W=0.080520 Total R=0.010943 Total Time=0.091463 Sleep=4.800000  Avg File Size =10000000.000000
T=9, Avg W=0.011966 Avg R=0.002069 Total W=0.059829 Total R=0.010347 Total Time=0.070176 Sleep=4.800000  Avg File Size =10000000.000000
T=6, Avg W=0.013065 Avg R=0.001826 Total W=0.065323 Total R=0.009128 Total Time=0.074451 Sleep=5.000000  Avg File Size =10000000.000000
T=1, Avg W=0.015399 Avg R=0.003005 Total W=0.076996 Total R=0.015025 Total Time=0.092021 Sleep=5.200000  Avg File Size =10000000.000000
T=8, Avg W=0.012883 Avg R=0.002303 Total W=0.064416 Total R=0.011513 Total Time=0.075930 Sleep=5.200000  Avg File Size =10000000.000000
T=3, Avg W=0.015850 Avg R=0.002492 Total W=0.079251 Total R=0.012458 Total Time=0.091709 Sleep=5.400000  Avg File Size =10000000.000000
T=0, Avg W=0.013430 Avg R=0.002697 Total W=0.067151 Total R=0.013487 Total Time=0.080637 Sleep=5.600000  Avg File Size =10000000.000000
T=5, Avg W=0.016659 Avg R=0.002387 Total W=0.083293 Total R=0.011934 Total Time=0.095226 Sleep=5.600000  Avg File Size =10000000.000000

T=-1, Avg W=0.014593 Avg R=0.002354 Total W=0.072963 Total R=0.011769 Total Time=0.084732 Sleep=5.100000  Avg File Size =10000000.000000
Wall time 28.000000, CPU time 0.880000
Tue Sep 12 13:36:26 2017

Monday, 7 March 2016

Fedora 22 Apache Tomcat and Httpd. Publishing an application in minutes.

Recalling from the previous article "Quest of the Holy Cloud" I got a provider and started a simple VM over there.
One of my first actions was to baptize my server and give it a fancy hostname.
Now lets come to the juicy part. In this article I am going to build a simple application server to handle PDF trans-code to images with a custom Java application I built.
The actions I am going to demonstrate are how to:
  • Setup OpenJKD on Fedora 22
  • Install Ghostscript libraries required for my application.
  • Download, install and configure Apache Tomcat 7
  • Install and configure Apache HTTPd.
  • Installing Open JDK

Install OpenJDK

The first step is really easy. We need a JDK or a JRE in order to run Tomcat that hosts our application. The straight option is to use opensource community JAVA: OpenJDK.
To do so, I entered the following commands:
# dnf install java
Last metadata expiration check performed 1:09:31 ago on Mon Mar  7 12:20:26 2016.
To check where java is and what has been installed:
# which java
# java -version
openjdk version "1.8.0_72"
OpenJDK Runtime Environment (build 1.8.0_72-b15)
OpenJDK 64-Bit Server VM (build 25.72-b15, mixed mode)

Install Ghostscript

Most of the software I wrote rely to Ghostscript shared libraries that are called from the corresponding Java API. To install them I entered the following commands:
# dnf install ghostscript
Last metadata expiration check performed 1:15:36 ago on Mon Mar  7 12:20:26 2016.
The library got installed at:
# ls -lh /lib64/libgs*
-rwxr-xr-x. 1 root root 16M Mar 31  2015 /lib64/

# file  /lib64/
/lib64/ ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=6601d742a4829cb3e4fe8197f1b1457f665ce130, stripped

Install Apache Tomcat 7

Apache Tomcat 7 can be downloaded from here as a tar.gz file by picking up a binary distribution as follows:
# cd /opt
# wget
# tar -xvf apache-tomcat-7.0.68.tar.gz

Now tomcat is not provided as a service from Fedora. To do so, we need to create a simple start script in /etc/init.d:

# cd /etc/init.d
# vi tomcat
paste the following to the script tomcat:
# start/ stop Tomcat script
# Since you are using OpneJDK put this as your java home
export JAVA_HOME
export PATH
# Where you have placed tomcat

case $1 in
exit 0
Now tomcat needs to be registered as a Linux service. To do so add those commands:
# cd /etc/init.d
# chmod 755 tomcat  
# chkconfig --add tomcat  
# chkconfig --level 234 tomcat on  
# chkconfig --list tomcat 

Installing Apache HTTPD

This comes as a standard service supported from Fedora distribution. To install it:
# dnf install httpd
For a very fast configuration of http you can edit httpd.conf and add a simple virtual host:
#  vi /etc/httpd/conf/httpd.conf
# add where "Listen 80" is:
Listen My.Host.IP.Here:80

    DocumentRoot "/www/"

    # Other directives here

Since in modern Cloud environments the linux firewall IP Tables may block everything, here are the commands to unlock the ports:
iptables -A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
iptables -A INPUT -p tcp -m tcp --dport 443 -j ACCEPT
You can start the HTTP service and get its status:

# service httpd start
Redirecting to /bin/systemctl start  httpd.service
# service httpd status
Redirecting to /bin/systemctl status  httpd.service
 httpd.service - The Apache HTTP Server
   Loaded: loaded (/usr/lib/systemd/system/httpd.service; disabled; vendor preset: disabled)
   Active: active (running) since Mon 2016-03-07 14:09:27 UTC; 4s ago
 Main PID: 1760 (httpd)
   Status: "Processing requests..."
   CGroup: /system.slice/httpd.service
           ├─1760 /usr/sbin/httpd -DFOREGROUND
           ├─1761 /usr/sbin/httpd -DFOREGROUND
           ├─1762 /usr/sbin/httpd -DFOREGROUND
           ├─1763 /usr/sbin/httpd -DFOREGROUND
           ├─1764 /usr/sbin/httpd -DFOREGROUND
           └─1765 /usr/sbin/httpd -DFOREGROUND

Mar 07 14:09:27 systemd[1]: Starting The Apache HTTP Server...
Mar 07 14:09:27 systemd[1]: Started The Apache HTTP Server.
The deployment directory for tomcat where you can place your WAR files is: /opt/apache-tomcat-7.0.68/webapps/ since I have donwloaded and installed tomcat on /opt.
You can use WinSCP to copy your WAR file there:

# ls -lh  /opt/apache-tomcat-7.0.68/webapps/
total 27M
drwxr-xr-x. 14 root root 4.0K Mar  3 11:00 docs
drwxr-xr-x.  7 root root 4.0K Mar  3 11:00 examples
drwxr-xr-x.  5 root root 4.0K Mar  3 11:00 host-manager
drwxr-xr-x.  5 root root 4.0K Mar  3 11:00 manager
drwxr-xr-x.  3 root root 4.0K Mar  3 11:00 ROOT
drwxr-xr-x.  4 root root 4.0K Mar  4 16:59 zsecure-pdf
-rw-r--r--.  1 root root  27M Mar  4 16:59 zsecure-pdf.war

Friday, 4 March 2016

Set you linux host name and domain

Recently I have created a new VM linux server on CloudSigma. The Server runs Fedora 22. In order to setup the hostname and network domain I have changed the following files:

[root@illumine ~]# cat  /etc/host

[root@illumine ~]# cat /etc/hostname

[root@illumine ~]# cat /etc/hosts   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

178.XXX.XXX.132 securepdf

Test it using ping:

[root@securepdf ~]# ping securepdf
PING (178.XXX.XXX.132) 56(84) bytes of data.
64 bytes from (178.XXX.XXX.132): icmp_seq=1 ttl=64 time=0.036 ms

Tuesday, 5 November 2013

My Linux Shell Cook-Book

How many times we struggle ourselves to remember how to write correctly a for loop in shell ?
When was the last time we did a search for string comparison in bash and test for a file timestamp?

My favourite treatment for weak memory or "shell syntax blackouts" is a simple text file called
MyBashCooks.txt which takes proudly always the first tab in Notepad++ when I am coding scripts.

I think it is void to mention that this small file is hand on the web and travels with me when I am visiting clients, generally, it is one of the very few things I have handy and don't move a second step without it.

The recipes in my cookbook share a common secret: single command line, so you can use them on the command prompt as is or break them in several lines inside your scripts.

I think we say too much on this, please enjoy and if you find a mistake, please drop me a line:

# If then else in a single line
r=ee; if [ "$r" == "" ] ;  then  echo "is null" ; else  echo "not null"; fi

# If then else ARITHMETIC Operations
x=0;if [ $x -eq 0 ]; then echo "x=0";else echo "other"; fi

# Check if word belongs to string and if it does, run a shell command 
# Compatible with all shells
string="this is a string "; word="str";test "${string#*$word}" != "$string" && echo "$word found in $string"

# Negative of the previous
string="this is a string "; word="xxx";test "${string#*$word}" != "$string" || echo "$word not found in $string"

# Combined case of found and not found....
string="this is a string "; word="xxx";test "${string#*$word}" != "$string" && echo "Executes when found." || echo "Executes when NOT found."

# single line while loop
while true; do  echo '----'; sleep 1 ; done

while true; do top -b -n 1  |  egrep  'top|Tasks|Cpu|Mem|Swap|PID|mysql' | grep -v root   ;sleep 10; echo '----' ; done

# single line for loop
for h in {app0,app1,app2,app3,index0,index1,index2,index3};  do echo $h; sleep 1;  done

# Tests the existence  of word in a string: works in all POSIX shells (bash, dash, korn...) 
string="my name is MyBighrase";word="e i"; echo "search [$word] in [$string]";test "${string#*$word}" != "$string" && echo "$word found in $string"

# From UNIX timestamp to date:
echo 1365436826 | gawk '{print strftime("%c", $0)}'

# From Date to timestamp:
date --date='04/04/2013 20:30:01' +"%s"

# Format Date output
date +"%m-%d-%y"

# Search for a string across directories with grep
find -RH theString /this/directory

# Serach for line numbers and cut a file only from line to line:
this one searches for all line numbers with date 20130607
cat  slow.log | grep -n 130607  
we get the lines of this date exists and isolate those numbers in a new file.
cat  slow.log | sed -n '16582739,17369638p' > slow_20130607.txt

# Monitoring Linux Processes

while true ; do top -b -n 1 -p 2163 >> mysql_3000.txt ; sleep 15 ;  done &

while true ; do date >> tmpfs_3000.txt ; df -h | grep mysql ; sleep 15 ;  done >> tmpfs_3000.txt &

while true ; do date >> process_3000.txt;  echo "show full processlist;" |  mysql -uroot -pmypass | grep -v Sleep >> process_3000.txt  ; sleep 15 ; done &