Tuesday, January 19, 2010

wget through proxy

export http-proxy=http://user:passwd@proxy:port
wget "http://URL AND FILE"

Show files and dirs in tree view [Linux tree command :P ]

ls -R | grep ":$" | sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^/ /' -e 's/-/|/'

Add/Remove mysql users..

add Users and Password

GRANT ALL PRIVILEGES ON *.* TO 'root'@'localhost' IDENTIFIED BY 'password' WITH GRANT OPTION;
GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'password' WITH GRANT OPTION;

remove users

mysql> show grants for idiot@localhost;
mysql> revoke all privileges, grant option from idiot@localhost;
mysql> drop user idiot@localhost;FLUSH PRIVILEGES;

Mount ISO file.

mount -o loop disk1.iso /mnt/disk4

Monday, January 18, 2010

linux proxy setting

export http_proxy='http://proxy:3128'
export http_proxy='http://username:pasword@proxy:3128'

Lock root

usermod -L root

Grep

grep more values

less filename grep -E "Rasika|rasika|Rasika"

grep content inside files

grep -ir "Rasika" ./

If df -h shows wrong space.

If you see df -h shows wrong output after you deleted some files. Use following command

ls -ld /proc/*/fd/* 2>&1 fgrep '(deleted)'

to see which links are not removed after removing files and remove them manually.
This will happen usually when you try to remove running logs like apache logs.

Creating logfile archives with logrotate

Many of the services installed on Linux machines will produce logfiles which grow, and grow, and grow. If left unchecked you can easily fill a disk with a large collection of logfiles if you're not careful.
The most common method of keeping logfile growth in check is to use logrotate, and many Debian packages are setup to work with this by default.
The most obvious package which uses it is Apache, the webserver, which by default keeps its logfiles in the directory /var/log/apache (or /var/log/apache2).
If you examine this directory you will see that there are a bunch of logfiles which are archived:
root@mystery:~# ls -l /var/log/apache2/
access.log
access.log.1
access.log.2.gz
access.log.3.gz
access.log.4.gz
access.log.5.gz
error.log
error.log.1
error.log.2.gz
error.log.3.gz
error.log.4.gz
error.log.5.gz

Here the current logfiles access.log, error.log are kept raw as are yesterday's logfiles (access.log.1 and error.log.1). Previous logfiles are compressed with gzip and only kept for five weeks. (I know it's five weeks and not five days because I've looked at the configuration - It's not clear from this output though!)
The process that is in charge of compressing and rotating these logfiles is called logrotate and it is executed once per day upon Debian installations.
As we saw when we were looking at scheduling commands with cron there is a directory called /etc/cron.daily which contains scripts which are executed once per day. Here you will find the logrotate driver script.

Every day this script runs and examines two things:

01. The configuration file /etc/logrotate.conf
02.The configuration directory /etc/logrotate.d

The latter is where most of our packages are configured. This directory contains configuration files which other packages have installed. For example if you install apache the file /etc/logrotate.d/apache will be installed.
Many servers such as exim the mailserver will install their own configuration file, and you can add your own.
A typical logrotate configuration file looks like this:

/var/log/apache/*.log {
weekly
missingok
rotate 52
compress
delaycompress
notifempty
create 640 root adm
sharedscripts
postrotate
if [ -f /var/run/apache.pid ]; then
/etc/init.d/apache restart > /dev/null
fi
endscript
}

You can see several important things here. The most obvious is the list of files that will be matched by this configuration file:

/var/log/apache/*.log {
...
}

After this we have a collection of configuration terms, a different one on each line. In the example above we have:

weekly

The files should be rotated every week. Opposite: daily

rotate nn

We should keep no more than nn files.

compress

Compress older files with gzip. Opposite: nocompress

delaycompress

Don't compress yesterdays files. Opposite: compress

notifempty

Don't do any rotation if the logfile is empty. Opposite: ifempty

create xx user group

If we have to create the new file give it the given mode, owner, and group.

sharedscripts

Run any given prerotate or postrotate script for each logfile individually. Opposite:nosharedscripts.

postrotate + endscript

Anything between these is executed after the rotation process. Opposite : prerotate

Hopefully that should have made sense!

The upshot of this script is that any file which matches /var/log/apache/*.log is rotated every week, compressed, if it's non-empty. The new file is created with the file mode of 640, and after the rotation has finished the server is restarted.
If we wish to install a local service which creates a logfile we can cause it to be rotated very easily, just by adding a new logrotate configuration file.
Assuming we have a new service "fred" which produces its output in /var/log/fred/output.log we can cause this to be rotated every day with a script like this:

/var/log/fred/*.log {
daily
missingok
rotate 7
compress
delaycompress
create 640 fred fred
sharedscripts /etc/init.d/fred restart
endscript
}

This will:
Run daily.
Keep no more than 7 days worth of logfiles at any one time.
Not complain if there is a logfile missing.
Compress the older files, but not yesterdays.
Create the new logfiles as being owned by the user and group fred.
Restart the service after rotating the logfiles.

Any of the existing files in the logrotate directory can be examined for more examples - and the manpage documents all the options you may use in a clear manner:

logrotate --debug --force nameofyourfile

Addin Users and set permissions.

CREATING AND ADDING PERMISSIONS
Adding New Users
#useradd testuser
#passwd testuser
 
Create group
#groupadd testgroup
 
Add User to group
#usermod testgroup testuser
 
List file and directory permissions
#ll
#ls -l
Example;
#ls -l myfile
-rwxr-x--- 1 george administrators 10 2006-03-09 21:31 myfile
Its name, "myfile";
Its permissions, "-rwxr-x---";
Its owner, "george";
Its group, "administrators";
drwxr-xr-x 2 root root 4096 Oct 16 11:06 Desktop
"d" says it is a directory.
 
Letter Permission
r Read
w Write
x Execute, Go through (for directories)
- No permission
 
Letter Type of users
 
u User (owner of the file)
g Group (group to which belong the file)
o Other (users who are neither a member of the Group nor the owner of the file)
a All (everybody)
 
 
Specify permissions
 
#chmod g+w myfile
g represents the group of the file (administrators).
w represents the write permission.
+ represents the fact that the permission is added.

#chmod o+r myfile (adds read permission to the others on myfile)

#chmod ug+rx myfile (adds read and execute permissions to both the owner (user) and the group on myfile)

#chmod a-rwx myfile (removes all permissions to everybody (all) on myfile)

#chmod a=rx *.txt (defines permissions to be read and write to everybody on all files suffixed by .txt)
 
Specify permissions using numbers
 
Permission Value
- 0
x 1
w 2
r 4
ex: #chmod +x testfile (Set executable permissions)
Permission Value
--- 0
--x 1
-w- 2
-wx 3
r-- 4
r-x 5
rw- 6
rwx 7
#chmod 755 myfile (rwxr-xr-x, all rights to the owner, other people only read and execute)
#chmod 644 myfile (rw-r--r--, owner car read and write, other people only read)
#chmod 777 myfile (can be considered bad practice in some cases, full permissions to everybody.)
 
Changing file owner or group : chown, chgrp
#chown testuser myfile
#chgrp testgroup myfile
#chown testuser:testgroup myfile
For tmp folders...
#ls -l
drwxrwxrwt 10 root root 4096 2006-03-10 12:40 tmp
The "t" in the end of the permissions is called the "sticky bit". It replaces the "x" and indicates that in this directory, files can only be deleted by their owners, the owner of the directory or the root superuser. This way, it is not enough for a user to have write permission on /tmp, he also needs to be the owner of the file to be able to delete it.
In order to set or to remove the sticky bit, use the following commands:
chmod +t tmp
chmod -t tmp

Apache Load test

Using Apache bench mark tool (Linux).

ab -n 10000 -c 1000 http://url/index.html

-c Concurrency
-n Number of requests

load test through proxy server

ab -n 10000 -c 1000 -X 172.16.11.17:8118 http://172.16.13.100/index.html

-X Proxy:port