Showing posts with label Linux. Show all posts
Showing posts with label Linux. Show all posts

Friday, April 5, 2013

Resuming a failed file copy

Let's say that you want to copy a very big file, in slow or faulty media like a cheap USB stick or disk, and the copy fails leaving you with an incomplete file:

olimpo:/Linux_minimal# ls -la
total 18739092
drwx------ 2 root root        4096 Apr  3 16:22 .
drwxr-xr-x 8 root root        4096 Apr  3 15:11 ..
-rw-r--r-- 1 root root 19170066432 Apr  3 16:46 Linux.dsk
-rw-r--r-- 1 root root         211 May  2  2010 create.txt
-rwxr-xr-x 1 root root         154 Apr  2 16:02 launch

olimpo:/Linux_minimal# ls -la ../Linux_test2
total 19417644
drwxr-xr-x 2 root root        4096 Mar 28 16:55 .
drwxr-xr-x 8 root root        4096 Apr  3 15:11 ..
-rw-r--r-- 1 root root 19864223744 Apr  2 15:47 Linux.dsk
-rw-r--r-- 1 root root         211 May  2  2010 create.txt
-rwxr-xr-x 1 root root         154 Apr  2 16:02 launch

What a frustration! No matter how many times you try to copy the file, the copy operation aborts before doing the task.

Fortunately you can resume the copy with rsync:

olimpo:/Linux_minimal# rsync --progress --partial --append ../Linux_test2/Linux.dsk Linux.dsk
Linux.dsk
 19864223744 100%   10.87MB/s    0:01:00 (xfer#1, to-check=0/1)

sent 694242119 bytes  received 31 bytes  11107874.40 bytes/sec
total size is 19864223744  speedup is 28.61

And just to be sure, you can check the files with diff (the slow way) or md5sum (the quick way):

olimpo:/Linux_minimal# md5sum Linux.dsk
eccc1436fe7a10acb44973449fd430bc  Linux.dsk

olimpo:/Linux_minimal# md5sum ../Linux_test2/Linux.dsk
eccc1436fe7a10acb44973449fd430bc  ../Linux_test2/Linux.dsk

More information:

How To Resume Failed copy ( cp command ) where it left off?

Saturday, May 12, 2012

Configuring a Tata Photon+ USB device in Linux

It has been a long time since my last post because I changed jobs three months ago, and in fact this post might give a hint about my new situation; anyway I hope to keep posting Linux and UNIX-related messages and this one is about dealing with a mobile broadband USB device. It is not so difficult to work with this kind of devices in Linux, and is very simple to configure it with NetworkManager, but first you need to have installed the usb-modeswitch software that changes the device's state from USB storage to USB modem in order to use the USB dongle as a modem. In Debian Squeeze you can do it installing the usb-modeswitch package. If you don't have the usb-modeswitch software you will see just an USB storage device, something like this:
May 12 15:23:38 olimpo kernel: [   64.172559] usb 2-1: new full speed USB device using uhci_hcd and address 2
May 12 15:23:38 olimpo kernel: [   64.339573] usb 2-1: New USB device found, idVendor=12d1, idProduct=1446
May 12 15:23:38 olimpo kernel: [   64.339584] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=4
May 12 15:23:38 olimpo kernel: [   64.339593] usb 2-1: Product: HUAWEI Mobile
May 12 15:23:38 olimpo kernel: [   64.339600] usb 2-1: Manufacturer: HUAWEI TECHNOLOGIES
May 12 15:23:38 olimpo kernel: [   64.339607] usb 2-1: SerialNumber: XXXXXXXXXXXXXXXXXX
May 12 15:23:38 olimpo kernel: [   64.339850] usb 2-1: configuration #1 chosen from 1 choice
May 12 15:23:38 olimpo kernel: [   64.350119] scsi4 : SCSI emulation for USB Mass Storage devices
May 12 15:23:38 olimpo kernel: [   64.360054] usb-storage: device found at 2
May 12 15:23:38 olimpo kernel: [   64.360061] usb-storage: waiting for device to settle before scanning
After installing the usb-modeswitch software you can plug your USB modem, and you will see at the end something like this:
May 12 15:23:40 olimpo usb_modeswitch: switching 12d1:1446 (HUAWEI TECHNOLOGIES: HUAWEI Mobile)
May 12 15:23:46 olimpo kernel: [   70.408061] usb 2-1: new full speed USB device using uhci_hcd and address 3
May 12 15:23:46 olimpo kernel: [   70.576318] usb 2-1: New USB device found, idVendor=12d1, idProduct=140b
May 12 15:23:46 olimpo kernel: [   70.576330] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=4
May 12 15:23:46 olimpo kernel: [   70.576339] usb 2-1: Product: HUAWEI Mobile
May 12 15:23:46 olimpo kernel: [   70.576345] usb 2-1: Manufacturer: HUAWEI TECHNOLOGIES
May 12 15:23:46 olimpo kernel: [   70.576352] usb 2-1: SerialNumber: XXXXXXXXXXXXXXXXXX
May 12 15:23:46 olimpo kernel: [   70.576600] usb 2-1: configuration #1 chosen from 1 choice
May 12 15:23:46 olimpo kernel: [   70.587360] scsi8 : SCSI emulation for USB Mass Storage devices
May 12 15:23:46 olimpo kernel: [   70.588245] usb-storage: device found at 3
May 12 15:23:46 olimpo kernel: [   70.588252] usb-storage: waiting for device to settle before scanning
May 12 15:23:46 olimpo kernel: [   70.785972] usbcore: registered new interface driver usbserial
May 12 15:23:46 olimpo usb_modeswitch: switched to 12d1:140b (HUAWEI TECHNOLOGIES: HUAWEI Mobile)
May 12 15:23:46 olimpo kernel: [   70.787067] USB Serial support registered for generic
May 12 15:23:46 olimpo kernel: [   70.788192] usbcore: registered new interface driver usbserial_generic
May 12 15:23:46 olimpo kernel: [   70.788201] usbserial: USB Serial Driver core
May 12 15:23:46 olimpo kernel: [   71.042606] USB Serial support registered for GSM modem (1-port)
May 12 15:23:46 olimpo kernel: [   71.048829] option 2-1:1.0: GSM modem (1-port) converter detected
May 12 15:23:46 olimpo kernel: [   71.050191] usb 2-1: GSM modem (1-port) converter now attached to ttyUSB0
May 12 15:23:46 olimpo kernel: [   71.050242] option 2-1:1.1: GSM modem (1-port) converter detected
May 12 15:23:46 olimpo kernel: [   71.050660] usb 2-1: GSM modem (1-port) converter now attached to ttyUSB1
May 12 15:23:46 olimpo kernel: [   71.050702] option 2-1:1.2: GSM modem (1-port) converter detected
May 12 15:23:46 olimpo kernel: [   71.056719] usb 2-1: GSM modem (1-port) converter now attached to ttyUSB2
May 12 15:23:46 olimpo kernel: [   71.056795] usbcore: registered new interface driver option
May 12 15:23:46 olimpo kernel: [   71.056803] option: v0.7.2:USB Driver for GSM modems
This is the most important part, to have detected your USB dongle as a GSM modem. If you like you can configure it easily with NetworkManager, but if you prefer something that you can do in a terminal then install the wvdial package and fill the /etc/wvdial.conf and /etc/ppp/chap-secrets files with this:
olimpo:/var/log# cat /etc/wvdial.conf
[Dialer Defaults]
Init1 = ATZ
Init2 = ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
Modem Type = USB Modem
ISDN = 0
Phone = #777
New PPPD = yes
Modem = /dev/ttyUSB0
Username = internet
Password = internet
Baud = 230400

olimpo:/var/log# cat /etc/ppp/chap-secrets
# Secrets for authentication using CHAP
# client server secret   IP addresses
internet * internet
And if you're using Debian you can also issue the dpkg-reconfigure wvdial command and give all the information required to the configuration program. Then you can launch wvdial as root from a terminal prompt and connect to the Internet, and you will see something like this in the syslog:
May 12 15:47:06 olimpo pppd[11490]: pppd 2.4.5 started by root, uid 0
May 12 15:47:06 olimpo kernel: [ 1770.688570] PPP generic driver version 2.4.2
May 12 15:47:06 olimpo pppd[11490]: Using interface ppp0
May 12 15:47:06 olimpo pppd[11490]: Connect: ppp0 <--> /dev/ttyUSB0
May 12 15:47:09 olimpo pppd[11490]: CHAP authentication succeeded
May 12 15:47:09 olimpo pppd[11490]: CHAP authentication succeeded
May 12 15:47:09 olimpo kernel: [ 1773.210976] PPP BSD Compression module registered
May 12 15:47:09 olimpo kernel: [ 1773.225216] PPP Deflate Compression module registered
May 12 15:47:09 olimpo pppd[11490]: local  IP address 59.161.17.54
May 12 15:47:09 olimpo pppd[11490]: remote IP address 172.29.145.65
May 12 15:47:09 olimpo pppd[11490]: primary   DNS address 4.2.2.3
May 12 15:47:09 olimpo pppd[11490]: secondary DNS address 121.242.190.181

...

May 12 15:54:11 olimpo pppd[11490]: Terminating on signal 15
May 12 15:54:11 olimpo pppd[11490]: Connect time 7.1 minutes.
May 12 15:54:11 olimpo pppd[11490]: Sent 460447 bytes, received 1883330 bytes.
Of course, this procedure might be helpful for configuring other USB modems as well.

Tuesday, February 7, 2012

Adding numbers in a file

If you have a small file filled with numbers and are tasked to add all of them, you can do it with bc and paste:

cronos@olimpo:~$ cat file.txt
1|11
2|22
3|33
4|44
5|55
|
6|66
7|77
8|88
9|99

cronos@olimpo:~$ cat file.txt | cut -d '|' -f 1 | grep -v '^$' | paste -sd '+' - | bc
45

With cut you select the field to sum, with grep you select out empty fields, with paste you add at the end of each line a + sign, and with bc you do the addition.

But if you have a very big file this won't work; you can instead use awk:

cronos@olimpo:~$ cat file.txt | awk 'BEGIN {FS="|";OFMT="%.2f"} {a+=$1;b+=$2} END {print "First field:",a,"Second field:",b}'

First field: 45 Second field: 495

With FS you set the field delimiter and with OFMT you set the number format, in this case two decimal places with no scientific notation. With awk a nice extra is to be able to add two or more columns at the same time.

More information:

Sum of numbers in file - UNIX alternatives

Monday, January 30, 2012

Oracle and OCFS2 not releasing space used

Sometimes you have to drop a tablespace to recover storage space and use it to create other tablespace, like when you have to resize an undo tablespace. And usually is not a big issue unless you have OCFS2 file systems:

oracle@myserver$ df -m|egrep '(Filesystem)|(/mydb/undo)'
Filesystem 1M-blocks Used Available Use% Mounted on
24576 24572 4 100% /mydb/undo

oracle@myserver$ sqlplus '/ as sysdba'

SQL*Plus: Release 10.2.0.4.0 - Production on Thu Jan 26 22:57:35 2012

Copyright (c) 1982, 2007, Oracle. All Rights Reserved.


Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Data Mining
and Real Application Testing options

SQL> drop tablespace myundo including contents and datafiles;

Tablespace dropped.

SQL> create undo tablespace mynewundo datafile '/mydb/undo/undo.dbf' size 10g;
create undo tablespace mynewundo datafile '/mydb/undo/undo.dbf' size 10g
*
ERROR at line 1:
ORA-01119: error in creating database file '/mydb/undo/undo.dbf'
ORA-27044: unable to write the header block of file
Linux-ia64 Error: 28: No space left on device
Additional information: 4

SQL> quit
Disconnected from Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Data Mining
and Real Application Testing options

oracle@myserver$ df -m|egrep '(Filesystem)|(/mydb/undo)'
Filesystem 1M-blocks Used Available Use% Mounted on
24576 24572 4 100% /mydb/undo

oracle@myserver$ ls -la /mydb/undo
total 16
drwxr-xr-x 4 root root 4096 Jan 20 2010 .
drwxr-xr-x 4 root root 4096 Dec 17 2009 ..
drwxr-xr-x 2 root root 4096 Dec 17 2009 lost+found

As you might notice there is no space recovered from dropping that undo tablespace, and since we need to create a new undo tablespace in the same file system we have to do something.

I don't have a lot of dabatases placed in OCFS2 file systems so I didn't care about finding the root cause of this problem, and since I had this database with no users connected I just created a small undo tablespace somewhere else, shut down the instance, and after doing it I got that space back:

oracle@myserver$ df -m|egrep '(Filesystem)|(/mydb/undo)'
Filesystem 1M-blocks Used Available Use% Mounted on
24576 359 24217 2% /mydb/undo

oracle@myserver$ ls -la /mydb/undo
total 16
drwxr-xr-x 4 root root 4096 Jan 20 2010 .
drwxr-xr-x 4 root root 4096 Dec 17 2009 ..
drwxr-xr-x 2 root root 4096 Dec 17 2009 lost+found

oracle@myserver$ mount|grep /mydb/undo
/dev/mapper/myvg-undo on /mydb/undo type ocfs2 (rw,_netdev,datavolume,nointr,heartbeat=local)

oracle@myserver$ uname -a
Linux myserver.localdomain 2.6.9-67.EL #1 SMP Wed Nov 7 13:43:35 EST 2007 ia64 ia64 ia64 GNU/Linux

oracle@myserver$ /sbin/lsmod|grep ocf
ocfs2 744096 26
debugfs 27340 2 ocfs2
ocfs2_dlmfs 53416 1
ocfs2_dlm 424192 2 ocfs2,ocfs2_dlmfs
ocfs2_nodemanager 316791 32 ocfs2,ocfs2_dlmfs,ocfs2_dlm
configfs 65468 2 ocfs2_nodemanager
jbd 149272 2 ocfs2,ext3

oracle@myserver$ /sbin/modinfo ocfs2
filename: /lib/modules/2.6.9-67.EL/kernel/fs/ocfs2/ocfs2.ko
license: GPL
author: Oracle
version: 1.2.9 1B43458FD47258934A48F1C
description: OCFS2 1.2.9 Mon May 19 14:04:34 PDT 2008 (build a693806cb619dd7f225004092b675ede)
depends: ocfs2_nodemanager,ocfs2_dlm,jbd,debugfs
vermagic: 2.6.9-67.EL SMP ia64gcc-3.4

I know this is just a workaround and not a good one, but it works fine and you can move on the next thing.

Wednesday, January 25, 2012

How to configure a VPN PPTP connection

If you need to open a VPN PPTP connection from a Linux computer, you can use the pptp client to configure a PPP connection and launch the pppd daemon to make the connection. You need to know at least the VPN server, an authorized user to make the connection and its password, and we will work with the root account for practical purposes but it's not mandatory.

First, if you don't have the pptp client you have to install it or configure the PPP connection by hand or by other means, in Debian Squeeze you could install the pptp-linux package.

Next, this is the easy way to configure your VPN PPTP connection:

pptpsetup --create MYVPN --server myvpnserver --username myvpnuser --password mypassword

You have to substitute myvpnserver, myvpnuser and mypassword with the VPN server, the user and password respectively; MYVPN is the name of the PPP configuration you will create and is chosen by you.

After this you will have a new configuration file in /etc/ppp/peers and also an extra line in /etc/ppp/chap-secrets:

olimpo:~# cat /etc/ppp/peers/MYVPN
# written by pptpsetup
pty "pptp myvpnserver --nolaunchpppd"
lock
noauth
nobsdcomp
nodeflate
name myvpnuser
remotename MYVPN
ipparam MYVPN

olimpo:~# cat /etc/ppp/chap-secrets
# Secrets for authentication using CHAP
# client server secret IP addresses

# added by pptpsetup for MYVPN
myvpnuser MYVPN "mypassword" *

If you want to configure the pppd daemon by hand, you just have to create a file in /etc/ppp/peers and add the authentication information in /etc/ppp/chap-secrets and it must be the same.

At this time you can open the PPP connection launching the pppd daemon:

olimpo:~# pppd call MYVPN updetach
Using interface ppp0
Connect: ppp0 <--> /dev/pts/3
CHAP authentication succeeded
local IP address 10.5.15.222
remote IP address 10.5.15.127

olimpo:~# tail --lines=16 /var/log/syslog
Jan 25 13:04:12 olimpo pppd[29036]: pppd 2.4.5 started by root, uid 0
Jan 25 13:04:12 olimpo pppd[29036]: Using interface ppp0
Jan 25 13:04:12 olimpo pppd[29036]: Connect: ppp0 <--> /dev/pts/3
Jan 25 13:04:13 olimpo pptp[29037]: anon log[main:pptp.c:314]: The synchronous pptp option is NOT activated
Jan 25 13:04:13 olimpo pptp[29041]: anon log[ctrlp_rep:pptp_ctrl.c:251]: Sent control packet type is 1 'Start-Control-Connection-Request'
Jan 25 13:04:13 olimpo pptp[29041]: anon log[ctrlp_disp:pptp_ctrl.c:739]: Received Start Control Connection Reply
Jan 25 13:04:13 olimpo pptp[29041]: anon log[ctrlp_disp:pptp_ctrl.c:773]: Client connection established.
Jan 25 13:04:14 olimpo NetworkManager[1919]: SCPlugin-Ifupdown: devices added (path: /sys/devices/virtual/net/ppp0, iface: ppp0)
Jan 25 13:04:14 olimpo NetworkManager[1919]: SCPlugin-Ifupdown: device added (path: /sys/devices/virtual/net/ppp0, iface: ppp0): no ifupdown configuration found.
Jan 25 13:04:14 olimpo pptp[29041]: anon log[ctrlp_rep:pptp_ctrl.c:251]: Sent control packet type is 7 'Outgoing-Call-Request'
Jan 25 13:04:14 olimpo pptp[29041]: anon log[ctrlp_disp:pptp_ctrl.c:858]: Received Outgoing Call Reply.
Jan 25 13:04:14 olimpo pptp[29041]: anon log[ctrlp_disp:pptp_ctrl.c:897]: Outgoing call established (call ID 0, peer's call ID 8918).
Jan 25 13:04:14 olimpo modem-manager: (net/ppp0): could not get port's parent device
Jan 25 13:04:17 olimpo pppd[29036]: CHAP authentication succeeded
Jan 25 13:04:17 olimpo pppd[29036]: local IP address 10.5.15.222
Jan 25 13:04:17 olimpo pppd[29036]: remote IP address 10.5.15.127

olimpo:~# ifconfig ppp0
ppp0 Link encap:Point-to-Point Protocol
inet addr:10.5.15.222 P-t-P:10.5.15.127 Mask:255.255.255.255
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1
RX packets:7 errors:0 dropped:0 overruns:0 frame:0
TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:3
RX bytes:99 (99.0 B) TX bytes:93 (93.0 B)

olimpo:~# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.5.15.127 * 255.255.255.255 UH 0 0 0 ppp0
static.customer 10.123.46.1 255.255.255.255 UGH 0 0 0 br0
10.123.46.0 * 255.255.255.0 U 0 0 0 br0
default 10.123.46.1 0.0.0.0 UG 0 0 0 br0

That's it, you have a VPN connection open to you VPN server! Or it's almost done, because we have a routing problem.

olimpo:~# ping 10.5.15.123
PING 10.5.15.123 (10.5.15.123) 56(84) bytes of data.
^C
--- 10.5.15.123 ping statistics ---
7 packets transmitted, 0 received, 100% packet loss, time 6047ms

olimpo:~# traceroute 10.5.15.123
traceroute to 10.5.15.123 (10.5.15.123), 30 hops max, 60 byte packets
1 10.123.81.4 (10.123.81.4) 0.407 ms 0.429 ms 0.492 ms
2 10.123.148.1 (10.123.148.1) 0.313 ms 0.352 ms 0.345 ms
3 * * *
4 * * *
5 * * *
6 *^C

As you might notice in the route information, there is no route to the VPN network from your Linux computer. You could append defaultroute to the pppd command, but this way you will have connectivity ONLY to the VPN network; to put it simple: is like if you were attached directly to the other network. If you had internet access and there is no way to access internet in the VPN network then you'll have no internet access, or if you had access to other networks after using pppd with defaultroute you will no longer have access to them.

And in fact that's the way a VPN is supposed to work. For example, if you got a trojan in your computer and that program is tailored specifically for your organization, and you connect your computer to the VPN network and still have access to the internet, then that trojan might steal information and send it to someone else easily. Or you might thing that leaving your VPN connection open and your computer accessible from the internet would be handy to share your VPN connection; this way you're creating a security hole in the VPN network so be careful and do it if you really need to.

Therefore, if you need to keep connectivity to other networks and don't want to append defaultroute to the pppd command, you have to create the route by hand. In this case, we want to access the VPN network 10.5.15.0/24, therefore we will issue this route command:

olimpo:~# route add -net 10.5.15.0 netmask 255.255.255.0 dev ppp0

As you can see in the route and traceroute information, this time you can reach the example server (10.5.15.123) through your PPP gateway (10.5.15.127):

olimpo:~# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.5.15.127 * 255.255.255.255 UH 0 0 0 ppp0
static.customer 10.123.46.1 255.255.255.255 UGH 0 0 0 br0
10.123.46.0 * 255.255.255.0 U 0 0 0 br0
10.5.15.0 * 255.255.255.0 U 0 0 0 ppp0
default 10.123.46.1 0.0.0.0 UG 0 0 0 br0

olimpo:~# traceroute 10.5.15.123
traceroute to 10.5.15.123 (10.5.15.123), 30 hops max, 60 byte packets
1 10.5.15.127 (10.5.15.127) 163.061 ms 163.027 ms 163.010 ms
2 10.5.15.123 (10.5.15.123) 174.981 ms 174.967 ms 174.954 ms

Finally, in order to end your PPP connection you have to end the pppd daemon (gracefully); you might do this getting the pppd daemon PID and issuing a kill command:

olimpo:~# ps ax|grep pppd
10094 pts/1 S+ 0:00 grep pppd
29037 pts/1 S 0:00 pptp myvpnserver --nolaunchpppd
29041 pts/1 S 0:00 pptp myvpnserver --nolaunchpppd
29044 pts/1 S 0:00 pppd call MYVPN updetach

olimpo:~# cat /var/run/ppp0.pid
29044

olimpo:~/bin# kill -TERM 29044

olimpo:~/bin# tail --lines=11 /var/log/syslog
Jan 25 15:06:43 olimpo pppd[29044]: Terminating on signal 15
Jan 25 15:06:43 olimpo pppd[29044]: Modem hangup
Jan 25 15:06:43 olimpo pppd[29044]: Connect time 122.5 minutes.
Jan 25 15:06:43 olimpo pppd[29044]: Sent 1140 bytes, received 1056 bytes.
Jan 25 15:06:43 olimpo pptp[29041]: anon log[callmgr_main:pptp_callmgr.c:258]: Closing connection (shutdown)
Jan 25 15:06:43 olimpo pptp[29041]: anon log[ctrlp_rep:pptp_ctrl.c:251]: Sent control packet type is 12 'Call-Clear-Request'
Jan 25 15:06:43 olimpo pptp[29041]: anon log[call_callback:pptp_callmgr.c:79]: Closing connection (call state)
Jan 25 15:06:44 olimpo pppd[29044]: Connection terminated.
Jan 25 15:06:44 olimpo avahi-daemon[1488]: Withdrawing workstation service for ppp0.
Jan 25 15:06:44 olimpo NetworkManager[1919]: SCPlugin-Ifupdown: devices removed (path: /sys/devices/virtual/net/ppp0, iface: ppp0)
Jan 25 15:06:49 olimpo pppd[29044]: Exit.

olimpo:~# ps ax|grep pppd
11587 pts/1 S+ 0:00 grep pppd

More information:

The Point-to-Point Protocol

Monday, November 28, 2011

About the RHCSA certification

From time to time there are people that ask me how I got the OCP certification and I regret not having posted about it, therefore now that I'm a shiny, brand-new Red Hat Certified System Administrator I would like to write about this.

First of all, this is a practical exam so if you enjoy taking multiple choice quizzes this is not the case. You can check all the information about this test in the Red Hat Certified System Administrator page. Something worth of reading it is the RHCSA Exam Objectives page, mandatory if you choose to take the test without enrolling into Red Hat recommended courses. By the way, if you are unsure if you have the level required for passing the test without enrolling into a course, it would be a good idea to take the pre-assessment quiz; you have to fill in a form with personal information in order to take the quiz, but if you take the certification test you'll have to give it anyway.

I prefer a lot to study a book and practice with my home computers, and for this test I studied the RHCE Red Hat Certified Engineer Linux Study Guide, by Michael Jang. Talking about other certification book, I read some time ago a review that blasted that book because there were some things that weren't covered in the book that were in the test, and unfortunately this is the case for this book too. But taking into account that no book can just put the exact test with its answers and that at this time I don't know a guide for studying for the new Red Hat Enterprise Linux 6 exams, I think that Jang's book is a good one for preparing for the RHCSA exam. I recommend reading the whole book and PRACTICING its exercises, even if this book is intended to be a guide for the (now obsolete) RHCT and RHCE exams; if you study the RHCE part you'll be a lot more confident while taking the test than if you just study the RHCT part.

Finally, as any good certification test, it's aimed to check your skills and knowledge about a concrete field, therefore having experience in this topics will help a lot and would be decisive in passing the test; if you don't have experience in some topics try at lest to practice it a lot before taking the test.

I think that holding a certification is a good way to back your skills and studying for the test is a good opportunity to keep up to date your knowledge, therefore I wish good luck for everybody that decide to take the RHCSA test.

Tuesday, October 25, 2011

Getting partition's size

Getting the size of a partition in Linux is very simple if you're logged in as root:

olimpo:~# fdisk -l /dev/sda

Disk /dev/sda: 160.0 GB, 160000000000 bytes
255 heads, 63 sectors/track, 19452 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xabd592d5

Device Boot Start End Blocks Id System
/dev/sda1 * 1 1567 12586896 7 HPFS/NTFS
/dev/sda2 1568 2546 7863817+ 17 Hidden HPFS/NTFS
/dev/sda3 2547 2796 2008125 82 Linux swap / Solaris
/dev/sda4 2797 19452 133789320 83 Linux

But it might be that you're just a DBA with access only to some partitions instead of the whole disk, and you got this server with no idea about partition sizes. If this is the case, then you can still know the size of your partitions with fdisk:

oracle@olimpo:/myASM$ /sbin/fdisk -l /dev/sda
Cannot open /dev/sda
oracle@olimpo:/myASM$ ls -la
total 8
drwxr-xr-x 2 oracle dba 4096 Oct 25 12:15 .
drwxr-xr-x 113 root root 4096 Oct 25 12:15 ..
brw-rw---- 1 oracle dba 8, 4 Oct 3 08:44 sda4
oracle@olimpo:/myASM$ /sbin/fdisk -s sda4
133789320

In Solaris is a bit different as the -s flag is not the same as in Linux:

oracle@myserver:/myASM$ ls -la
total 4
drwxr-xr-x 2 oracle dba 512 Oct 22 2008 .
drwxr-xr-x 38 root root 1024 Oct 18 16:57 ..
crw-r--r-- 1 oracle dba 32, 24 Oct 22 2008 disk1
crw-r--r-- 1 oracle dba 32, 25 Oct 25 09:31 disk2
crw-r--r-- 1 oracle dba 32, 26 Oct 22 2008 disk3

oracle@myserver:/myASM$ /sbin/fdisk -Rg disk2
* Label geometry for device disk2
* PCYL NCYL ACYL BCYL NHEAD NSECT SECSIZ
35135 35135 2 0 64 255 512

oracle@myserver:/myASM$ echo "(35135*64*255*512)/1024/1024"|bc
279982

With the last command you calculate the partition size in megabytes.

Tuesday, October 18, 2011

Playing with ssh, xauth and DISPLAY

If you need to connect by ssh to an UNIX server and later change user (by means of su or otherwise), but you want to keep X11 forwarding to your screen, then look at this example using xauth and DISPLAY. The key is getting cookie information with xauth list and identifying the screen opened by ssh (usually screen 10 of the server), and after changing user pasting the whole line after the xauth add command:

myuser@olimpo:~$ ssh -X -l myuser myserver1
Password:
Last login: Mon Oct 17 18:00:46 2011 from olimpo
$ xclock
$ xauth list
myserver1.mynet.net:10 MIT-MAGIC-COOKIE-1 eb3713bfc73799efede775bf1a04cb19
$ su - oracle
Password:
myserver1:/home/oracle> xclock
Error: Can't open display:
myserver1:/home/oracle> xauth add myserver1.mynet.net:10 MIT-MAGIC-COOKIE-1 eb3713bfc73799efede775bf1a04cb19
myserver1:/home/oracle> export DISPLAY=myserver1.mynet.net:10
myserver1:/home/oracle> xclock
myserver1:/home/oracle> uname -n
myserver1
myserver1:/home/oracle>

The uname -n command from above example was an extra for the following case: this time you try to login to other server but you get a xauth error message and are unable to forward any X11 window:

myuser@olimpo:~$ ssh -X -l oracle myserver2
Password:
/usr/openwin/bin/xauth: (stdin):1: bad display name "unix:10.0" in "add" command

:/home/oracle> xclock
X11 connection rejected because of wrong authentication.
X connection to localhost:10.0 broken (explicit kill or server shutdown).
:/home/oracle> echo $DISPLAY
localhost:10.0
:/home/oracle> uname -n

:/home/oracle>

As you might have noticed the DISPLAY variable seems correct, but there is a problem with the host name: there is no host name set; that's why xauth fails at login time. You have to set properly the host name according to your UNIX version in order to correct this problem.

More information:

Getting X11 forwarding through ssh working after running su

Thursday, October 6, 2011

Connect from Linux to an X server

This trick is a very simple but useful one: if you want to open an X session in your Linux computer from other UNIX computer, like if you were in front of its console, then you can run as root a command like this:

X -query unixserver :1

This way you're telling X to connect to the greeter running in unixserver and display it in graphic console number one (you might be running an X session in :0 already). In order to work this the greeter in the UNIX computer must be running and listening for connections; for example, it would be xdm, kdm or gdm in Linux, dtlogin in HP-UX and so on.

Thursday, August 18, 2011

I'm a hacker! And Hyk-proxy


Ah, I'm a real hacker at last! That has been my dream since I started learning UNIX 20 years ago; being like Richard Stallman, Linus Torvalds or Alan Cox, and master any UNIX system.

Or at least so says Websense when I try to see this blog; I'm afraid they mean this new popular meaning of hacker, something like "people that enjoys stealing money with computers as a way of life". But I can't remember posting something like that in this blog; is it possible that knowledge of the ps command or the SQL select statement qualifies someone as a hacker? If so, then I can understand why there is a shortage of IT skilled people ...

Anyway, I started this blog as a mean to have access anywhere to my technical memories but Websense makes this impossible at my new assignment, unless I use something called a web proxy. I found Hyk-proxy, that is a nice proxy server that runs on Google Apps Engine service, and a client-side agent that acts as a proxy for browsers and sends requests to the GAE proxy server.

I have been using this software just one day but I think it's pretty good; I can access restricted sites like Blogger and everything seems fine and smooth even in my work's computer (just 512 Mb. of RAM), and when you complete the deployment of the server in GAE you can use the same software client in Linux or Windows or even carry it in an USB stick, because everything is written in Java.

I'm not sure if it's my fault or Hyk-proxy cannot handle HTTPS sites well, but I had some issues logging in Google and with Gmail (certificates); as long as Websense is not so strict at my workplace I decided to use Hyk-proxy for HTTP sites and a direct connect (or squid) for HTTPS sites. But there is a catch: you can send just one gigabyte and receive another gigabyte of data per day as a free service in GAE, therefore if you enjoy watching movies at YouTube or things like that then you might run out of quota very quickly.

I'm not going to explain this "hack" here because there is a well-written howto here, but in order to be able to deploy and run Hyk-proxy you will need to have a Google account, access to Google App Engine, Java, Eclipse and Google App Engine SDK installed in your computer, and download the Hyk-proxy software. As you can see it's not so easy to deploy it, but the howto and documentation are well-written and the setup scripts are very simple, and this software works like a charm.

Then, if you want to access some sites to work better or you just want to waste your time at the office, become a hacker for a few minutes and try Hyk-proxy.

Monday, May 9, 2011

Oracle instances dying and read-only filesystems

You have a Linux server running one or more Oracle instances, and some day all the instances are down except their listeners. You check the alert log and found absolutely nothing, and if you're lucky enough you try to start up instances and get an error message about a read-only filesystem.

myserver> dbstart
ORACLE_HOME_LISTNER is not SET, unable to auto-start Oracle Net Listener
Usage: /oracle/product/10.2.0/bin/dbstart ORACLE_HOME
touch: cannot touch `/oracle/product/10.2.0/startup.log': Read-only file system
chmod: changing permissions of `/oracle/product/10.2.0/startup.log': Read-only file system
Processing Database instance "mydb01": log file /oracle/product/10.2.0/startup.log
/oracle/product/10.2.0/bin/dbstart: line 361: /oracle/product/10.2.0/startup.log: Read-only file system
touch: cannot touch `/oracle/product/10.2.0/startup.log': Read-only file system
chmod: changing permissions of `/oracle/product/10.2.0/startup.log': Read-only file system
Processing Database instance "mydb02": log file /oracle/product/10.2.0/startup.log
/oracle/product/10.2.0/bin/dbstart: line 361: /oracle/product/10.2.0/startup.log: Read-only file system

myserver> touch /oracle/product/10.2.0/test.txt
touch: cannot touch `/oracle/product/10.2.0/test.txt': Read-only file system

You try to create an archive with touch in the Oracle's filesystem and you can't, so you check server up time, load and mounted filesystems:

myserver> uptime
11:34:30 up 138 days, 18:02, 1 user, load average: 0.00, 0.00, 0.00

myserver> mount
/dev/mapper/VolGroup01-LogVol00 on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
/dev/cciss/c0d0p1 on /boot type ext3 (rw)
tmpfs on /dev/shm type tmpfs (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
/dev/mapper/appsvg-lv001 on /oracle type ext3 (rw,_netdev)

Everything seems fine but you notice the option _netdev in the Oracle filesystem. When you're configuring filesystems in /etc/fstab and some of them are network-dependent you put this option so the operating system won't try to mount them before having network connectivity.

Therefore, you have a filesystem that is accessed by network so it might be with iSCSI:

myserver> /sbin/lsmod|grep iscsi
iscsi_tcp 19785 3
libiscsi_tcp 21957 2 iscsi_tcp,cxgb3i
libiscsi2 42181 5 ib_iser,iscsi_tcp,bnx2i,cxgb3i,libiscsi_tcp
scsi_transport_iscsi2 37709 7 ib_iser,iscsi_tcp,bnx2i,cxgb3i,libiscsi2
scsi_transport_iscsi 6085 1 scsi_transport_iscsi2
scsi_mod 141717 23 mptctl,ib_iser,iscsi_tcp,bnx2i,cxgb3i,libiscsi2,scsi_transport_iscsi2,scsi_dh,sg,qla2xxx,scsi_transport_fc,mptspi,scsi_transport_spi,mptsas,mptscsih,scsi_transport_sas,usb_storage,cciss,hpahcisr,sd_mod

myserver> /sbin/iscsiadm -m session -P 2
iscsiadm: Maybe you are not root?
iscsiadm: Could not lock discovery DB: /var/lock/iscsi/lock.write: Permission denied
Target: iqn.2000-03.com.someprovider:mycompany:87:mycmp1
Current Portal: 10.0.57.23:3260,1
Persistent Portal: 10.0.57.34:3260,1
**********
Interface:
**********
Iface Name: default
Iface Transport: tcp
Iface Initiatorname: iqn.1994-05.com.redhat:1234abc56d78
Iface IPaddress: 10.0.57.85
Iface HWaddress:
Iface Netdev:
SID: 1
iSCSI Connection State: Unknown
iSCSI Session State: LOGGED_IN
Internal iscsid Session State: Unknown
************************
Negotiated iSCSI params:
************************
HeaderDigest: None
DataDigest: None
MaxRecvDataSegmentLength: 262144
MaxXmitDataSegmentLength: 262144
FirstBurstLength: 262144
MaxBurstLength: 1048576
ImmediateData: Yes
InitialR2T: No
MaxOutstandingR2T: 1

That's the problem! Oracle instances die because cannot write database files, and if the logs are placed in the same filesystems Oracle cannot write error messages either. This connectivity problem would be by heavy load, network glitches, problems with the disk appliance or something else, but if the problem is not so bad your sysadmin might change some iSCSI parameters to help a bit:

node.conn[0].timeo.noop_out_interval = 0
node.conn[0].timeo.noop_out_timeout = 0
node.session.timeo.replacement_timeout = 86400

More information:

Filesystems becoming read-only on iSCSI
Linux* Open-iSCSI

Wednesday, April 13, 2011

Memory overhead

Talking about computers the more the better is usual, but this is not true all the time. By example, look at the amount of memory and load in bigservr; this server has 64 gigabytes of memory but is used just 3 gigabytes, and even if there are no running processes and the CPU usage is almost zero the load average is somewhat high. In comparison, smallsrv is busier and even having used swap memory the load average is lower than in bigservr.

The management of hardware by the Linux kernel requires resources, and in order to manage a lot of memory the kernel requires ... a lot of memory structures. Therefore, if you aren't going to use a lot of memory it would be better (and cheaper) to install just the memory you need.

myuser@bigservr> top

top - 08:50:32 up 6 days, 13:00, 4 users, load average: 0.71, 0.74, 0.66
Tasks: 272 total, 1 running, 271 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.8%us, 0.1%sy, 0.0%ni, 99.1%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 65791000k total, 3227812k used, 62563188k free, 309716k buffers
Swap: 5406712k total, 0k used, 5406712k free, 2649252k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
10768 root 15 0 16420 2456 2124 S 0.3 0.0 0:39.00 cmaeventd
22241 myuser 15 0 2336 1136 800 R 0.3 0.0 0:01.14 top
1 root 15 0 2072 616 532 S 0.0 0.0 0:08.29 init
2 root RT -5 0 0 0 S 0.0 0.0 0:01.17 migration/0
3 root 39 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/0
4 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/0
5 root RT -5 0 0 0 S 0.0 0.0 0:01.44 migration/1
6 root 34 19 0 0 0 S 0.0 0.0 0:00.07 ksoftirqd/1
7 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/1
8 root RT -5 0 0 0 S 0.0 0.0 0:01.01 migration/2
9 root 34 19 0 0 0 S 0.0 0.0 0:00.02 ksoftirqd/2
10 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/2
11 root RT -5 0 0 0 S 0.0 0.0 0:01.72 migration/3

root@smallsrv:~# top

top - 08:51:20 up 29 min, 6 users, load average: 0.16, 0.21, 0.46
Tasks: 184 total, 2 running, 182 sleeping, 0 stopped, 0 zombie
Cpu(s): 7.0%us, 12.3%sy, 0.0%ni, 79.3%id, 1.3%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 1016864k total, 987644k used, 29220k free, 44384k buffers
Swap: 2980016k total, 204780k used, 2775236k free, 604592k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2599 myuser 10 -10 271m 186m 177m S 9.6 18.8 2:55.94 vmware-vmx
2377 myuser 20 0 388m 103m 19m S 4.3 10.4 0:50.69 firefox-bin
1030 root 20 0 105m 20m 9428 S 3.0 2.1 1:00.57 Xorg
2435 myuser 20 0 73624 13m 9.9m S 1.3 1.4 0:15.40 plugin-containe
3479 root 20 0 2472 1196 880 R 0.7 0.1 0:00.06 top
16 root 15 -5 0 0 0 S 0.3 0.0 0:00.39 ata/0
2028 myuser 20 0 110m 10m 5832 S 0.3 1.1 0:12.17 compiz.real
2206 myuser 20 0 113m 11m 8324 S 0.3 1.2 0:04.71 gnome-terminal
2938 root 0 -20 0 0 0 R 0.3 0.0 0:08.41 vmware-rtc
1 root 20 0 2532 1420 1096 S 0.0 0.1 0:00.98 init
2 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 kthreadd
3 root RT -5 0 0 0 S 0.0 0.0 0:00.00 migration/0
4 root 15 -5 0 0 0 S 0.0 0.0 0:00.09 ksoftirqd/0
5 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/0
6 root 15 -5 0 0 0 S 0.0 0.0 0:00.02 events/0
7 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 cpuset
8 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 khelper