Progress instances dying and read-only filesystems

Some day a user tells you one (or a lot) of your Progress database instances is down so you check first if it is running with ps and proutil commands. The instance is in fact down, therefore you check the database log just to find nothing. You try to start up the instance with proserve, and you finally realize why the instance died: the database's filesystem changed to read-only mode.

This might be usual if you have a misconfigured remote filesystem or if you use it heavily, for example iSCSI, and it has to be solved by the sysadmin in charge of that server.

[root@myserver]# ps ax|grep mydb
18187 pts/2    S+     0:00 grep mydb
[root@myserver]# proutil /progdbs/mydb.db -C holder
PROGRESS Version 9.1E as of Tue Oct 12 14:40:19 EDT 2004

** Cannot find or open file /progdbs/mydb.lk, errno = 17. (43)
[root@myserver]# cd /progdbs
[root@myserver]# tail --lines=10 mydb.lg
14:59:09 SRV     1: Previous message sent on behalf of user 183. (5512)
14:59:10 SRV     1: User is SOMEUSER. (708)
14:59:10 SRV     1: Previous message sent on behalf of user 183. (5512)
14:59:16 SRV     1: Logout usernum 183, userid SOMEUSER, on W1C3XDV1. (739)
14:59:44 SRV     1: Login usernum 183, userid ANUSER, on W1C3XDV1. (742)
14:59:44 SRV     1: User is OTHERUSER. (708)
14:59:44 SRV     1: Previous message sent on behalf of user 183. (5512)
14:59:44 SRV     1: User is OTHERUSER. (708)
14:59:44 SRV     1: Previous message sent on behalf of user 183. (5512)
14:59:44 SRV     1: User is OTHERUSER. (708)
14:59:44 SR
[root@myserver]# proserve /progdbs/mydb -pf /progdbs/mydb.pf
PROGRESS Version 9.1E as of Tue Oct 12 14:40:19 EDT 2004
10:23:38 BROKER   : ** Cannot find or open file /progdbs/mydb.lk, errno = 17. (43)
10:23:38 BROKER   : ** This process terminated with exit code 2. (8619)
[root@myserver]# ls -la /progdbs/mydb.lk
-r--r--r--  1 root root 38 Dec 21 11:44 /progdbs/mydb.lk
[root@myserver]# touch /progdbs/hello.txt
touch: cannot touch `/progdbs/hello.txt': Read-only file system