Modify a python package installed as a single EGG file

Sometimes, you want to modify a python package installed in your python library. How about if you see the package youjust installed is a single .egg file? You could still open it with your favorite decompress program, such as 7zip, since we know .egg file is just a ZIP file. After decompress you get folders which contain regular.py and .pyc file for you to work on. Then you can put the decompressed folder into your site-package directory keeping the same name (use original .egg name as directory name).

Make post request using m2urllib2 from M2Crypto

I was debugging a POST request initiated by m2urllib2.urlopen(req) without success. All I got are GET request. Till this morning I realized one parameter has been causing problems.

Old request (GET request)

req = m2urllib2.Request('https://openam.example.com:8445/openam/json/piv/authenticate','None', headers)
res = m2urllib2.urlopen(req)

Revised request (POST request)

req = m2urllib2.Request('https://openam.example.com:8445/openam/json/piv/authenticate','', headers)
res = m2urllib2.urlopen(req)

Pay attention to the second parameter in m2urllib2.Request, if I use 'None', it would be GET request; if I use '', it would be POST request. I could not find out why.

Configure Firefox with Smart Card Reader in Ubuntu

Firefox in Ubuntu does not support Smart Card reader out box. In this tutorial, I used SCR3310 V2.0 as example. Reader my other post regarding how to set up this reader in Ubuntu and enable pcscd service.

1. We need libopensc.so.3 file from OpenSC package

#Install opensc
sudo apt-get install opensc

2. Open Firefox, click "Open Menu" >>"Preferences">>"Advanced">>"Certificates" >>"Security Devices">>"Load"

in the "Load PKCS#11 Device" window, put

Module Name: PIV Card Reader (or any name as you wish)
Module filename: "/usr/lib/opensc-pkcs11.so"

3. Restart Firefox you should be able to see all your certificates on PIV card under "Certificates">>"Your Certificate" tab.

Understand Bash History and how to Clear

A good article about bash history and how to clear bash history.

In a pinch, you can clear all your bash shell’s command history by typing this command:

history -c
However, there are a few things about how bash shell’s history that worth a closer look.

The bash shell stores the command history in a file, this is typically ~/.bash_history. It is a simple text file with one command per line. Whenever an interactive instance of the bash shell is created, the bash shell loads the history into the memory. As long as the shell is running all new typed commands are appended in the history, but they only exist in memory. The history will be written to~/.bash_history when the shell is terminated (or if you issue the history -w command, whichever comes first).

There is nothing wrong with that, as long as you use only one instance of the bash shell at a time. Consider this: you log into your computer and open two terminal windows. You do some work on the first shell, then you do some other work on the second shell. When done, you want to clear all recorded history and exit. You go to the first shell and type history -c to clear the history, and then exit. Then you go to the second shell and simply type exit, thinking you already have cleared the history. You think you have cleaned up everything, but no, if you check the contents of your ~/.bash_history all the history from the second bash instance will be there, looking at you right in the eyes.

One thing you can do is to scrupulously make sure that you execute history -c before exiting every bash instance you are running. But this is kind of stressful, plus you may always run into the case of forgetting to clear the history, because people simply forget.

Another thing you may try is to add the history -c command in your ./bash_logout script, but you have to keep in mind that it works only with login shells, like the shell you are given from the console, or when running bash with the –login parameter. This will NOT work with the shells opened by terminal emulators (like the GNOME Terminal, for example).

A more effective approach would be to simply add a crontab entry that will empty the ~/.bash_history file in a reasonable frequency. This will do:

*/1 * * * * cat /dev/null > ~/.bash_history
This command in your crontab will cause the bash shell’s history file to be emptied every one minute. Please not that you cannot runhistory -c from crontab, because history is in fact an internal bash shell’s command, not a real program that can be executed from crontab.

Add New Root User in OpenDJ

Ever wondered how to add another directory administrator to OPenDJ? Here is how.
1. Create a file admin2.ldif

dn: cn=EMEN2 Admin,cn=Root DNs,cn=config
cn: EMEN2 Admin
sn: Admin
givenName: EMEN2
objectClass: inetOrgPerson
objectClass: person
objectClass: top
objectClass: ds-cfg-root-dn-user
objectClass: organizationalPerson
userPassword: juSugU4w
ds-cfg-alternate-bind-dn: cn=EMEN2 Admin

2. run the following command:

/path to opendj/bin/ldapmodify -h localhost -p 1389 -D "cn=Directory Manager" -w password_for_Directory Manager -a -f "admin2.ldif"

Setup SCR3310 V2.0 Smart Card Reader in Ubuntu 14.04 64Bit

Here I am going to demonstrate how to enable SCR3310 V2.0 smart card reader in Ubuntu 14.04 box.

Requirements

libusb and libusb-dev
pcsc-lite-1.8.11
Smart Card reader driver 64Bit
pcsc-tools (Optional)

Install

1.Install libusb and libusb-dev

sudo apt-get install libusb-1.0-0 libusb-1.0-0-dev

2.Download and install pcsclite

#download
wget https://alioth.debian.org/frs/download.php/file/3991/pcsc-lite-1.8.11.tar.bz2
tar jvxf pcsc-lite-1.8.11.tar.bz2
cd pcsc-lite-1.8.11/
#make sure that you have enable libusab and disable libudev
./configure --enable-libusb --disable-libudev
make
sudo make install
sudo make fix-rights

3. Download and install 64Bit card reader driver

wget http://support.identive-group.com/download/driver/scmccid_5.0.29_linux_64bit.tar.gz
tar zvxf scmccid_5.0.29_linux_64bit.tar.gz
cd scmccid_5.0.29_linux_64bit/
sudo ./install.sh

4.Start pcscd service

sudo /usr/local/sbin/pcscd

Test Card Reader

1. install pcsc_tools

#install
sudo apt-get install pcsc-tools
#scan card
pcsc_scan

"am_agent_init failed" during OpenAM Web Policy Agent starts up

Background

OpenAM was deployed on Apache Tomcat and Web Agent was deployed in httpd server. Both of them were running in the SAME CentOS server. Yes, I know you might laugh at me: SAME SERVER?
If start service one by by one from command line, there was no error in either the log file of web agent AGENT_DIR/apache22_agent/Agent_##/logs/debug/amAgent or on web pages which uses web agent service.

#Start service one by one
/OPENDJ_DIR/bin/start-ds
/TOMCAT_DIR/bin/startup.sh
/etc/init.d/httpd start 

However, there were errors when you put them either in one run file rc.local with the same run order or in the same order but separated using chkconfig.

#Same order in rc.local file:
OPENDJ_ROOT="/opt/sso/opendj"
TOMCAT_ROOT="/opt/sso/apache-tomcat-7.0.52"
export OPENDJ_ROOT
export TOMCAT_ROOT
cd ${OPENDJ_ROOT}
/bin/su - nctrsso -c "${OPENDJ_ROOT}/bin/start-ds --quiet"
cd ${TOMCAT_ROOT}
/bin/su - nctrsso -c "${TOMCAT_ROOT}/bin/startup.sh"
/etc/init.d/httpd start

Using chkconfig tool to set run order.

#same order but seperate using chkconfig
S11ssoopendj -> ../init.d/ssoopendj
S18ssotomcat -> ../init.d/ssotomcat
S99httpd -> ../init.d/httpd

Symptom

The error were, "403 forbidden" when you visited web page hosted on httpd which use web agent service and "am_agent_init failed" in amAgent log file.

#detailed amAgent error
2014-08-14 09:23:20.351    Error 4841:47170489365360 all: Connection::request() socket not available, error 20
2014-08-14 09:23:20.351    Error 4841:47170489365360 Agent Profile Service: isRESTServiceAvailable(): An error occured while doing naming request. HTTP error
2014-08-14 09:23:20.352    Error 4841:47170489365360 all: agent_worker_init() am_agent_init failed

If I restart httpd server now, both error will go away.

This makes me think that maybe it because when httpd starts and initiates web agent, Tomcat has not fully started yet. 

Checking time stamps of apache log file and amAgent log file, I knew Tomcat started to init on 9:23:22 till 9:24:20; and httpd started to init web agent on 9:23:20. These data prove I am right.

# key timing in tomcat log file
Aug 14, 2014 9:23:22 AM org.apache.catalina.core.AprLifecycleListener init
Aug 14, 2014 9:24:20 AM org.apache.catalina.startup.Catalina start
INFO: Server startup in 57158 ms
# key timing in httpd log file
2014-08-14 09:23:20.292       -1 4840:47170489365360 all: Version: 4.0.0-SNAPSHOT

Using ps auxcheck the start time also proves that httpd initiates before tomcat fully started.

Solution

Run these init processes in the same script, rc.local, and let the calling thread sleep 120 seconds before initiating httpd for now, until a better solution is found.

OPENDJ_ROOT="/opt/sso/opendj"
TOMCAT_ROOT="/opt/sso/apache-tomcat-7.0.52"
export OPENDJ_ROOT
export TOMCAT_ROOT
cd ${OPENDJ_ROOT}
/bin/su - nctrsso -c "${OPENDJ_ROOT}/bin/start-ds --quiet"
cd ${TOMCAT_ROOT}
/bin/su - nctrsso -c "${TOMCAT_ROOT}/bin/startup.sh"
sleep 120
/etc/init.d/httpd start
exit 0