Occasionally for research or classes you need a different version of python and pip than what is installed on the COE machines. Sometimes you can get away with just using a virtual environment that allows you to install custom python packages ( see /eci/kb/index.php?action=artikel&cat=15&id=174&artlang=en for instructions on how to set it up) and sometimes you need something more due to your software needing to reach outside the python environment. Due to security and privacy concerns we do not grant root, dnf, yum, and/or sudo access to our lab machines. However, that does not mean you cannot get your desired python package another way.
Open up a terminal (all terminal commands will either be offset or in quotes for clarity)
type: “quota -v”
If you do not have at least 4 GB free stop and clean up your account before proceeding.
Change directory to your desktop using “cd Desktop” If you do not change directory you will get permission denied errors. By installing on the desktop, if you mess up, you only mess yourself up.
*Disclaimer: all commands are assuming centos operating system (linux01.engr.ucsb.edu to linux28.engr.ucsb.edu in the Honea Lab a.k.a back room of CSIL)
tar zxfv Python-2.7.11.tgz
find ~/python -type d | xargs chmod 0755
compiled the source following its guideline
make && make install
prefix option, it is mandatory for this to work. The value of prefix option is to specify where to put the related output of
make command, by default it is in the
/usr/local/ and we don't want that so we use our own customized directory. For most people that will be /fs/student/yourCOEaccount
Here comes another important step. By the default, if we type
python command, it will use the default python of the system. We are going to update the environment variables to force the shell to use our new python. Edit
~/.bashrc_profile and add the following lines to the end:
Finally, refresh the current session by running the command:
You might need to logout and login again for the environment to update properly. At this point, you should be able to see a new python. To check, run this command:
it should show you the path to the
python binary file, which is located in your home directory:
Pip is a program used to help us easily install python packages, it is similar to
rubygems in Ruby world. After installing python locally as described in the first step, it is very easy to install
Run the following command to install pip as a local user
wget --no-check-certificate https://bootstrap.pypa.io/get-pip.py -O - | python - --user
After finishing the installation, we need to update our PATH variable. Open ~/.bashrc_profile and add the following line:
Again, reload the session by the command
source ~/.bashrc_profile or logout and login. Then, check if
pip command is available:
It should show a path pointing to your local directory:
pip installed as a local user, you can install any other packages you want without worrying about other parts of the whole system. This is extremely useful in case you want to experiment with new things.
Special thanks to bluehost and Lazy Dog whose tutorial I modified for this article as well as ulutan
who worked out with me all the modifications needed to run it on UCSB COE centos boxes.
3. Install Cuda (optional for most users)
Cuda is the Nvidia API. There are several programs that use it. Tensorflow, Caffe, Pytorch all use Cuda in various capacities. To get cuda go to https://developer.nvidia.com/cuda-downloads
click on Linux >> x86_64 >> Centos >> 6 >> runfile(local) >> download
navigate to your desktop again like in part 1 and create a cuda folder: “mkdir cuda”
Drop the tar ball file into said folder and unball the file. type “sh cuda_8.0.61_375.26_linux.run”
Follow the instructions on the screen. They should be fairly straightforward if you made it this far.
1) Accept the EULA agreement
2) Do not install the Nvidia driver, there is one already installed on the computers. Installing a second will give you the black screen of death on your account and make you very unhappy.
3) Install the CUDA toolkit
4) Enter the toolkit location since the default install location will give you errors: ~/Desktop/cuda
5) Do not install symbolic link
6) Do not install the cuda samples unless you want to go over quota and have issues
You will then have to modify your bashrc file again so that the PATH includes ~/Desktop/cuda/bin
If you did everything right, typing “./nvcc -V” should now give you the cuda version number.
Launch your virtual environment and have fun!
To read more about launching virtual environments, or if you read this massive chunk of instructions and decided you only really need to install one python package, see:
Tags: permission, python, toolbox