How to Setup Jupyter Notebook Server as a daemon service
Create a new user
Create a new user on your system with a home directory. Refer to documentation of the linux version you are using to set this up. This is a recommended method since it will allow smooth and dedicated user setup for this Jupyter Server.
Check packages required
You should have these packages installed
python3 --version pip3 --version virtualenv --version
if not installed login as
sudo user and install
sudo apt update sudo apt install python3-dev python3-pip sudo pip3 install -U virtualenv # system-wide install
Setup Environment for development as jupyter notebook
user and create a new virtual environment by choosing a Python interpreter and making a ./venv directory to hold it:
virtualenv --system-site-packages -p python3 ~/venv
Activate the virtual environment using a shell-specific command:
source ~/venv/bin/activate # sh, bash, ksh, or zsh
When virtualenv is active, your shell prompt is prefixed with
Install packages within a virtual environment without affecting the host system setup. Start by upgrading pip:
pip install --upgrade pip pip list # show packages installed within the virtual environment
And to exit virtualenv later:
deactivate # don't exit until you're done using your virtual environment
# add a bash alias in .bashrc for venv activation # for zsh shell this is .zshrc alias venv="source /home/fermi/venv/bin/activate" source ~/.bashrc # run venv to activate virtual environment venv
Install Jupyter as
sudo python3 -m pip install jupyter # system-wide install
user where you created virtual environment
Setup A notebook configuration file. Check to see if you have a notebook configuration file, jupyter_notebook_config.py. The default location for this file is your Jupyter folder located in your home directory:
- Windows: C:\Users\USERNAME.jupyter\jupyter_notebook_config.py
- OS X: /Users/USERNAME/.jupyter/jupyter_notebook_config.py
- Linux: /home/USERNAME/.jupyter/jupyter_notebook_config.py
If you don’t already have a Jupyter folder, or if your Jupyter folder doesn’t contain a notebook configuration file, run the following command:
jupyter notebook --generate-config # output Writing default config to: /home/fermi/.jupyter/jupyter_notebook_config.py
Navigate to jupyter_notebook_config.py in the ~/[PATH] specified above and open with an editor.
Set configuration for Jupyter
jupyter_notebook_config.py. For more info refer to documentation.
Change ip and port as you see fit. IP
0.0.0.0 will broadcast in LAN incase you want to access from another host other than localhost.
## The IP address the notebook server will listen on. c.NotebookApp.ip = '0.0.0.0' #default= localhost ## The port the notebook server will listen on. c.NotebookApp.port = 7000 #default=8888
Startinf Jupyter for the first time
Start the notebook
jupyter-notebook # output [I 11:33:48.980 NotebookApp] Writing notebook server cookie secret to /run/user/1002/jupyter/notebook_cookie_secret [I 11:33:49.355 NotebookApp] Serving notebooks from local directory: /home/fermi [I 11:33:49.355 NotebookApp] The Jupyter Notebook is running at: [I 11:33:49.355 NotebookApp] http://(TSERV or 127.0.0.1):7000/?token=ea0e3be651a7244793651e7aadc6e8994d4db30ce5d8ef1b [I 11:33:49.355 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation). [W 11:33:49.356 NotebookApp] No web browser found: could not locate runnable browser. [C 11:33:49.356 NotebookApp] Copy/paste this URL into your browser when you connect for the first time, to login with a token: http://(TSERV or 127.0.0.1):7000/?token=ea0e3be651a7244793651e7aadc6e8994d4db30ce5d8ef1b [I 11:34:18.338 NotebookApp] 302 GET /?token=ea0e3be651a7244793651e7aadc6e8994d4db30ce5d8ef1b (220.127.116.11) 0.89ms
Visit the url from the commandline output in you browser. In this example the url is:
http://(TSERV or 127.0.0.1):7000/?token=ea0e3be651a7244793651e7aadc6e8994d4db30ce5d8ef1b.
You can also replace
http://(TSERV or 127.0.0.1) with the
HOSTNAME of the server/ machine. You should be able to login.
Logout and then login again: Set a password
When you logout and login the second time you will be asked to set password. Use the
token and set a new password which will then be encrypted now saved in the
/home/fermi/.jupyter/jupyter_notebook_config.json. You can manually check contents of this file.
Now exit the server ^C and start the server again. You wont see any token generated since a password auth is in place. Now we are ready to deply this as
jupyter a service.
Setting up notebook server as a service
Setup Environment for your service. I use a file for setting all the ENVIRONMENT_VARIABLES for this service to use
# create env file in /home/fermi/.jupyter/env touch /home/fermi/.jupyter/env
Add ENVIRONMENT VARIABLES in to the file as follows. Add
CUDA libraries and paths if you have.
You can lookup your
.bashrc to check if you have custom variables.
# file: /home/fermi/.jupyter/env # Note that I am including venv path so that jupyter.service uses this venv binaries. PATH=/home/fermi/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/local/cuda/bin:$PATH # in case you have GPU installed provide paths to CUDA libs LD_LIBRARY_PATH=/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64:$LD_LIBRARY_PATH CUDA_HOME=/usr/local/cuda CUDA_VISIBLE_DEVICES=0
Now login as
# create a service file sudo touch /lib/systemd/system/jupyter.service # open and edit sudo nano /lib/systemd/system/jupyter.service
The following is a modified version of this service config.
# service name: jupyter.service # path: /lib/systemd/system/jupyter.service # After Ubuntu 16.04, Systemd becomes the default. # Ref: https://gist.github.com/whophil/5a2eab328d2f8c16bb31c9ceaf23164f [Unit] Description=Jupyter Notebook Server [Service] Type=simple PIDFile=/run/jupyter.pid EnvironmentFile=/home/fermi/.jupyter/env # Jupyter Notebook: change PATHs as needed for your system ExecStart=/usr/local/bin/jupyter-notebook --config=/home/fermi/.jupyter/jupyter_notebook_config.py # In case you want to use jupyter-lab # ExecStart=/usr/local/bin/jupyter-lab --config=/home/fermi/.jupyter/jupyter_notebook_config.py User=fermi Group=fermi WorkingDirectory=/home/fermi/users Restart=always RestartSec=10 #KillMode=mixed [Install] WantedBy=multi-user.target
Recap and a things to note:
- service will run as user
fermi. Since we have provided default access to fermi's home directory and our workspace
- the python venv is enabled by using settings
- you can change the default to
By default ipykernel is installed along with jupyter when we installed the jupyter notebook in previous step. However we also have to provide a kernel for the virtual environment we just created.
user and list all existing kernels
Create a custom kernel with python3 from the virtual env with a custom name venvPy3
source ~/venv/bin/activate which python # this should echo > /home/fermi/venv/bin/python python --version # check version just to make sure its the same we installed > Python 3.5.2 # from venv install kernel python3 -m ipykernel install --user --name=venvPy3 jupyter-kernelspec list Available kernels: venvpy3 /home/fermi/.local/share/jupyter/kernels/venvpy3 python3 /usr/local/share/jupyter/kernels/
You should also be able to see native kernels as shown here
python3 from native python in
This is happening because of the ipykernel that is installed in the systems python libraries.
To install packages with native kernels use
This is however not required for the virtualenv.
For our application we would like to disable native kernels and provide user access to only the venv for simplicity. We can add as many kernels as we like in the venv.
First check where is ipykernel installed. Login as
pip3 list | grep ipykernel
ipykernel from system python libs.
sudo python3 -m pip uninstall ipykernel
Login as jupyter notebook
user and activate
venv. Now from install
ipykernel in venv.
# from venv install kernel pip install ipykernel # list existing kernels # install custom venv kernel python3 -m ipykernel install --user --name=venvPy3
You can add as many kernel as needed for multiple versions of python from within the venv.
All kernels created inside venv are available in the
user home at
Start the jupyter service
Now we are ready to start the service. Enable and start the daemon
# Enable the service sudo systemctl enable jupyter.service Created symlink from /etc/systemd/system/multi-user.target.wants/jupyter.service to /lib/systemd/system/jupyter.service. # Reload and Restart Service sudo systemctl daemon-reload sudo systemctl restart jupyter.service # Now check status of the jupyter service daemon sudo service jupyter status
output : the daemon is running
● jupyter.service - Jupyter Notebook Server Loaded: loaded (/lib/systemd/system/jupyter.service; enabled; vendor preset: enabled) Active: active (running) since Sat 2018-11-24 12:09:05 PST; 21s ago Main PID: 8012 (jupyter-noteboo) Tasks: 1 Memory: 40.6M CPU: 824ms CGroup: /system.slice/jupyter.service └─8012 /usr/bin/python3 /usr/local/bin/jupyter-notebook --config=/home/fermi/.jupyter/jupyter_notebook_config.py
login to the jupyter notebook and check
Visit the URL, create a new notebook.
Verify the venv is loaded
! python --version ! pip --version ! which python ! which pip
should generate output
Python 3.5.2 pip 18.1 from /home/fermi/venv/lib/python3.5/site-packages/pip (python 3.5) /home/fermi/venv/bin/python /home/fermi/venv/bin/pip
As you see the venv is active. Also you should be able to see the 'Python3' venv kernel in the kernel list.
To check kernels available run this from commandline
jupyter-kernelspec list python3 /usr/local/share/jupyter/kernels/python3
To add and rename kernels please refer to
Check permissions for pip install from jupyter
From a new notebook run:
! pip install pandas
If you dont get any permission errors during the install you are good to go.
This concludes the full installation process for the Jupyer Notebook Server as a service daemon.