Serving IOP4 in production#
Setting up a production web server#
Warning
This section requires some familiarity with system administration and web server configuration.
IOP4 comes with the iop4site project to allow a fast and easy way of serving results. However, it should only be used for local debugging and not offered to the public. For production use, you should use a proper web server. Here we provide some details on how to set up IOP4 with nginx and gunicorn. This guide is not complete and should be taken only as a starting point, since setting up a real server is a complex task and depends on how you want to integrate IOP4 with your existing infrastructure.
This setup has been tested on Ubuntu Server 22.04 LTS. You need to install nginx and gunicorn (within your virtual environment).
Example nginx site configuration (to be placed at /etc/nginx/sites-available/)
server {
listen 80;
server_name _;
return 301 $scheme://domain$request_uri;
}
server {
server_name domain domain2;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
alias /path/to/static/;
}
location / {
include proxy_params;
client_max_body_size 300M;
proxy_redirect off;
proxy_buffering off;
proxy_pass http://unix:/run/gunicorn.sock:/;
}
}
where you should replace domain, domain2 with the domain names that you
will be using, and /path/to/static/ should be accessible by the user
running nginx (e.g. under /var/www/html/mysite/static/).
Example gunicorn socket configuration (to be placed at /etc/systemd/system/)
[Unit]
Description=gunicorn socket
[Socket]
ListenStream=/run/gunicorn.sock
[Install]
WantedBy=sockets.target
Example gunicorn service configuration (to be placed at /etc/systemd/system/)
[Unit]
Description=gunicorn daemon
Requires=gunicorn.socket
After=network.target
[Service]
User=vhega
Group=www-data
WorkingDirectory=/path/to/your/django/site
ExecStart=/path/to/gunicorn/executable \
--access-logfile - \
--worker-class 'gevent' \
--workers 8 \
--timeout 30 \
--bind unix:/run/gunicorn.sock \
vhegasite.wsgi:application
[Install]
WantedBy=multi-user.target
where /path/to/your/django/site is the path to your Django site project
(your modified iop4site or a different one) and
/path/to/gunicorn/executable is the path to the gunicorn executable (usually
within your virtual environment, you can get it by typing which gunicorn
when your virtual environment is activated).
The most important settings to change in your new settings.py file within
your Django project are:
SECRET_KEY = "new generated key"
# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True
INTERNAL_IPS = []
ALLOWED_HOSTS = ["domain", "domain2"]
# Configure static files to the path served by nginx
STATIC_ROOT = '/path/to/static/'
Creating a cron job for reducing new observations#
You might be interested in creating a cron job that routinely reduces new observations every morning. Here we provide an example. This assumes you are working in a linux system. Other OS provide different methods to create jobs.
Use crontab -e to edit your crontab file and add a line like the following
00 08 * * * /home/vhega/run_iop4_daily.sh > /home/vhega/run_iop4_daily.log 2>&1
Then, create a file run_iop4_daily.sh, give it execution permissions (chmod +x run_iop4_daily.sh) and add the following content:
#!/usr/bin/bash
# save the current datetime
printf -v date '%(%Y-%m-%d_%H%M)T' -1
echo "#########################################"
echo "Run daily job for IOP4: $date"
echo "#########################################"
. /home/vhega/miniconda3/bin/activate iop4
# make sure all files created by iop4 are editable by the current user only
umask 0022
# Run iop4 for new observations (i.e. last night)
iop4 --discover-missing -o log_file=/home/vhega/iop4data/logs/daily_$date.log
# save return code
rc=$?
# Create and send a summary of the results for last night, passing the return code
iop4-night-summary --fromaddr '{{YOUR SENDER ADDRESS}}' \
--mailto '{{ADDRESS 1}},{{ADDRESS 2}},{{ADDRESS 3}}' \
--contact-name '{{CONTACT NAME}}' \
--contact-email '{{CONTACT EMAIL}}' \
--site-url '{{DEPLOYMENT SITE URL}}' \
--saveto "/home/vhega/iop4data/logs/daily_$date.html"
--rc $rc
The above script will run iop4 every morning, disovering and proccessing new observations.
The last command is optional, and will send an email with a summary with the
results from last night to the specified email addresses.
The cron job does not need to be run on a daily basis, and you can run it
whenever you expect new observations to become available in the telescope
remote archives (e.g. every few hours). Alternatively, you can pass an argument
to iop4-night-summary specifying the night that you want to generate the
summary for. The email will be in HTML format (viewable in any modern browser or
email client), and can optionally be saved to any path. You can indicate the url
of your deployed site so the links in the email (e.g. for files with error)
point directly to the corresponding page in the iop4 web interface or admin
site.
Note: for iop4-night-summary to be able to send an email, your computer needs a local SMTP server (e.g. postfix).