...
title | Work in progress |
---|
Table of Contents |
---|
Setting up NOCLook
...
Code Block |
---|
$ sudo apt-get install python-setuptools git libpq-dev postgresql python-dev postfix nginx uwsgi $ sudo easy_install pip $ sudo pip install virtualenv $ sudo adduser --disabled-password --home /var/opt/norduni ni # sudo apt-get install git-core python-virtualenv openjdk-6-jdk build-essential postgresql python-psycopg2 libpq-dev python-dev |
We are using postgresql but you can use any SQL database that Django supports. See Django database documentation for other supported SQL databases.
...
Neo4j database
Set password for database user and create a new databaseOracle java is recommended for Neo4j.
Code Block |
---|
$ sudo apt-u postgres psql postgres \password postgres Write password Write password again Ctrl+D sudo -u postgres createdb norduni |
NORDUni repository
add-repository ppa:webupd8team/java
$ sudo apt-get update
$ sudo apt-get install oracle-java8-installer |
Download neo4j-community from http://neo4j.com/download/Get the NORDUni files.
Code Block |
---|
$ sudo -u ni -i $ pwd tar xvfz neo4j-community-2.1.7-unix.tar.gz $ sudo mv neo4j-community-2.1.7 /var/opt/norduni. $ git clone git://git.nordu.net/norduni.git |
Python environment
Make a virtual python environment.
Code Block |
---|
$ virtualenv norduni_environment
|
Making a virtual environment is also just a suggestion but it makes it easier to keep your system clean.
Python requirements
Install required python modules.
Code Block |
---|
$ . norduni_environment/bin/activate
$ pip install -r norduni/requirements.txt
|
Django settings
Change the django settings.
Code Block |
---|
cd norduni/src/niweb/niweb/
cp generic_settings.py settings.py
vi settings.py
|
Change at least the following settings.
Code Block |
---|
# Database settings
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'norduni',
'USER': 'postgres',
'PASSWORD': 'secret',
'HOST': 'localhost'
}
}
|
Neo4j >1.5 embedded with Python bindings
Install JPype and Neo4j-embedded.
Download jpype. (http://sourceforge.net/projects/jpype/files/)
Code Block |
---|
pip install neo4j-embedded
export JAVA_HOME=/usr/lib/jvm/java-6-openjdk/
pip install /path/to/JPype-version.zip
|
Code Block |
---|
# Django Generic Login
(r'^accounts/login/$', 'django.contrib.auth.views.login'),
# Federated login
#(r'^accounts/', include('niweb.apps.fedlogin.urls')),
|
Code Block |
---|
python manage.py syncdb
python manage.py runserver 0.0.0.0:80
|
Now you should be able surf to your machines ip and see the NOCLook app started.
It is time to collect and insert some data.
Deploying NOCLook
Comment out the static media url in /opt/norduni/src/niweb/urls.py.
Code Block | ||
---|---|---|
| ||
# Static serve
#(r'^site_media/(?P<path>.*)$', 'django.views.static.serve',
# {'document_root': settings.STATIC_DEV_MEDIA}),
|
Install nginx, postfix and gunicorn.
Code Block |
---|
sudo apt-get install nginx postfix
pip install gunicorn
|
Create a gunicorn start file.
...
title | /opt/norduni/src/start_noclook.sh |
---|
...
sudo ln -s /var/opt/neo4j-community-2.1.7 /var/opt/neo4j-community
$ cd /var/opt/neo4j-community
$ sudo ./bin/neo4j-installer install |
Set property keys to auto index in neo4j.
Code Block |
---|
$ sudo vi /var/opt/neo4j-community/conf/neo4j.properties
Add or update the following lines.
# Autoindexing
# Enable auto-indexing for nodes, default is false
node_auto_indexing=true
# The node property keys to be auto-indexed, if enabled
node_keys_indexable=name, description, ip_address, ip_addresses, as_number, hostname, hostnames, telenor_tn1_number, nordunet_id, version
# Enable auto-indexing for relationships, default is false
relationship_auto_indexing=true
# The relationship property keys to be auto-indexed, if enabled
relationship_keys_indexable=ip_address |
Increase the number of files the neo4j user may concurrently access. A restart is required for the settings to take effect.
Code Block |
---|
$ sudo vi /etc/security/limits.conf
Add the lines below to limits.conf.
# User neo4j allowed concurrent files
neo4j soft nofile 40000
neo4j hard nofile 40000 |
Code Block |
---|
$ sudo vi /etc/pam.d/su
Uncomment the following line.
session required pam_limits.so |
After the restart neo4-service should be running.
Code Block |
---|
$ sudo service neo4j-service status
Neo4j Server is running at pid 1475 |
Create full text index for nodes and relationships.
Code Block |
---|
$ curl -D - -H "Content-Type: application/json" --data '{"name" : "node_auto_index","config" : {"type" : "fulltext","provider" : "lucene"}}' -X POST http://localhost:7474/db/data/index/node/
HTTP/1.1 201 Created
*snip*
$ curl -D - -H "Content-Type: application/json" --data '{"name" : "relationship_auto_index","config" : {"type" : "fulltext","provider" : "lucene"}}' -X POST http://localhost:7474/db/data/index/relationship/
HTTP/1.1 201 Created
*snip* |
Postgres database
Set password for database user and create a new database
Code Block |
---|
sudo -u postgres psql postgres
\password postgres
Write password
Write password again
Ctrl+D
sudo -u postgres createdb norduni
|
NORDUni repository
Get the NORDUni files.
Code Block |
---|
$ sudo -u ni -i
$ pwd
/var/opt/norduni
$ git clone git://git.nordu.net/norduni.git
|
Python environment
Make a virtual python environment.
Code Block |
---|
$ virtualenv norduni_environment
|
Making a virtual environment is also just a suggestion but it makes it easier to keep your system clean.
Python requirements
Install required python modules.
Code Block |
---|
$ . norduni_environment/bin/activate
$ pip install -r norduni/requirements.txt
|
Django settings
Change the django settings.
Code Block |
---|
$ cd norduni/src/niweb/niweb/
$ cp generic_settings.py settings.py
$ vi settings.py
|
Change at least the database settings.
Code Block |
---|
# Database settings
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'norduni',
'USER': 'postgres',
'PASSWORD': 'secret',
'HOST': 'localhost'
}
}
|
Check if your settings are ok.
Code Block |
---|
$ python manage.py syncdb
$ python manage.py runserver 0.0.0.0:80
|
Now you should be able connect to your machines with your browser and see the NOCLook app index page.
Note | ||
---|---|---|
| ||
Below is not done yet. |
Deploying NOCLook
Create a uwsgi configuration file.
Configure nginx.
Code Block | ||
---|---|---|
| ||
server { listen 80; root /opt/norduni/src/niweb; server_name ni.example.net; access_log /var/log/ni/noclook-access.log; error_log /var/log/ni/noclook-error.log; location /static/ { root /opt/norduni/src/niweb/; autoindex on; access_log off; expires 30d; } location / { proxy_pass_header Server; proxy_set_header Host $http_host; proxy_redirect off; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Scheme $scheme; proxy_connect_timeout 10; proxy_read_timeout 10; proxy_pass http://localhost:8000/_addr; } } |
Install supervisord and set up the following start script.
Code Block |
---|
easy_install supervisor
echo_supervisord_conf > /etc/supervisord.conf
|
Code Block | ||
---|---|---|
| ||
description proxy_set_header X-Scheme $scheme; "supervisord" start on runlevel [2345] stop on runlevel [!2345] respawn exec /usr/local/bin/supervisord --nodaemon --configuration /etc/supervisord.conf |
Add the noclook start script to the supervisor configuration.
...
proxy_connect_timeout 10;
proxy_read_timeout 10;
proxy_pass http://localhost:8000/;
}
}
|
Collecting and processing network data
...