...
Download neo4j-community from http://neo4j.com/download/. NORDUnet and SUNET run 2.1.8. 2.3.2 has been tested and did not work as expected.
Code Block |
---|
$ tar xvfz neo4j-community-2.1.78-unix.tar.gz $ sudo mv neo4j-community-2.1.78 /var/opt/. $ sudo ln -s /var/opt/neo4j-community-2.1.78 /var/opt/neo4j-community $ cd /var/opt/neo4j-community $ sudo ./bin/neo4j-installer install |
...
Code Block |
---|
sudo chown -R ni:www-data /tmp/django_cache sudo chmod -R g+w /tmp/django_cache sudo chown -R ni:www-data /var/opt/norduni/norduni/src/niweb/logs/ sudo chmod -R g+w /var/opt/norduni/norduni/src/niweb/logs/ |
nginx
Setup new dhparam file 2048 should suffice, but if you like you can go with 4096 instead:
Code Block |
---|
$ sudo openssl dhparam -out /etc/ssl/dhparams.pem 2048 |
Configure nginx.
Code Block | ||
---|---|---|
| ||
$ sudo vi /etc/nginx/sites-available/default The following configuration should be a good start. upstream django { server 127.0.0.1:8001; # for a web port socket } server { listen 80; listen [::]:80; server_name ni.nordu.net; rewrite ^ https://$server_name$request_uri? permanent; } server { listen 443; listen [::]:443 default ipv6only=on; ## listen for ipv6 ssl on; ssl_certificate /etc/ssl/ni_nordu_net.crt; ssl_certificate_key /etc/ssl/ni_nordu_net.key; # PFS settings from httphttps://people.adams.edu/~cdmiller/posts/NSA_SSL_settings_for_nginx_and_apache/cipherli.st # NOTE: these settings exclude Win XP with IE 6ssl_prefer_server_ciphers on; # ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_prefer_server_ciphers on"EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH"; ssl_ciphers DHE-RSA-CAMELLIA256-SHA:DHE-RSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:DES-CBC3-SHA:!ADH:!aNULLsession_cache shared:SSL:10m; ssl_ecdh_curve secp384r1; ssl_dhparam /etc/ssl/dhparams.pem; server_name ni.nordu.net; location /static/ { alias /var/opt/norduni/norduni/src/niweb/niweb/static/; autoindex on; access_log off; expires 30d; } location / { include /etc/nginx/uwsgi_params; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_redirect off; uwsgi_pass django; } } |
...
You then need to uncomment the lines in settings.py that imports and sets up djangosaml2. You also have to create a pysaml2 configuration.
All this is best described in the documentation at https://pypi.python.org/pypi/djangosaml2.
...
Local saml metadata
To insert data you need to stop any python process that is using the Neo4j database. We hope to get the option to load more database instances in read-only mode in a near future then this could be avoided.
NORDUnet has a GIT repository called nistore and it is cloned to /var/opt/norduni/nistore/.
To start have a look at the NERDS README then clone the NERDS project.
Code Block |
---|
cd /var/opt/norduni/
mkdir tools
cd tools
git clone https://github.com/fredrikt/nerds.git
|
speed up login you can use local metadata. This metadata still needs to be updated and verified, and for that you can use https://github.com/NORDUnet/metadata-updater
You need to configure djangosaml2 to use local metadata, and you will have to add the meta-dataupdater to cron, preferably by running crontab -e as the ni user. Once an hour is reasonable, once a day can be ok, once a week might be tiresome when the cert expires.
Collecting and processing network data
To insert data you need to stop any python process that is using the Neo4j database. We hope to get the option to load more database instances in read-only mode in a near future then this could be avoided.
NORDUnet has a GIT repository called nistore and it is cloned to /var/opt/norduni/nistore/.
To start have a look at the NERDS README then clone the NERDS project.
Code Block |
---|
cd /var/opt/norduni/
mkdir tools
cd tools
git clone https://github.com/fredrikt/nerds.git
|
Juniper Configuration Producer/Juniper Configuration Producer/Consumer
The Juniper configuration producer can load Juniper configuration directly from the router via SSH or Juniper configuration files in XML format from disk.
...
Change the path at the top of the script to be able to import norduni_client.py.
Code Block |
---|
[data]
juniper_conf = /path/to/juniper/json
nmap_services = /path/to/nmap/json
alcatel_isis = /path/to/alcate/json
noclook = #Used for loading backup.
|
Then run:
Code Block |
---|
python noclook_consumer.py -C template.conf -I
|
Setting up a local/development NOCLook
[data]
juniper_conf = /path/to/juniper/json
nmap_services = /path/to/nmap/json
alcatel_isis = /path/to/alcate/json
noclook = #Used for loading backup.
|
Then run:
Code Block |
---|
python noclook_consumer.py -C template.conf -I
|
Setting up a local/development NOCLook
Code Block |
---|
# Clone a convenience repo
$ git clone https://github.com/NORDUnet/norduni-developer
$ cd norduni-developer
# Start dependencies
$ ./start.sh
# Clone NOCLook project repo
$ git clone https://git.nordu.net/norduni.git
$ cd norduni |
Code Block |
$ git clone https://git.nordu.net/norduni.git # Download neo4j docker image and start it $ docker pull tpires/neo4j $ docker run -d -v /path_to_repo/norduni/docker/neo4j.properties:/var/lib/neo4j/conf/neo4j.properties -v /opt/docker/neo4jdata:/var/lib/neo4j/data -p 7474:7474 tpires/neo4j # Create the indexes with curl $ curl -D - -H "Content-Type: application/json" --data '{"name" : "node_auto_index","config" : {"type" : "fulltext","provider" : "lucene"}}' -X POST http://localhost:7474/db/data/index/node/ $ curl -D - -H "Content-Type: application/json" --data '{"name" : "relationship_auto_index","config" : {"type" : "fulltext","provider" : "lucene"}}' -X POST http://localhost:7474/db/data/index/relationship/ # Create a virtualenv and activate it $ virtualenv env $ . env/bin/activate # Install the python packages $ pip install paver $ pip install -r /path_to_repo/requirements/dev.txt # Create a settings file $ cp /path_to_repo/src/niweb/dotenv /path_to/repo/src/niweb/.devenv # Sync the db $ python /path_to_repo/src/niweb/manage.py syncdb $ python /path_to_repo/src/niweb/manage.py migrate apps.noclook # Run the app $ python /path_to_repo/src/niweb/manage.py migrate actstream $ python /path_to_repo/src/niweb/manage.py migrate tastypie runserver |
Upgrading to newest versions
This is the general procedure for upgrading to newest version of norduni.
Code Block |
---|
# stash current local changes and update $ git stash $ git pull origin master $ git stash apply # Run the appmigrations $ python /path_to_repo/src/niweb/manage.py runservermigrate # OptionalPip postgres instead of sqlite3, don't forget to change database settings in settings.py. # Download postgres docker image and start it $ docker pull orchardup/postgresql $ docker run -d -p 5432:5432 -e POSTGRESQL_USER=ni -e POSTGRESQL_PASS=docker -e POSTGRESQL_DB=norduni -v /opt/docker/postgresql_data/:/var/lib/postgresql/ orchardup/postgresql update requirements $ pip install -U -r requirements/prod.txt # Collect statics $ python /path_to_repo/src/niweb/manage.py collectstatic # Restart uwsgi $ sudo services uwsgi restart |