I've graphite server and intermittently its daemon fails with this error,
HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - string 'datapoints' not found on 'http://192.168.12.15:8000/render/?target=sys.example_com.snmp.if_octets-Gi0_0_0.rx&format=json&from=-5min' - 723 bytes in 0.002 second response time |time=0.002067s;;;0.000000 size=723B;;;0
So I restarted apache which fixed the issue. But I would like to fix the root cause of this issue, I can't restart apache every time. I want to fix this permanently. This is what I found in error logs,
[Tue Aug 01 20:16:01] [wsgi:error] RuntimeError: populate() isn't reentrant
[Tue Aug 01 20:16:21] [wsgi:error] Target WSGI script '/usr/lib/python2.7/dist-packages/graphite/wsgi.py' cannot be loaded as Python module.
[Tue Aug 01 20:16:21] [wsgi:error] Exception occurred processing WSGI script '/usr/lib/python2.7/dist-packages/graphite/wsgi.py'.
[Tue Aug 01 20:16:21] [wsgi:error] Traceback (most recent call last):
[Tue Aug 01 20:16:21] [wsgi:error] File "/usr/lib/python2.7/dist-packages/graphite/wsgi.py", line 14, in <module>
[Tue Aug 01 20:16:21] [wsgi:error] application = get_wsgi_application()
[Tue Aug 01 20:16:21] [wsgi:error] File "/usr/lib/python2.7/dist-packages/django/core/wsgi.py", line 14, in get_wsgi_application
[Tue Aug 01 20:16:21] [wsgi:error] django.setup()
[Tue Aug 01 20:16:21] [wsgi:error] File "/usr/lib/python2.7/dist-packages/django/__init__.py", line 21, in setup
[Tue Aug 01 20:16:21] [wsgi:error] apps.populate(settings.INSTALLED_APPS)
[Tue Aug 01 20:16:21] [wsgi:error] File "/usr/lib/python2.7/dist-packages/django/apps/registry.py", line 78, in populate
This is my /usr/lib/python2.7/dist-packages/graphite/wsgi.py
file,
import os
import sys
try:
from importlib import import_module
except ImportError:
from django.utils.importlib import import_module
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'graphite.settings') # noqa
from django.conf import settings
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
try:
import whitenoise
except ImportError:
whitenoise = False
else:
# WhiteNoise < 2.0.1 does not support Python 2.6
if sys.version_info[:2] < (2, 7):
whitenoise_version = tuple(map(
int, getattr(whitenoise, '__version__', '0').split('.')))
if whitenoise_version < (2, 0, 1):
whitenoise = False
if whitenoise:
from whitenoise.django import DjangoWhiteNoise
application = DjangoWhiteNoise(application)
prefix = "/".join((settings.URL_PREFIX.strip('/'), 'static'))
for directory in settings.STATICFILES_DIRS:
application.add_files(directory, prefix=prefix)
for app_path in settings.INSTALLED_APPS:
module = import_module(app_path)
directory = os.path.join(os.path.dirname(module.__file__), 'static')
if os.path.isdir(directory):
application.add_files(directory, prefix=prefix)
Update
Here is my config file for graphite which I'm using,
<VirtualHost *:8000>
ServerName graphite-web
## Vhost docroot
DocumentRoot "/var/www"
## Alias declarations for resources outside the DocumentRoot
Alias /static "/var/lib/graphite/webapp/content"
## Directories, there should at least be a declaration for /var/www
<Directory "/var/www">
Options Indexes FollowSymLinks MultiViews
AllowOverride None
Require all granted
</Directory>
## Logging
ErrorLog "/var/log/apache2/graphite-web_error.log"
ServerSignature Off
CustomLog "/var/log/apache2/graphite-web_access.log" combined
WSGIScriptAlias / "/usr/lib/python2.7/dist-packages/graphite/wsgi.py"
</VirtualHost>
What is causing this problem ? Any help on this please ?