Skip to content

Getting v0.7.0-dev to work with gunicorn / celery

Hi,

I've upgraded burp-ui 0.6.6 to 0.7.0-dev, and can't seem to get it to work with gunicorn. Running directly with flask (eg /usr/local/bin/burp-ui -c /etc/burp/burpui.cfg -- -h 0.0.0.0 -p 5000) works.

Updated burpui.cfg file according to breakings stated in the documentation. Still, no way to get burp-ui running with gunicorn.

Here's my output:

gunicorn -k gevent -w 4 'burpui:create_app(conf="/etc/burp/burpui.cfg")'
[2019-06-20 12:03:55 +0200] [5341] [INFO] Starting gunicorn 19.9.0
[2019-06-20 12:03:55 +0200] [5341] [INFO] Listening at: http://127.0.0.1:8000 (5341)
[2019-06-20 12:03:55 +0200] [5341] [INFO] Using worker: gevent
[2019-06-20 12:03:55 +0200] [5344] [INFO] Booting worker with pid: 5344
[2019-06-20 12:03:55 +0200] [5345] [INFO] Booting worker with pid: 5345
[2019-06-20 12:03:55 +0200] [5346] [INFO] Booting worker with pid: 5346
[2019-06-20 12:03:55 +0200] [5347] [INFO] Booting worker with pid: 5347
[2019-06-20 12:03:56 +0200] [5344] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/ggevent.py", line 203, in init_process
    super(GeventWorker, self).init_process()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
    self.load_wsgi()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
    return self.load_wsgiapp()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 362, in import_app
    app = eval(obj, vars(mod))
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.6/site-packages/burpui/app.py", line 161, in create_app
    if app.storage and app.storage.lower() == 'redis':
AttributeError: 'ProxyFix' object has no attribute 'storage'
[2019-06-20 12:03:56 +0200] [5344] [INFO] Worker exiting (pid: 5344)
[2019-06-20 12:03:56 +0200] [5345] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/ggevent.py", line 203, in init_process
    super(GeventWorker, self).init_process()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
    self.load_wsgi()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
    return self.load_wsgiapp()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 362, in import_app
    app = eval(obj, vars(mod))
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.6/site-packages/burpui/app.py", line 161, in create_app
    if app.storage and app.storage.lower() == 'redis':
AttributeError: 'ProxyFix' object has no attribute 'storage'
[2019-06-20 12:03:56 +0200] [5345] [INFO] Worker exiting (pid: 5345)
[2019-06-20 12:03:56 +0200] [5346] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/ggevent.py", line 203, in init_process
    super(GeventWorker, self).init_process()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
    self.load_wsgi()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
    return self.load_wsgiapp()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 362, in import_app
    app = eval(obj, vars(mod))
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.6/site-packages/burpui/app.py", line 161, in create_app
    if app.storage and app.storage.lower() == 'redis':
AttributeError: 'ProxyFix' object has no attribute 'storage'
[2019-06-20 12:03:56 +0200] [5346] [INFO] Worker exiting (pid: 5346)
[2019-06-20 12:03:56 +0200] [5347] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/ggevent.py", line 203, in init_process
    super(GeventWorker, self).init_process()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
    self.load_wsgi()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
    return self.load_wsgiapp()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 362, in import_app
    app = eval(obj, vars(mod))
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.6/site-packages/burpui/app.py", line 161, in create_app
    if app.storage and app.storage.lower() == 'redis':
AttributeError: 'ProxyFix' object has no attribute 'storage'
[2019-06-20 12:03:56 +0200] [5347] [INFO] Worker exiting (pid: 5347)
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 210, in run
    self.sleep()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 360, in sleep
    ready = select.select([self.PIPE[0]], [], [], 1.0)
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 245, in handle_chld
    self.reap_workers()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 525, in reap_workers
    raise HaltServer(reason, self.WORKER_BOOT_ERROR)
gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/bin/gunicorn", line 11, in <module>
    sys.exit(run())
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 61, in run
    WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 223, in run
    super(Application, self).run()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 72, in run
    Arbiter(self).run()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 232, in run
    self.halt(reason=inst.reason, exit_status=inst.exit_status)
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 345, in halt
    self.stop()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 393, in stop
    time.sleep(0.1)
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 245, in handle_chld
    self.reap_workers()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 525, in reap_workers
    raise HaltServer(reason, self.WORKER_BOOT_ERROR)
gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>

Running Celery confirms my problem:

juin 20 12:23:42 somehost.local systemd[1]: Started Burp-UI celery service.
juin 20 12:23:44 somehost.local bui-celery[5688]: Error:
juin 20 12:23:44 somehost.local bui-celery[5688]: Unable to load celery application.
juin 20 12:23:44 somehost.local bui-celery[5688]: 'proxyfix' object has no attribute 'storage'
juin 20 12:23:44 somehost.local systemd[1]: bui-celery.service: main process exited, code=exited, status=1/FAILURE
juin 20 12:23:44 somehost.local systemd[1]: Unit bui-celery.service entered failed state.
juin 20 12:23:44 somehost.local systemd[1]: bui-celery.service failed.

My burpui.cfg file:

[Global]
backend = multi
auth = basic
acl = basic
audit = basic
plugins = none

[UI]
refresh = 180
liverefresh = 5
ignore_labels = color:.*, custom:.*
format_labels = s/^os:\s*//
default_strip = 0

[Production]
storage = redis
session = redis
cache = redis
redis = localhost:6379
celery = true
database = sqlite:////var/lib/burpui/store.db
limiter = false
ratio = 60/minute
prefix = none
num_proxies = 0
proxy_fix_args = "{'x_for': {num_proxies}, 'x_host': {num_proxies}, 'x_prefix': {num_proxies}}"

[WebSocket]
enabled = true
embedded = false
broker = redis
url = none
debug = false

[Security]
includes = /etc/burp
enforce = false
revoke = true
cookietime = 1
sessiontime = 1
scookie = true
appsecret = somestring

[Experimental]
zip64 = false
noserverrestore = false

[Burp]
burpbin = /usr/sbin/burp
stripbin = /usr/sbin/vss_strip
bconfcli = /etc/burp/burp.conf
bconfsrv = /etc/burp/burp-server.conf
tmpdir = /tmp/bui
timeout = 15
deep_inspection = false

[Parallel]
host = ::1
port = 11111
timeout = 15
password = password123456
ssl = true
concurrency = 2
init_wait = 15

[BASIC:AUDIT]
priority = 100
level = WARNING
logfile = /var/log/burp-ui.log
max_bytes = 30 * 1024 * 1024
rotate = 5


[BASIC:AUTH]
priority = 100
admin = pbkdf2:sha256:Somestuff



extended = pbkdf2:sha256:Somestuff
assume_rw = pbkdf2:sha256:Somestuff
legacy = pbkdf2:sha256:Somestuff

[BASIC:ACL]
priority = 100
admin = admin
superadmin = '{"agents":{"agent1":{"rw":["client.*","server.*"]},"agent2":{"rw":["client.*","server.*"]}}}'
+moderator = "", superadmin

[Agent:agent1]
host = 127.0.0.1
port = 10000
password = somepass
ssl = false
timeout = 300

[Agent:agent2]
host = 127.0.0.1
port = 10001
password = someotherpass
ssl = false
timeout = 300

I noticed that the booleans of [ACL] get rewritten to passwords btw, tested that twice.

extended = pbkdf2:sha256:Somestuff
assume_rw = pbkdf2:sha256:Somestuff
legacy = pbkdf2:sha256:Somestuff

I've tried to comment out num_proxies and proxy_fix_args, but the result is the same.

System: CentOS 7.6 x64 uptodate with python3.6 (hence using /usr/local/bin instead of /usr/bin for most stuff)

Thanks

Edited by Orsirs de Jong