Getting v0.7.0-dev to work with gunicorn / celery
Hi,
I've upgraded burp-ui 0.6.6 to 0.7.0-dev, and can't seem to get it to work with gunicorn.
Running directly with flask (eg /usr/local/bin/burp-ui -c /etc/burp/burpui.cfg -- -h 0.0.0.0 -p 5000
) works.
Updated burpui.cfg file according to breakings stated in the documentation. Still, no way to get burp-ui running with gunicorn.
Here's my output:
gunicorn -k gevent -w 4 'burpui:create_app(conf="/etc/burp/burpui.cfg")'
[2019-06-20 12:03:55 +0200] [5341] [INFO] Starting gunicorn 19.9.0
[2019-06-20 12:03:55 +0200] [5341] [INFO] Listening at: http://127.0.0.1:8000 (5341)
[2019-06-20 12:03:55 +0200] [5341] [INFO] Using worker: gevent
[2019-06-20 12:03:55 +0200] [5344] [INFO] Booting worker with pid: 5344
[2019-06-20 12:03:55 +0200] [5345] [INFO] Booting worker with pid: 5345
[2019-06-20 12:03:55 +0200] [5346] [INFO] Booting worker with pid: 5346
[2019-06-20 12:03:55 +0200] [5347] [INFO] Booting worker with pid: 5347
[2019-06-20 12:03:56 +0200] [5344] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/ggevent.py", line 203, in init_process
super(GeventWorker, self).init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 362, in import_app
app = eval(obj, vars(mod))
File "<string>", line 1, in <module>
File "/usr/local/lib/python3.6/site-packages/burpui/app.py", line 161, in create_app
if app.storage and app.storage.lower() == 'redis':
AttributeError: 'ProxyFix' object has no attribute 'storage'
[2019-06-20 12:03:56 +0200] [5344] [INFO] Worker exiting (pid: 5344)
[2019-06-20 12:03:56 +0200] [5345] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/ggevent.py", line 203, in init_process
super(GeventWorker, self).init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 362, in import_app
app = eval(obj, vars(mod))
File "<string>", line 1, in <module>
File "/usr/local/lib/python3.6/site-packages/burpui/app.py", line 161, in create_app
if app.storage and app.storage.lower() == 'redis':
AttributeError: 'ProxyFix' object has no attribute 'storage'
[2019-06-20 12:03:56 +0200] [5345] [INFO] Worker exiting (pid: 5345)
[2019-06-20 12:03:56 +0200] [5346] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/ggevent.py", line 203, in init_process
super(GeventWorker, self).init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 362, in import_app
app = eval(obj, vars(mod))
File "<string>", line 1, in <module>
File "/usr/local/lib/python3.6/site-packages/burpui/app.py", line 161, in create_app
if app.storage and app.storage.lower() == 'redis':
AttributeError: 'ProxyFix' object has no attribute 'storage'
[2019-06-20 12:03:56 +0200] [5346] [INFO] Worker exiting (pid: 5346)
[2019-06-20 12:03:56 +0200] [5347] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/ggevent.py", line 203, in init_process
super(GeventWorker, self).init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 362, in import_app
app = eval(obj, vars(mod))
File "<string>", line 1, in <module>
File "/usr/local/lib/python3.6/site-packages/burpui/app.py", line 161, in create_app
if app.storage and app.storage.lower() == 'redis':
AttributeError: 'ProxyFix' object has no attribute 'storage'
[2019-06-20 12:03:56 +0200] [5347] [INFO] Worker exiting (pid: 5347)
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 210, in run
self.sleep()
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 360, in sleep
ready = select.select([self.PIPE[0]], [], [], 1.0)
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 245, in handle_chld
self.reap_workers()
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 525, in reap_workers
raise HaltServer(reason, self.WORKER_BOOT_ERROR)
gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/bin/gunicorn", line 11, in <module>
sys.exit(run())
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 61, in run
WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 223, in run
super(Application, self).run()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 72, in run
Arbiter(self).run()
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 232, in run
self.halt(reason=inst.reason, exit_status=inst.exit_status)
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 345, in halt
self.stop()
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 393, in stop
time.sleep(0.1)
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 245, in handle_chld
self.reap_workers()
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 525, in reap_workers
raise HaltServer(reason, self.WORKER_BOOT_ERROR)
gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>
Running Celery confirms my problem:
juin 20 12:23:42 somehost.local systemd[1]: Started Burp-UI celery service.
juin 20 12:23:44 somehost.local bui-celery[5688]: Error:
juin 20 12:23:44 somehost.local bui-celery[5688]: Unable to load celery application.
juin 20 12:23:44 somehost.local bui-celery[5688]: 'proxyfix' object has no attribute 'storage'
juin 20 12:23:44 somehost.local systemd[1]: bui-celery.service: main process exited, code=exited, status=1/FAILURE
juin 20 12:23:44 somehost.local systemd[1]: Unit bui-celery.service entered failed state.
juin 20 12:23:44 somehost.local systemd[1]: bui-celery.service failed.
My burpui.cfg file:
[Global]
backend = multi
auth = basic
acl = basic
audit = basic
plugins = none
[UI]
refresh = 180
liverefresh = 5
ignore_labels = color:.*, custom:.*
format_labels = s/^os:\s*//
default_strip = 0
[Production]
storage = redis
session = redis
cache = redis
redis = localhost:6379
celery = true
database = sqlite:////var/lib/burpui/store.db
limiter = false
ratio = 60/minute
prefix = none
num_proxies = 0
proxy_fix_args = "{'x_for': {num_proxies}, 'x_host': {num_proxies}, 'x_prefix': {num_proxies}}"
[WebSocket]
enabled = true
embedded = false
broker = redis
url = none
debug = false
[Security]
includes = /etc/burp
enforce = false
revoke = true
cookietime = 1
sessiontime = 1
scookie = true
appsecret = somestring
[Experimental]
zip64 = false
noserverrestore = false
[Burp]
burpbin = /usr/sbin/burp
stripbin = /usr/sbin/vss_strip
bconfcli = /etc/burp/burp.conf
bconfsrv = /etc/burp/burp-server.conf
tmpdir = /tmp/bui
timeout = 15
deep_inspection = false
[Parallel]
host = ::1
port = 11111
timeout = 15
password = password123456
ssl = true
concurrency = 2
init_wait = 15
[BASIC:AUDIT]
priority = 100
level = WARNING
logfile = /var/log/burp-ui.log
max_bytes = 30 * 1024 * 1024
rotate = 5
[BASIC:AUTH]
priority = 100
admin = pbkdf2:sha256:Somestuff
extended = pbkdf2:sha256:Somestuff
assume_rw = pbkdf2:sha256:Somestuff
legacy = pbkdf2:sha256:Somestuff
[BASIC:ACL]
priority = 100
admin = admin
superadmin = '{"agents":{"agent1":{"rw":["client.*","server.*"]},"agent2":{"rw":["client.*","server.*"]}}}'
+moderator = "", superadmin
[Agent:agent1]
host = 127.0.0.1
port = 10000
password = somepass
ssl = false
timeout = 300
[Agent:agent2]
host = 127.0.0.1
port = 10001
password = someotherpass
ssl = false
timeout = 300
I noticed that the booleans of [ACL] get rewritten to passwords btw, tested that twice.
extended = pbkdf2:sha256:Somestuff
assume_rw = pbkdf2:sha256:Somestuff
legacy = pbkdf2:sha256:Somestuff
I've tried to comment out num_proxies and proxy_fix_args, but the result is the same.
System: CentOS 7.6 x64 uptodate with python3.6 (hence using /usr/local/bin instead of /usr/bin for most stuff)
Thanks